IPv6 address facts in older versions of facter

Facter is a companion tool to the puppet configuration management system that allows you to retrieve metadata about the system puppet is running on. This is then made available to puppet for use in config file templates or making informed decisions on how to go about configuring a system.

One thing I noticed when going through the list of variables output by facter was that no IPv6 address information was being reported. This was a bit of a roadblock as the majority of our systems are dual-stacked (IPv4 and IPv6), with many configuration files needing to know what IPv6 addresses to listen on. Whilst an ipaddress6 fact was added to facter 1.59, I’m working with a somewhat older version of Puppet (0.25) and Facter (1.56), these both ship with Ubuntu 10.04 LTS so I’d rather stick with them if at all possible, rather than resorting to a backports repository.

Luckily, the puppet guide provided a quick solution – custom facts. It was a simple matter of grabbing the relevant code from github and dropping it into a directory in the $FACTERLIB path. Running the facter command will now print out an ipaddress6 variable:

 $ facter ipaddress6
 2001:6f8:900:909::2

Of course, this only makes the fact available to the environment you are testing on. If you are running a puppetmaster server, you’ll need to follow the guide to distributing plugins as modules if you want to distribute the fact around your network. This can be done on a per-module basis, or globally as a top-scope variable by creating a skeleton module with a blank init.pp and the ipaddress6.rb file in the lib/facter directory.

To test if the variable is available, add the following to your site.pp manifest on the puppetmaster server:

 class puppet_v6_marker {
      file {
            '/tmp/puppet-v6':
            ensure => present,
            mode   => 600,
            owner  => root,
            content => $::ipaddress6,
            group  => root
      }
}

include puppet_v6_marker

Your puppet clients should now output their IPv6 address to /tmp/puppet-v6.

IPv6 addresses not available at boot

One of things that’s always bugged me regarding Linux and IPv6 is the
behaviour that’s exhibited during boot time. Specifically, the short delay before IPv6 addresses transition from their “tentative” state on an interface to being fully available for use by various daemons and services. With IPv4, you can be pretty much guaranteed that you can bind to any of the configured addresses at boot time, under normal circumstances.

With IPv6 on Linux, things aren’t so straightforward. Duplicate Address Detection (DAD), which basically does what it says on the tin, introduces a short delay before addresses are fully configured, the address has been added to the network interface, but not really.

I recently came across this whilst attempting to get BIND to listen on some secondary service addresses on a particular machine. BIND would not er, bind, to the IPv6 addresses at boot, failing with messages like this:

bind9 could not listen on UDP socket: address not available

Modifying /etc/init.d/bind9 to print the output of “ip addr show” to a file at the time BIND attempted to start up showed the tell-tale “tentative” flag on each IPv6 address being added to eth0. Since the addresses are in this state, BIND or other daemons will refuse to listen on them.

The problem has become very noticeable since parallel boot systems such as Upstart have become the default in quite a few Linux distros. Daemons will often fire up before the network is fully ready and in some extreme cases network filesystems that reference hostnames may fail to mount if you are using an IPv6 DNS resolver. Of course this isn’t the case across the board, some daemons and services appear to handle the unavailability of an IPv6 address somewhat gracefully, backing off and trying again a short time later rather than simply giving up on the first go.

Anyway, a simple “hairy hack” to get over this problem is to add something like the following to your startup script:

sleep 5

Yes, a one line sleep command to make the daemon wait a short while before actually starting. This seems to ensure that the IPv6 address has moved out of the tentative state, but it’s still somewhat silly..

SpamAssassin DNS lookups over IPv6

With the recent surge in AIB phishing mails, I thought it might be worth looking at our SpamAssassin setup to see if there was anything that could be done to filter them out.

AIB helpfully publish SPF records for the aib.ie domain, so the first port of call was SAs SPF lookups. Which I noticed weren’t happening, despite being enabled some time ago. Reloading SA resulting in the following log message in spamd.log:

Tue Aug 11 11:26:52 2009 [2221] warn: Error creating a DNS resolver
socket: at /usr/share/perl5/Mail/SpamAssassin/DnsResolver.pm line 233.

Aw snap, SA can’t do any DNS lookups, at all. Which means all of the DNS-based tests will be silently skipped. Going back through the spamd.log, I noticed these messages first started occurring when we enabled IPv6 DNS resolvers a number of months ago. Hmm, perl is obviously missing something fundamental.

So, it turns out a perl library required to create IPv6 sockets wasn’t installed, IO::Socket::INET6. This is conveniently packaged in Ubuntu:

aptitude install libio-socket-inet6-perl

A quick reload of spamassassin and we can say goodbye to all those “URGENT NOTIFICATION”s about our AIB online banking accounts. Although, the spammers have now copped on and aren’t even bothering to send from aib.ie addresses anymore..

Heartbeat and IPv6

I’ve been playing around with heartbeat quite a bit recently. It’s quite a mature piece of software with some cool features, however the documentation is bit unstructured and lacking in some areas, especially where IPv6 support is concerned. Heartbeat *supports* IPv6 address takeover through the IPv6addr resource but it’s not exactly clear on how you go about setting it up.

Anyway, after reading through some forum posts, here is a sample haresources file that will give you IPv4 and IPv6 address failover. We are using eth1 as our primary network interface:

my.failover.address 193.1.219.93
my.failover.address IPv6addr::2001:770:18:2:0:0:c101:db5d/64/eth1

The important thing is to specify the IPv6 address fully, ie: pad it out with 0’s, no :: shortcuts!

With v6 support recently added to IPVS, it should now be possible to do full IPv6 failover and load balancing.