17.5 Diskless Operation

Updated by Jean-François Dockès.

A FreeBSD machine can boot over the network and operate without a local disk, using file systems mounted from an NFS server. No system modification is necessary, beyond standard configuration files. Such a system is easy to set up because all the necessary elements are readily available:

There are many ways to set up diskless workstations. Many elements are involved, and most can be customized to suit local taste. The following will describe the setup of a complete system, emphasizing simplicity and compatibility with the standard FreeBSD startup scripts. The system described has the following characteristics:

Caution: As described, this system is insecure. It should live in a protected area of a network, and be untrusted by other hosts.

17.5.1 Setup Instructions

17.5.1.1 Configuring DHCP/BOOTP

There are two protocols that are commonly used to boot a workstation that retrieves its configuration over the network: BOOTP and DHCP. They are used at several points in the workstation bootstrap:

  • etherboot uses DHCP (by default) or BOOTP (needs a configuration option) to find the kernel. (PXE uses DHCP).

  • The kernel uses BOOTP to locate the NFS root.

It is possible to configure a system to use only BOOTP. The bootpd(8) server program is included in the base FreeBSD system.

However, DHCP has a number of advantages over BOOTP (nicer configuration files, possibility of using PXE, plus many others not directly related to diskless operation), and we shall describe both a pure BOOTP, and a BOOTP+DHCP configuration, with an emphasis on the latter, which will use the ISC DHCP software package.

17.5.1.1.1 Configuration Using ISC DHCP

The isc-dhcp server can answer both BOOTP and DHCP requests.

As of release 4.4, isc-dhcp is not part of the base system. You will first need to install the /usr/ports/net/isc-dhcp3 port or the corresponding package. Please refer to Chapter 4 for general information about ports and packages.

Once isc-dhcp is installed, it needs a configuration file to run, (normally named /usr/local/etc/dhcpd.conf). Here follows a commented example:

              default-lease-time 600;
              max-lease-time 7200;
              authoritative;
    
              option domain-name "example.com";
              option domain-name-servers 192.168.4.1;
              option routers 192.168.4.1;
    
              subnet 192.168.4.0 netmask 255.255.255.0 {
                use-host-decl-names on; (1)
                option subnet-mask 255.255.255.0;
                option broadcast-address 192.168.4.255;
    
                host margaux {
                  hardware ethernet 01:23:45:67:89:ab;
                  fixed-address margaux.example.com;
                  next-server 192.168.4.4;(2)
                  filename "/tftpboot/kernel.diskless";(3)
                  option root-path "192.168.4.4:/data/misc/diskless";(4)
                }
              }
             
(1)
This option tells dhcpd to send the value in the host declarations as the hostname for the diskless host. An alternate way would be to add an option host-name margaux inside the host declarations.
(2)
The next-server directive designates the TFTP server (the default is to use the same host as the DHCP server).
(3)
The filename directive defines the file that etherboot will load as a kernel.

Note: PXE appears to prefer a relative file name, and it loads pxeboot, not the kernel (option filename "pxeboot").

(4)
The root-path option defines the path to the root filesystem, in usual NFS notation

17.5.1.1.2 Configuration Using BOOTP

Here follows an equivalent bootpd configuration. This would be found in /etc/bootptab.

Please note that etherboot must be compiled with the non-default option NO_DHCP_SUPPORT in order to use BOOTP, and that PXE needs DHCP. The only obvious advantage of bootpd is that it exists in the base system.

              .def100:\
                :hn:ht=1:sa=192.168.4.4:vm=rfc1048:\
                :sm=255.255.255.0:\
                :ds=192.168.4.1:\
                :gw=192.168.4.1:\
                :hd="/tftpboot":\
                :bf="/kernel.diskless":\
                :rp="192.168.4.4:/data/misc/diskless":
    
              margaux:ha=0123456789ab:tc=.def100
             

17.5.1.2 Preparing a Boot Program with Etherboot

Etherboot's Web site contains extensive documentation mainly intended for Linux systems, but nonetheless containing useful information. The following will just outline how you would use etherboot on a FreeBSD system.

You must first install - and possibly compile - the etherboot package. The etherboot port can normally be found in /usr/ports/net/etherboot. If the ports tree is installed on your system, just typing make in this directory should take care of everything. Else refer to Chapter 4 for information about ports and packages.

For our setup, we shall use a boot floppy. For other methods (PROM, or dos program), please refer to the etherboot documentation.

To make a boot floppy, insert a floppy in the drive on the machine where you installed etherboot, then change your current directory to the src directory in the etherboot tree and type:

         # gmake bin32/devicetype.fd0
       

devicetype depends on the type of the Ethernet card in the diskless workstation. Refer to the NIC file in the same directory to determine the right devicetype.

17.5.1.3 Configuring the TFTP and NFS Servers

You need to enable tftpd on the TFTP server:

  1. Create a directory from which tftpd will serve the files, ie: /tftpboot

  2. Add this line to your /etc/inetd.conf:

        tftp    dgram   udp     wait    nobody  /usr/libexec/tftpd    tftpd /tftpboot
    

    Note: It appears that at least some PXE versions wants the TCP version of TFTP. In this case, add a second line, replacing dgram udp with stream tcp

  3. Tell inetd to reread its configuration file:

        # kill -HUP `cat /var/run/inetd.pid`
    

You can place the tftpboot directory anywhere on the server, but, of course, the actual location, the value in inetd.conf, and the value in dhcpd.conf must be consistent !

You also need to enable NFS service and export the appropriate filesystem on the NFS server

  1. Add this to /etc/rc.conf:

        nfs_server_enable="YES"
    
  2. Export the filesystem where the diskless root directory is located by adding the following to /etc/exports (adjust the volume mount point and workstation name!):

        /data/misc -alldirs -ro margaux
    
  3. Tell mountd to reread its configuration file. If you actually needed to configure NFS service at step 1, you probably want to reboot instead.

        # kill -HUP `cat /var/run/mountd.pid`
    

17.5.1.4 Building a Diskless Kernel

Create a kernel configuration file for the diskless client with the following options (in addition to the usual ones):

              options     BOOTP          # Use BOOTP to obtain IP address/hostname
              options     BOOTP_NFSROOT  # NFS mount root filesystem using BOOTP info
              options     BOOTP_COMPAT   # Workaround for broken bootp daemons.
       

You may also want to use BOOTP_NFSV3 and BOOTP_WIRED_TO (refer to LINT).

Build the kernel (See Chapter 9), and copy it to the tftp directory, under the name listed in dhcpd.conf

17.5.1.5 Preparing the root Filesystem

You need to create a root filesystem for the diskless workstations, in the location listed as root-path in dhcpd.conf.

The easiest way to do this is to use the /usr/share/examples/diskless/clone_root shell script. This script needs customization, at least to adjust the place where the filesystem will be created (the DEST variable).

Refer to the comments at the top of the script for instructions. They explain how the base filesystem is built, and how files may be selectively overridden by versions specific to diskless operation, to a subnetwork, or to an individual workstation. They also give examples for the diskless /etc/fstab and /etc/rc.conf

The README files in /usr/share/examples/diskless contain a lot of interesting background information, but, together with the other examples in the diskless directory, they actually document a configuration method which is distinct from the one used by clone_root and /etc/rc.diskless[12], which is a little confusing. Use them for reference only, except if you prefer the method that they describe, in which case you will need customized rc scripts

As of FreeBSD version 4.4-RELEASE, there is a small incompatibility between the clone_root script and the /etc/rc.diskless1 script. Please refer to PR conf/31200 for the small adjustment needed in clone_root. Also see PR conf/29870 about a small adjustment needed in /etc/rc.diskless2.

17.5.1.6 Configuring Swap

If needed, it is possible to do swapping over NFS, to a file on the server. The exact bootptab or dhcpd.conf options are a little mysterious and poorly documented. Anyway, here is what worked for me, using isc-dhcp 3.0rc11.

  1. Add the following lines to dhcpd.conf:

                      # Global section
                      option swap-path code 128 = string;
                      option swap-size code 129 = integer 32;
        
                      host margaux {
                        ... # Standard lines, see above
                        option swap-path "192.168.4.4:/netswapvolume/netswap";
                        option swap-size 64000;
                      }
               
    

    The idea is that, at least for a FreeBSD client, DHCP/BOOTP option code 128 is the path to the NFS swap file, and option code 129 is the swap size in kilobytes. Older versions of dhcpd allowed a syntax of option option-128 "..., which does not seem to work any more.

    /etc/bootptab would use the following syntax instead:

    T128="192.168.4.4:/netswapvolume/netswap":T129=64000

  2. On the NFS swap file server, create the swap file(s)

                      # mkdir /netswapvolume/netswap
                      # cd /netswapvolume/netswap
                      # dd if=/dev/zero bs=1024 count=64000 of=swap.192.168.4.6
                      # chmod 0600 swap.192.168.4.6
                   
    

    192.168.4.6 is the IP address for the diskless client

  3. On the NFS swap file server, add the following line to /etc/exports

                 /netswapvolume  -maproot=0:10 -alldirs margaux 
               
    

    Then tell mountd to reread the exports file, as above.

17.5.1.7 Miscellaneous Issues

17.5.1.7.1 Running with a read-only /usr

If the diskless workstation is configured to run X, you will have to adjust the xdm configuration file, which puts the error log on /usr by default.

17.5.1.7.2 Using a non-FreeBSD Server

When the server for the root filesystem is not running FreeBSD, you will have to create the root file system on a FreeBSD machine, then copy it to its destination, using tar or cpio.

In this situation, it seems that there are sometimes problems with the special files in /dev, because of differing minor/major number integer sizes. It seems that a solution to this problem is to run MAKEDEV on a FreeBSD machine, in a directory mounted through NFS from the final server.

This, and other documents, can be downloaded from ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/.

For questions about FreeBSD, read the documentation before contacting <questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.