ZFS

Configuration Procedure

  1. Boot from FreeBSD 9.x Installation DVD or Memstick.
  2. Open Shell 
    <Install> → <Yes> → <Keyboard Setting> → <Distribution Select> → <Shell>
  3. Enable Write to /tmp 
    umount -f /dev/md1
mdmfs -s 512M md1 /tmp
  4. Listing devices 
    dmesg | grep MB
    In following procedure, device names (e.g. ada0, ada1, ...) are denoted as deviceX.
  5. Create the necessary partitions on the disk(s) and add ZFS aware boot code.

    For every devices, deviceX

    • Initialize disk partitioning 
      gpart destroy -F deviceX
    • Configure disk partitioning as gpt 
      gpart create -s gpt deviceX
    • Create boot block 
      gpart add -s 128 -t freebsd-boot deviceX
    • Create partition for ZFS, labeled with diskX (e.g. disk0, disk1, ...) 
      gpart add -t freebsd-zfs -l diskX deviceX
    • Writing bootcode into boot block 
      gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 deviceX
  6. Create the pool.

    ZFS pool name: zroot

    zpool create zroot raidz1 /dev/gpt/diskX /dev/gpt/diskX ...
    (ignore any warnings regarding mounting)
  7. Set bootfs property, checksums and mountpoints. 
    zpool set bootfs=zroot zroot
zfs set checksum=fletcher4 zroot
zfs set mountpoint=/mnt zroot

    checksum: fletcher4 (on) : fletcher2 : sha256 : off

  8. At this point export and import the pool while preserving zroot.cache in /var/tmp. 
    zpool export zroot
zpool import -o cachefile=/var/tmp/zpool.cache zroot
  9. Create appropriate filesystems (feel free to improvise!). 
    zfs create zroot/usr
zfs create zroot/usr/home
zfs create zroot/var
zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp
zfs create -o compression=lzjb -o setuid=off zroot/usr/ports
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp

    lzjb (on) … for random read
    (high compression ratio) gzip-9  >  gzip-6 (gzip)  >  gzip-1  >  off (high speed) … for sequential read

    If you have enough storage:

    zfs create zroot/usr
zfs create zroot/usr/home
zfs create zroot/var
zfs create -o exec=on -o setuid=off zroot/tmp
zfs create -o setuid=off zroot/usr/ports
zfs create -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o exec=off -o setuid=off zroot/usr/src
zfs create -o compression=lz4 -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o compression=lz4 -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o compression=lz4 -o exec=on -o setuid=off zroot/var/tmp
  10. Add swap space and disable checksums. 
    zfs create -V 32G zroot/swap
zfs set org.freebsd:swap=on zroot/swap
zfs set checksum=off zroot/swap
  11. Create a symlink to /home and fix some permissions. 
    chmod 1777 /mnt/tmp
cd /mnt ; ln -s usr/home home
chmod 1777 /mnt/var/tmp
  12. Install FreeBSD. 
    sh
cd /usr/freebsd-dist
export DESTDIR=/mnt
for file in base.txz lib32.txz kernel.txz doc.txz ports.txz src.txz;
do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done
  13. Copy zpool.cache (very important!!!) 
    cp /var/tmp/zpool.cache /mnt/boot/zfs/zpool.cache
  14. Create the rc.conf, loader.conf and an empty fstab (otherwise the system will complain). 
    echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf
echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf
touch /mnt/etc/fstab
  15. Unmount everything and fix mountpoints for system boot. 
    zfs set readonly=on zroot/var/empty
zfs umount -af
zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/var zroot/var
  16. Reboot
  17. Invoke sysinstall (adjust time zone info, add a password for root, and other settings.)

Add/Remove cache

  • cache disk device: cachedev (e.g. ada0)
  • Add cache disk device to pool 
    zpool add zroot cache cachedev
  • Remove cache disk device from pool 
    zpool remove zroot cachedev

Replacing hardware disk device

  • old disk device: odev (e.g. ada0)
  • new disk device: ndev (e.g. ada4), 2nd partition is labeled with diskN
  1. Setup new disk device 
    gpart destroy -F ndev
gpart create -s gpt ndev
gpart add -s 128 -t freebsd-boot ndev
gpart add -t freebsd-zfs -l diskN ndev
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ndev
  2. Replace new disk device with old disk device 
    zpool replace zroot odevp2 ndevp2