6. ZFS Installation

Note: as of 2009-07-16, this page is getting outdated, and since I have switched to OpenSolaris for an in-kernel ZFS implementation, I probably won’t be maintaining it. Please see http://rudd-o.com/new-projects/zfs for a current list of ZFS-Fuse resources.

In order to use the big_writes patch to ZFS (you’ll want that for speed), you need to install fuse-2.8.0-pre2 or later. Because Ubuntu doesn’t provide a fuse pre-release and I’m getting obsessive about integrating with built-in package management, I decided to build a full-fledged package.

Install an Updated Fuse

First install some prerequisites:

$ sudo aptitude install libselinux-dev autotools-dev devscripts debhelper dpatch

Shortcut: you can download my prepared package sources for fuse_2.8.0-pre2, unpack them withtar xvf fuse_2.8.0*.gz
cd fuse-2.8.0*/, and skip forward to the build step.

Prepare directories and get the ubuntu fuse-utils package source:

$ mkdir -p fuse/ubuntu

$ cd fuse/ubuntu

$ apt-get source fuse-utils

$ cd ..

Now visit http://fuse.sourceforge.net/ and download the fuse source, version 2.8.0-pre2 or later (as of this writing, you still need the prerelease) into the current directory.

Next, we’ll apply the same differences used to build the ubuntu package to the clean fuse sources:

$ tar xzf fuse-*.tar.gz

$ cd fuse-*/

$ zcat ../ubuntu/fuse_*.diff.gz | patch -p1

$ chmod +x debian/rules

Since the fuse kernel module has been stable for several releases, its source has been moved into the kernel tree, but the ubuntu package sources haven’t caught up yet. So w’ell just remove fuse-source from our build. Find and remove this stanza from debian/control:

Package: fuse-source
Architecture: all
Depends: module-assistant, debhelper (>= 5), make, bzip2
Description: Source for the Fuse kernel module
 Simple interface for userspace programs to export a virtual
 filesystem to the Linux kernel.
 .
 This package provides the source code for the Fuse kernel modules.
 Kernel source or headers >= 2.6.9 are required to compile these modules.

In debian/rules find and comment out the code that’s specific to module-assistant:

# # Module-assistant stuff
# PACKAGE = fuse-modules
# MA_DIR ?= /usr/share/modass
# -include $(MA_DIR)/include/generic.make
# -include $(MA_DIR)/include/common-rules.make

. Finally, comment out most of the install-indep target:

install-indep:
	dh_testdir
	dh_testroot
	dh_clean -k -i
# 	mkdir -p $(KDESTDIR)/usr/src/modules/fuse/debian
# 	cp -a kernel/* $(KDESTDIR)/usr/src/modules/fuse
# 	cp debian/control.modules.in $(KDESTDIR)/usr/src/modules/fuse/debian
# 	cp debian/postrm.modules.in $(KDESTDIR)/usr/src/modules/fuse/debian
# 	cp debian/preinst.modules.in $(KDESTDIR)/usr/src/modules/fuse/debian
# 	cp debian/rules debian/changelog debian/copyright debian/compat debian/control
# 	  $(KDESTDIR)/usr/src/modules/fuse/debian
# 	sed -i 's!^(include /usr/share/dpatch/dpatch.make)$$!#1!' $(KDESTDIR)/usr/src/modules/fuse/debian/rules
# 	cp --parents $(DOCS) $(KDESTDIR)/usr/src/modules/fuse/
# 	cp ChangeLog $(KDESTDIR)/usr/src/modules/fuse/
# 	cd $(KDESTDIR)/usr/src && tar c modules | bzip2 -9 > fuse.tar.bz2 && rm -rf modules

Locate the place in debian/zfs-fuse.init that invokes start-stop-daemon –start… and on the previous line, insert

ulimit -s unlimited

to limit virtual memory use per this thread.

Register the new version in debian/changelog:

dch -v 2.8.0-pre2-0techarcana0 'update source to 2.8.0-pre2'

Now build your package:

$ dpkg-buildpackage -rfakeroot -uc -us -D -j5

Note: the -jN option adds N-way parallelism to the build process. Tune according to your hardware.

The new packages will appear in the parent directory of the current one. You can install the binaries as follows:

$ sudo dpkg -i ../*.deb

Install ZFS-Fuse

We’re going to be basing our work on a “standard” zfs-fuse package developed by

Filip Brcic. Visit Filip’s ppa and select your distribution from the pulldown menu next to the text “Display sources.list entries for:.” Copy the lines that appear in the box below and append them to your /etc/apt/sources.list. For my Intrepid Ibex installation, the text was:

deb http://ppa.launchpad.net/brcha/ppa/ubuntu intrepid main

deb-src http://ppa.launchpad.net/brcha/ppa/ubuntu intrepid main

Install the source and prepare its build environment:

$ apt-get source zfs-fuse

$ cd zfs-fuse*/

$ sudo aptitude install scons libaio-dev libz-dev xsltproc docbook-xsl

$ sudo apt-get build-dep zfs-fuse

Shortcut:you can download and apply my prepared patch

$ cat ../zfs-fuse-opt.patch | patch -p1

$ chmod +x debian/rules

and skip forward to the build step.

Fix a problem with the order in which zfs-fuse is stopped at shutdown. Find the line in debian/rules that reads:

dh_installinit --no-start -- start 35 S . stop 39 0 6 .

and change it to

dh_installinit --no-start -- start 35 S . # stop 39 0 6 .
dh_installinit --init-script=stop-zfs-fuse --no-start -- start 39 0 6 .

Then add the following as debian/zfs-fuse.stop-zfs-fuse:

#! /bin/bash
### BEGIN INIT INFO
# Provides:          stop-zfs-fuse
# Required-Start:    fuse
# Required-Stop:
# Default-Start:     0 6
# Default-Stop:
# Short-Description: Stop the Daemon for ZFS support via FUSE
# Description:       See the init.d/zfs-fuse script.  Because of the strange way various shutdown scripts are installed as "start" actions in rc0.d and rc6.d, we need this to shut down zfs-fuse in the right order.  Sorry if this line is too long; I don't know the protocol  
### END INIT INFO

# Author: Dave Abrahams <dave@boostpro.com>

set -u # Error on uninitialized variabled
set -e # Error on uncaught non-zero exit codes

NAME=stop-zfs-fuse
DAEMON=/sbin/zfs-fuse

[ -x "$DAEMON" ] || exit 0

case "$1" in
  start)
	/etc/init.d/zfs-fuse stop
	;;
  stop|restart|force-reload)
	# No-op
	;;
  *)
	echo "Usage: $NAME {start|stop|restart|force-reload}" >&2
	exit 3
	;;
esac

:

Download the cached_entries+big_writes patch and save it as debian/patches/05_cachedentries_bigwrites.dpatch.

Make it executable and add it to the package’s list of patches against the official zfs-fuse source:

$ chmod +x debian/patches/05_cachedentries_bigwrites.dpatch

$ echo 05_cachedentries_bigwrites >> debian/patches/00list

Prepend this to the file:

#! /bin/sh /usr/share/dpatch/dpatch-run
## 05_cachedentries_bigwrites.dpatch by Dave Abrahams <dave@boostpro.com>
##
## All lines beginning with `## DP:' are a description of the patch.
## DP: These are the optimizations from Rudd-O

@DPATCH@

Optional: install the following patch that increases the ZFS vdev cache to make up for Fuse having a small one. Save it as debian/patches/06_vdev_cache_size.dpatch.

#! /bin/sh /usr/share/dpatch/dpatch-run
## 06_vdev_cache_size.dpatch by Dave Abrahams
##
## All lines beginning with `## DP:' are a description of the patch.
## DP: These are optimizations from Debabrata Banerjee .
## DP: Debabrata says "they help compensate for fuse. Its io size is
## DP: still too small by an order of magnitude"

@DPATCH@
diff -r 008c531499cd src/lib/libzpool/vdev_cache.c
--- a/src/lib/libzpool/vdev_cache.c	2009-02-11 12:46:21.618533220 -0500
+++ b/src/lib/libzpool/vdev_cache.c	2009-02-11 12:45:53.708533511 -0500
@@ -74,9 +74,9 @@
  * track buffer).  At most zfs_vdev_cache_size bytes will be kept in each
  * vdev's vdev_cache.
  */
-int zfs_vdev_cache_max = 1<<14;			/* 16KB */
-int zfs_vdev_cache_size = 10ULL << 20;		/* 10MB */
-int zfs_vdev_cache_bshift = 16;
+int zfs_vdev_cache_max = 1<<16;			/* 64KB */
+int zfs_vdev_cache_size = 30ULL << 20;		/* 30MB */
+int zfs_vdev_cache_bshift = 17;

 #define	VCBS (1 << zfs_vdev_cache_bshift)	/* 64KB */

Make it executable and add it to the package’s list of patches against the official zfs-fuse source:

$ chmod +x debian/patches/06_vdev_cache_size.dpatch
$ echo 06_vdev_cache_size >> debian/patches/00list

, bump the package version number, build, and install the packages:

$ dch -n 'version bump to account for optimization patches'
$ dpkg-buildpackage -rfakeroot -uc -us -D -j5
$ sudo dpkg -i ../zfs-fuse*.deb