The last time I tested software RAID 1 under NetBSD I wasn’t really satisfied with the performance. Especially file system operations were much slower than I expected. Resulting from that my current home server uses an Intel SRCS14L RAID Controller instead of software RAID.
I had considered also buying a new RAID controller for my future home server. Areca’s ARC-1210 looked like the best candidate. But the ARC-1210 is not exactly cheap (as expected) and unfortunately has a fan. As reducing the noise level was one of the main goals to buy new server hardware I was put off.
After some consideration I decided to give RAIDframe another try. Encouraged by David Brownlee’s benchmark results I ran a similar benchmark on my hardware. To do that I wrote a shell script which consecutively creates RAID 1 volumes and FFSv1 file systems with all the combinations of parameters listed in the table below. Each of the file systems is mounted (not using soft dependences) and benchmarked by extracting the tar archives with the NetBSD 4.0 sources to it. The time required to complete each of the benchmark runs is recorded afterwards. Below are the averages of the results of two runs of my test script, smaller time values imply better performance of course.
|Block size / Fragment size
||32 sectors per stripe unit
||64 sectors per stripe unit
||128 sectors per stripe unit
|8KB / 1KB
|8KB / 2KB
|8KB / 4KB
|8KB / 8KB
|16KB / 2KB
|16KB / 4KB
|16KB / 8KB
|16KB / 16KB
|32KB / 4KB
|32KB / 8KB
|32KB / 16KB
|32KB / 32KB
Unsurprisingly I’m now using a RAID 1 volume with 128 sectors per stripe unit and file systems with a block size of 16KB and a fragment size of 2KB. I’m not sure whether these values always result in the best possible performance but they seem to work for David Brownlee and me.
David Holland organised NetBSD’s Hackathon 11 at the last weekend. The goal was to fix bugs in preparation for the NetBSD 5.0 release.
I decided that this was the best opportunity to fix PR kern/38100 which had rendered my ULTRA60 useless the past few month. I started with trying to reproduce the problem without using the automounter daemon but couldn’t cause a similar kernel panic. Fortunately Michael van Elst provided the crucial clue: the panics were related to NFS’s directory cookie translation option. If I manually mounted a filesystem using the -X option of mount_nfs(8) the attempt to unmount it afterwards would trigger a kernel panic. With futher help from Michael van Elst I came up with a fix. Using a kernel with this change my ULTRA60 survived mounting and unmounting filesystems via the automounter in rapid succession for an hour without problems.
Because I had plenty of time left I also ported a change from FreeBSD to fix PR bin/38584. I tried to fix further cases of bad handling of set -e in /bin/sh but that turned out to be less successful. In the end I gave up on these changes because the spec is somewhat debatable.
In the past NetBSD has been reported to deliver poor I/O performance when running under the Xen hypervisor. Because I’m considering running Xen on my home server in future I ran some benchmarks under NetBSD/xen myself recently.
I used my HP Proliant ML110 G4 for the benchmarks. The machine has an Intel Xeon 3040 CPU (Dual Core 1,87GHz), 5GB of DDR2 memory and a single 160GB SATA hard disk. I used the new 64Bit Xen support in NetBSD-current for booting NetBSD/xen because the machine normally runs NetBSD/amd64.
The two benchmarks consisted of:
- Extracting the NetBSD 4.0 source tar archives from a memory filesystem to an empty filesystem on the hard disk.
- Copying 32GB from /dev/zero to file on a (non empty) filesystem on the hard disk.
I’ve used an unmodified GENERIC kernel under NetBSD/amd64 and the XEN3_DOM0 kernel under NetBSD/xen. Both kernels were built from NetBSD 4.99.63 sources. The NetBSD/amd64 kernel could utilise both CPU cores and the full 5GB of main memory. Because NetBSD/xen doesn’t support SMP and cannot handle 5GB at the moment (see PR port-xen/38699) the NetBSD/xen kernel could only use one CPU core and 3.5GB. Under these conditions I expected NetBSD/amd64 to win by a mile. But I was in for a surprise:
As you can see the NetBSD/xen kernel actually provided slightly better I/O performance. The benchmarks are I/O bound and the NetBSD/amd64 could apparently not benefit from the second CPU core. My guess is that it was instead slowed down by the locking overhead of the SMP kernel.
The good news is however that NetBSD/xen delivers good I/O performance. Michael Graff had similar encouraging results when he compared VMware Server and Xen.