Design notes for a home firewall/fileserver

Goals

low power consumption, especially when idle
runs 24/7 for months or years, requiring little maintenance
fileserver for other local computers, for media files and backup
very secure firewall between a household LAN and the Internet
webserver for low-volume external access
NAT, DHCP, mail service
silent operation

The end result looks remarkably similar to a standard Intel-based PC. In fact, you can achieve almost all of these goals with a standard PC. My goals for low-power operation drove me to a configuration that is a bit off the beaten path.

The machine I built runs OpenBSD, largely due to the strong security orientation of the developers.

If you have a subset of these goals, there are better alternatives to the PC architecture. For example, if you don't need a fileserver or mail server, a very low-power system can be configured with any of the popular low-cost routers running user-supported software packages such as OpenWRT.

But if you want to serve files, there are few alternatives to the standard PC platform. There are fileservers which run webservers (such as the ReadyNAS), but such machines are not usually configured as firewalls. At one time, the Cobalt Qube was almost an ideal system, but that machine has limited disk-storage options.

In the end, I built a machine customized for my needs. Here is a rough list of the ingredients, assembled in Spring of 2004:

Raw Materials

mini-ITX motherboard (Via PD6000, which is no longer sold. The current alternative is the EK8000EG). This motherboard has two LAN adaptors onboard, essential for firewall operation. The CPU is fanless and unless it is mounted in a constrained space, no active cooling is necessary.
single RAM DIMM module with 1 gigabyte of RAM
2 gigabyte CompactFlash storage device, for booting and root filesystem. A CompactFlash-to-IDE adaptor is necessary (and available via EBay).
small ATX power supply (mine is rated for 150 watts).
two large hard-drives (mine are 200GB, but much larger sizes are available now).
OpenBSD installation media.

Please note that this system is far more powerful than necessary for simple webserving, NAT, DHCP, and fileserver activity. At one point, I had used a Pentium-based machine running at 60 MHz to do just these activities. But systems with such limited CPU capabilities are not usually available in a standard PC format nowadays.

Once I set up this system, and after a fair amount of software installation and customization, it acts as follows:

boots from the compact-flash drive
creates a RAM disk for volatile files (primarily logs)
sets up NAT, DHCP, webserver, and mail-serving daemons
mounts the hard drives and sets them to spin-down after 10 minutes of inactivity

Unless the system is actively serving files to another computer in the house, it is totally silent and consumes minimal power. The drives do spin up when files are needed, but in practice that happens infrequently. Users typically have their home directories on laptops, not on the fileserver. Mail is stored on the fileserver, but user mailboxes are kept on a CompactFlash drive.

Since CompactFlash media is subject to failure, the fileserver regularly makes multiple cascading backups.

Preparation and Assembly

Here is a rough guide to the installation of this system:

Set up the motherboard with the 1GB RAM DIMM and the CompactFlash device connected as the primary device on the first IDE controller. Attach a CDROM drive as the secondary storage device on the second IDE controller, or attach a floppy drive. The CDROM (or floppy) is only used for the initial software installation.
If you can use USB flash devices as the boot device, please consider doing so. The cost-per-gigabyte of USB drives is often better than that of the CompactFlash media. Please note that I attempted to use a USB flash disk and found that the USB bus would lockup about twice per month, forcing me to do a complete power-down reset in order to clear the bottleneck. I have no idea if this is an OpenBSD problem or a hardware fault with my motherboard. For the record, my OpenBSD dmesg entry for the USB device reads:
```
uhci0 at pci0 dev 16 function 0 "VIA VT83C572 USB" rev 0x80: irq 11
usb0 at uhci0: USB revision 1.0
```
Do a full installation of OpenBSD, using either CDROM or floppy install media. Go through the traditional network configuration. Use the CompactFlash drive as the root filesystem.
After you have confirmed that OpenBSD successfully boots from the CompactFlash drive, remove the floppy or CDROM drives.
Configure the motherboard BIOS so that all attached IDE drives will run in "low-power" mode. On the VIA PD6000, this means that the drives to automatically shut down after a certain period of inactivity.
In /etc/fstab, modify the mount-options field for each CompactFlash partition. Add the specifiers "softdep,noatime". These options are very important for filesystems running on limited-write-cycle media. "Softdep" causes the kernel to cluster write requests and issue them to the drive in a way that minimizes the number of write operations yet preserves filesystem consistency. "Noatime" means that the filesystem inodes are *not* updated when files are read from the drive (this eliminates extra write operations to the filesystem).
Create a RAM disk to hold frequently-modified files. In my case, I configured a RAM disk as the /var filesystem, which holds log files and certain info files used frequently.
To do this, I created a new file /etc/varstart with the following contents:
```
        mount_mfs -s 98304 -i 512 swap /var/
        (cd / ; tar pxzf /var.tgz )
```
The first command creates a memory disk of approximately 40 megabytes. The second command take a file called "var.tgz" from the root filesystem and unloads its contents into the newly-created /var/ filesystem.
Before /etc/varstart is run, you will have to create /var.tgz. This can be done initially with the command:
```
       tar czf /var.tgz /var
```
I also put that command into my rc.shutdown script, so that the var.tgz file is "freshened" with the last contents of /var/ when the machine is shutdown. Also, I execute that command whenever I manually change the contents of the /var/ hierarchy.
The /etc/varstart script must be executed during the system startup before any of the services which require /var/ are running. If varstart is executed too late, running services will be attached to the files in the original disk-resident version of /var/, which is undesirable. On OpenBSD, the varstart script is run within /etc/rc, replacing the section which mounts /var/.

After you've done all of these steps, you have an OpenBSD system which will boot from CompactFlash and all volatile files (such as system logs) are stored in RAM disk.

This is a good starting point, and if you don't need to run a fileserver, you can stop here (after tweaking the networking parameters so that NAT, DHCP, etc are running).

To attach the hard drives, shutdown the system. Connect the drives as the primary and secondary drives on the second IDE connector. It is crucial that the CompactFlash drive does *not* change its position as the primary drive on the first IDE connector: if you move it, your system will no longer boot from that drive and you will have to start over.

Formatting, configuring, and mounting the hard drives aren't particularly challenging and the necessary steps are covered in the OpenBSD documentation. You will want to run SMB and/or NFS so that your home machines can access the fileserver.

Customizations

Here are some customizations that you may desire:

Set up the "pf" packet-filtering system, so that services such as NFS, SMB, smtp, sunrpc, ftp etc cannot be accessed by random hosts on the Internet. You can also set up application-level filters (via /etc/hosts.deny and /etc/hosts.allow) for other forms of control. My system restricts incoming SSH connections to a small set of hosts.
Add a small USB storage drive. On my system, this is device /dev/sd0a, and is mounted as /home/. On this filesystem, each user has their Maildir directory and little else.
Note: it may be possible to boot directly from the USB storage drive. If so, please do this: it will free up an IDE position for additional hard-drive storage. However, if your motherboard supports SATA, conservation of IDE controllers may not be important.
Mount /tmp, /usr/ports on hard-drive partitions. These directories get used frequently and should not be mounted on the CompactFlash filesystem. Further, /var/tmp/ seems to be used as the location for the weekly updatedb operation (check /etc/weekly for details). If /var/tmp/ is part of the RAM filesystem created earlier, it may not have sufficient size to run the updatedb operation, and you'll want to use a larger disk-resident partition.
On a nightly basis, my system copies all of the /var/ contents to a backup file on a disk-based filesystem. This is done to keep all of the logs and other files in a safe location. If there is an unplanned powerdown or reset, at least there is a copy of the RAM drive that is less than 24 hours old. In such cases, I'll copy the disk backup of /var/ atop the /var.tgz file mentioned above
My installation uses qmail as the MTA. qmail conveniently uses directories in /var/ for email spooling, which are on the RAM drive. The qmail documentation very carefully specifies how to create robust filesystems for mail spooling: since my system isn't mission-critical, it is an acceptable tradeoff.
Do a web-search for the "life with qmail" page for excellent documentation.
apache is the default webserver in OpenBSD, and it conveniently reads config files and webserver content from /var/. Since my system has a small number of tiny files that are used by the webserver, /var/www/ is part of the RAM filesystem. Thus, the drives do not need to startup whenever my webserver is active.
The hard drives contain multiple backup partitions, so that all data is replicated at least once. My system runs nightly cron jobs that invoke rsync to keep the backup copies synchronized.
Connect a home UPS device. This has saved me from several unplanned power outages. Sofware exists which will cause the system to safely shutdown when the battery runs low.

Alternatives

The following principles guided me to these choices:

dual network adaptors
noiseless (low heat production, no fans necessary)
standard PC form factor

There are other ways to do this. For example, I could have used an old laptop (generally low power, and built-in battery backup). But laptops, especially the older models, typically don't have much disk capacity. Older BIOS systems may not be able to boot from USB flash drives. Such systems typically have limited memory: a K2 laptop from 1999 might have only 256MB of RAM.

$Header: /home/cvs/htdocs/stan/homesys.html,v 1.2 2007/06/13 18:21:16 chesnutt Exp $