My filesystem of choice for homelab purposes is ZFS. I tend to revisit certain configuration options every now and then, usually when working on my storage pools. This document aggregates all of that information.
Configuration
Alignment shift
The alignment shift (ashift) communicates the physical sector size of the underlying storage device (e.g. HDD, SSD), to ZFS. This value represents the number of bits — the exponent in a power of two. For older disks, this is usually 9 (512B), while for modern SSDs it’s 12 (4K) or even 13 (8K).
Important considerations for choosing the ashift value:
- It is immutable and configurable only when adding a new device (disk) to a pool (vdev). This includes zpool creation, as
zpool createrequires providing a new device. - ZFS tries to determine the proper
ashiftvalue by checking the disk’s sector size. You should however, set it manually, as disks sometimes report a smaller sector size than they actually use internally (e.g. 512B, while actually using 4K). The purpose of mis-reporting is to maintain backward compatibility with older operating systems, as these might not support larger (more modern) sector sizes. - It’s better to overshoot, than to undershoot. Choosing an oversized
ashiftis generally fine, while an undersized will cause performance issues, due to a phenomenon called read/write amplification. Simply put, ZFS will have to perform many reads/writes to the disk, for every single ZFS block that’s being written/read. This drastically decreases the performance of the pool.
You can check your disk’s sector size via fdisk:
)
)
Treat the output as a suggestion, as that’s what ZFS would check to determine the default ashift. The best approach however, is to double check with your disk supplier’s technical specification.
Set ashift during pool creation as follows:
Set ashift when adding devices:
Record size
This is a fairly complex topic, but the rule of thumb is:
| Purpose | recordsize |
|---|---|
| Database | 64KiB |
| VM disk image | 64KiB |
| Multi-MiB media files | 1M |
| Mixed-use (small and large files) | 128KiB |
If unset, ZFS will asume a default recordsize of 128KiB.
Access time
When atime is enabled (on), file access timestamps are written whenever a file is accessed. This increases the operational overhead and wears the disk out quicker. Unless you absolutely need to keep track of access times, disable this option:
Linux extended attributes
This section is TODO
TL;DR
Compression
Consider your dataset’s compressibility and enable compression if you can reasonably expect space savings from enabling dataset compression.
What compresses well:
- plain text files,
- uncompressed image files like PNGs or GIFs,
- spreadsheets,
- compressed files appearing repeatedly.
What does not compress well:
- anything that’s already compressed:
- media (movies, photos, music),
- ZIP/BZ2/LZMA/RAR archives.
Leaving compression enabled, on a filesystem dedicated to storing already compressed files, will mostly just waste CPU cycles, while providing little to no benefit.
LZ4 is a good all-around compression algorithm, and ever since 2020 it’s the new default in ZFS (when on). If you’re wondering about optimising the choice of algorithm for your specific workload, you’ll be better off looking at the official OpenZFS docs, they link benchmarks!
# Enable default compression, effectively LZ4
# Enable LZ4 explicitly
# Disable compression
Enabling snapshot directory visibility
ZFS supports auto-mounting a .zfs directory, at the root of each dataset. The .zfs/snapshot subdirectory will contain a directory for every single snapshot.
To enable this:
Operations
Rollback
List snapshots
Rollback
If there any newer snapshots present, you’ll have to add the -r option, otherwise you’ll hit the following error:
Per zfs help rollback:
-r Destroy any snapshots and bookmarks more recent than the one specified.