Filesystems turn one big array into many small ones

State of the note

As of Thursday Jun 5 2025, I’m pretty happy with how this note turned out. There’s more to come on filesystems in general, but this note feels well focused on its own.

The core job of a file system is to provide its users (humans, other programs, the operating system, …) a nice abstraction to interact with a storage device.

The physical device

Almost every storage device presents a relatively straightforward read/write interface to the CPU:

• To read data, you give it an address (we call that a block number) and it will tell you the data that lives there.
• To write data, you give it a block number and some data, and it will write it there.

Typically, the amount of data is something like 512 bytes, we call that the block size. Anything read/written to a hard drive happens in chunks of 512 bytes. This also means that a file that has a single byte in it has the same size as one with 512 bytes of data.

There is no such thing as an “empty” block: even straight from the factory, you can request to read a block of data. The device will respond with whatever is physically there. There is maybe nothing interesting, but at a hardware level, the bits are either 0 or 1. There is no such thing as “empty”.

The files

The filesystem’s entire job is to turn this long array into “files”. A file is essentially just a (much smaller) array: it has a beginning, and then we can keep reading it, until we reach the end.

On the hard drive, this file might fit within a single block, or it might span multiple. These blocks may be in a neat sequence, or they may be kinda randomly scattered. It is the job of the filesystem to keep track of that for you.

This allows us to do all the familiar operations we know of: create a file, write some data into it, save it. Create another file, and read/write/delete data from somewhere in the middle of that, and so on.

What about folders?

We typically think of filesystems as having files and folders. However, often times folders are just “special files that point to other files”. You can add data to them (create a new file), modify the data (renaming a file), and delete some data (delete a file). The filesystem typically does not let you do that directly, but fundamentally: a folder is a file that gets special treatment.

Crashes

One of the biggest concerns when designing a filesystem is to make sure it can crash and still recover.

The nightmare scenario is that your laptop suddenly loses battery in the middle of the filesystem doing a bunch of important modifications to how it organizes and keeps track of blocks and files.

When you reboot your computer, one of the first things that it does is look for your operating system. Your operating is, ultimately, stored on your hard drive… as a file in the filesystem. If the file system isn’t able to figure out where that is, your operating system effectively no longer exists.

It doesn’t take much for this to happen: if the filesystem misplaces a single block of that operating system code, it’s unlikely that the OS will start. There are ways to attempt to repair and fix this, but they are annoying and might still not succeed.

Meta

Thursday Jun 5 2025

• Add drawings/animations
• Flesh out the “crashes” portion into its own document