What Building a File Encryptor in Rust Taught Me About Authenticated Encryption

What this does

A command-line tool that encrypts a single file with a password. Under the hood: Argon2id turns your password into a 256-bit key, AES-256-GCM encrypts the file with that key, and everything needed to decrypt later (a random salt and nonce, plus the ciphertext) is packed into one self-contained output file.

Why I built it

I wanted hands-on experience with memory-safe systems programming applied to something that actually matters. Cryptographic code is exactly where memory bugs — buffer overflows, use-after-free, secrets leaking from uninitialized memory — do the most damage. Rust's ownership model is supposed to make entire classes of these bugs unrepresentable. I wanted to find out what that guarantee actually feels like while building something real, not a toy syntax exercise, and to come out the other side able to explain every cryptographic choice, not just the code that makes it work.

How it works — design decisions

Argon2id over SHA-256 or bcrypt

SHA-256 is fast — great for hashing files, terrible for passwords, since attackers can brute-force billions of guesses per second on GPUs. bcrypt is deliberately slow but memory-fixed (a few KB), so custom hardware (FPGAs/ASICs) can still parallelize attacks cheaply. Argon2id is memory-hard (19 MiB in this project, tunable) — a parallel attacker needs proportionally more RAM per attempt, and RAM is expensive to scale the way raw compute isn't. The "id" variant hybridizes Argon2i (side-channel resistant) and Argon2d (GPU resistant) for both properties, which is why OWASP recommends it as the default choice.

AES-GCM over AES-CBC

CBC gives confidentiality only, no integrity — it's malleable and historically vulnerable to padding oracle attacks (real precedent: POODLE, the ASP.NET padding oracle attack). GCM is an AEAD cipher: confidentiality plus a 16-byte authentication tag. Tampered ciphertext fails decryption outright instead of silently returning corrupted plaintext.

Nonce reuse under the same key is catastrophic: encrypting two messages with the same (key, nonce) generates the same keystream both times. XORing the two resulting ciphertexts cancels the keystream and exposes the XOR of the plaintexts — and can leak the authentication key itself, letting an attacker forge valid-looking ciphertexts. Always use a fresh random 96-bit nonce from the OS CSPRNG, never a counter or anything password-derived.

Self-contained output file over separate salt/nonce storage

The salt and nonce aren't secret — their security value is uniqueness, not secrecy — so nothing is lost by storing them in plaintext alongside the ciphertext. One file means one lifecycle; separate files can get lost, renamed, or separated independently, making decryption impossible even with the correct password. Fixed-length fields (16-byte salt, 12-byte nonce) mean decryption is just fixed-offset slicing — no parser, no format-version field, minimal surface area for bugs.

Password entry: interactive prompt, not a CLI argument

A --password flag would land in shell history and be visible to other processes via ps/Task Manager — a real, practical leak. Using rpassword to read from stdin with terminal echo disabled avoids both.

Zeroizing secrets in memory, not just relying on drop

When a String or byte array holding a password or derived key goes out of scope, Rust deallocates the memory — but deallocation doesn't overwrite the bytes. The old secret can sit in freed memory until something else happens to reuse that address, recoverable via a crash dump, a swapped-out memory page, or a debugger attached to the running process.

A manual "zero it out in a loop before it drops" approach doesn't reliably work either: the compiler's optimizer is allowed to notice that a write to memory that's about to be freed and never read again has no observable effect, and can legally delete the loop entirely. The zeroize crate solves this with a volatile write — a write the compiler is specifically forbidden from optimizing away — and its Zeroizing<T> wrapper triggers that write automatically on drop, including on early returns via ?. Wrapping the password and derived key in Zeroizing<T> required zero changes to the rest of the code, since it transparently derefs to the underlying type everywhere it's used.

One unified, deliberately vague error type

Every fallible operation — file I/O, key derivation, encryption, file-format parsing — converts into a single AppError enum so the CLI can use Rust's ? operator throughout instead of .unwrap(). The one subtle decision: the Cipher variant deliberately discards the real underlying error detail. Whether decryption failed because of a wrong password or a tampered/corrupted file is indistinguishable to the end user, on purpose — that distinction is itself information an attacker could use to probe the system.

Bugs and gotchas hit while building

• Toolchain mismatch. clap's newer releases require edition 2024, which needs Rust 1.85+. My local Rust was 1.75.0, many months stale. Fixed with rustup update stable. Lesson: trust rustc --version and update the toolchain — don't pin old crate versions to route around an outdated compiler.

• A misplaced Cargo.toml key. argon2 and rand were accidentally declared under [package] instead of [dependencies]. Cargo silently ignored them, producing "unresolved import" errors that looked exactly like a missing crate rather than a misplaced TOML key.

• Stale IDE cache, not a real error. rust-analyzer reported an "unsupported metadata version" error right after a toolchain upgrade. It was leftover proc-macro cache from the old compiler — cargo build had been succeeding the entire time. Lesson: cargo build/cargo run output is ground truth over editor squiggles.

• A panic is not "failing cleanly." Decrypting with the wrong password initially crashed the whole process with a raw Rust panic and stack trace via .unwrap(). It didn't corrupt anything — the panic happened before the output file was ever written — but a stack trace isn't a clean user-facing failure. Fixing this was the actual point of Step 6: replacing every .unwrap() with a unified error type and ?.

• main() -> Result<_, E> prints Debug, not Display. Rust's built-in support for a Result-returning main automatically prints the error — but using the type's Debug implementation, not Display. My carefully worded, non-specific error message never appeared; instead I got the bare enum variant name (Error: Cipher). The fix was to handle the Result manually in main and print with {} instead of relying on the automatic path.

Concrete numbers worth remembering

• A 14-byte plaintext test case produced a 30-byte ciphertext — the 16-byte GCM authentication tag is constant overhead, independent of file size.

• Full packed file layout for that test case: 16-byte salt + 12-byte nonce

  • 30-byte ciphertext = 58 bytes total.

What I'd improve

What I'd improve

• Add integration tests (in a tests/ directory) that exercise the compiled binary end-to-end as a black box, rather than only unit-testing internal functions.

• Support streaming large files instead of reading the whole file into memory, if this ever needed to scale beyond small files.

• Consider a file-format version byte, so a future breaking change to the layout could be detected and handled gracefully instead of silently misinterpreting old files.

How to run it

Full setup, build, and usage instructions are in the GitHub repository.