For MailRules v0.3.0View this page on GitHub

Backup and the master key

What the data directory holds, how to back it up and restore it, and what happens if the master key is lost.

Everything MailRules keeps is in its data directory: /data in the Docker image (the mailrules-data volume), /var/lib/mailrules with the systemd unit, and ./data otherwise. See Choosing a data directory.

What is in the data directory

  • mailrules.db, with mailrules.db-wal and mailrules.db-shm next to it while MailRules runs: the SQLite database. It holds your rules, sender rules, settings, decisions, the action log used for undo, the contacts learned from your Sent folder, short email snippets, and your mailbox passwords and model keys (and, in the hosted version, the addresses of notification destinations) in encrypted form.
  • master.key: the key that protects those passwords and keys. Each account on the install (each team, in the hosted version) has its own data key that seals its secrets, and the database keeps those data keys only wrapped by the master key. The master key is generated on first run, readable only by the user MailRules runs as, and never stored in the database. It is still the one thing to back up next to the database: without it no data key, and so no secret, can be opened.
  • log.key, only once LOG_REDACT_ADDRESSES has been on: the key the log’s address tags are made with. It opens nothing; losing it only gives every address a new tag. See Secrets and logs.

The database never holds full email bodies. Your email itself stays on your mail server.

Back up the data directory as a whole, and keep the backup as private as your passwords: the database and the key together unlock them.

Backing up

The database is written continuously, so copy it while MailRules is stopped:

Docker:

docker stop mailrules
docker run --rm -v mailrules-data:/data:ro -v "$PWD":/backup alpine \
  tar czf /backup/mailrules-backup.tar.gz -C /data .
docker start mailrules

With Compose, use docker compose -f deploy/docker-compose.yml stop and start. Compose names the volume after the project, usually deploy_mailrules-data; docker volume ls shows the exact name.

systemd:

sudo systemctl stop mailrules
sudo tar czf mailrules-backup.tar.gz -C /var/lib/mailrules .
sudo systemctl start mailrules

Binary or Homebrew: stop mailrules serve, copy the data directory, start it again.

Back up before every upgrade. A new version updates the database when it first starts, and MailRules has no command to undo that, so the backup is how you go back.

To keep a readable copy of just your rules as well, export them: Export rules on the Rules screen, or mailrules rules export rules.yaml (see Rules).

Restoring

  1. Stop MailRules.
  2. Replace the contents of the data directory with the backup, including master.key (unless you keep the key elsewhere; see below).
  3. Make sure the files belong to the user MailRules runs as: uid 65532 in the Docker image, mailrules with the systemd unit.
  4. Start MailRules.

With Docker, for a backup made as above:

docker stop mailrules
docker run --rm -v mailrules-data:/data -v "$PWD":/backup alpine \
  sh -c 'rm -rf /data/* && tar xzf /backup/mailrules-backup.tar.gz -C /data && chown -R 65532:65532 /data'
docker start mailrules

Restore the key before you start MailRules. If master.key is missing from the data directory, neither MAILRULES_MASTER_KEY nor MAILRULES_MASTER_KEY_FILE is set, and the database holds stored mailbox passwords or model keys, MailRules does not start. It does not make a new key, because the passwords and keys stored under the old one could then never be read. It stops with a message that names the data directory:

mailrules: the master key is missing: there is no master.key in /var/lib/mailrules, and neither
MAILRULES_MASTER_KEY nor MAILRULES_MASTER_KEY_FILE is set. The database there holds 2 stored
mailbox passwords and model keys sealed under it, and MailRules will not make a new key: …

Put the master.key from your backup into the data directory, or point MAILRULES_MASTER_KEY_FILE at it, and start again. If the key is gone for good, see below. A fresh install, whose database holds no passwords or keys yet, generates its key on first start as before.

MailRules makes the same check when the key is there but is the wrong one: it stops if the key opens none of the stored passwords and keys, for example because master.key came from another install. A key that opens only some of them is not stopped; the mailboxes it cannot open say decrypt failed.

If the master key is lost

The stored mailbox passwords and model keys cannot be recovered without it. Nothing else is lost: rules, settings, activity and undo history are not encrypted.

MailRules refuses to start without the key while it holds stored passwords or keys (see Restoring). To go on without the old key, start it once with MAILRULES_NEW_MASTER_KEY=true, or mailrules serve --new-master-key. It then generates a new master.key (or, if a key is given in the environment or a file, goes on with that one), and logs a warning. The data keys the old master key wrapped are replaced, and the passwords and keys sealed under them are deleted, since nothing could ever open them again. With Docker, add -e MAILRULES_NEW_MASTER_KEY=true to the docker run command; with Compose, put MAILRULES_NEW_MASTER_KEY=true in deploy/.env for that start. Set it for one start only: leaving it on would let a later lost key go unnoticed.

You will see mailboxes stopped because their sign-in failed, and model keys shown as not set. To recover:

  1. For each mailbox, open Mailboxes → Edit and enter the app password under New app password. You may want to make a new app password with your provider.
  2. In Settings → Model API keys, enter each key again. Keys set in the environment are not affected.

There is no command to change the master key of an existing install, so treat a lost key and a replaced key the same way.

Keeping the key outside the data directory

To keep the key apart from the database, for example on a different disk or in a secrets store:

  • MAILRULES_MASTER_KEY_FILE names a file holding the key. If that file is missing, MailRules refuses to start instead of making a new key, whatever the database holds.
  • MAILRULES_MASTER_KEY holds the key itself.

Set only one of them. The key is 32 random bytes, base64-encoded, the same format as master.key. To move an existing install, copy the contents of master.key into the new file, set MAILRULES_MASTER_KEY_FILE, restart, and then remove master.key from the data directory. For a new install you can make a key with openssl rand -base64 32.

Commands such as mailrules accounts add and mailrules accounts test read stored passwords, so they need the same setting as the daemon.

When a key service holds the keys (hosted version)

A build can have a module keep the key that wraps the data keys in a key service, such as a cloud KMS with keys in hardware security modules, instead of the master key. The hosted version does. Its key never leaves the service, so a backup of the database opens nothing without access to that service’s key: back up the database, and keep the key service’s key from being deleted (a KMS destroys a key only after a waiting period you set). The master key is not generated in such a build.

Moving an install that used the master key to a key service:

  1. Keep MAILRULES_MASTER_KEY (or MAILRULES_MASTER_KEY_FILE) set, and start the build with the module.
  2. At that start, MailRules wraps each data key again under the key service, once per account, and reseals under it the copies of removed mailboxes’ secrets that wait for their module. Passwords and keys stay as they are: they are sealed under the data keys, which do not change. A data key the key service has wrapped is never opened with the master key again.
  3. When the start has nothing left that needs the master key, it logs nothing in the database needs the master key any more: it can be retired. Until then it logs the master key is still needed, with how many data keys, secrets and removed mailboxes’ copies still need it; the next start moves what the key service could not take while it was unreachable.
  4. Once a start has said it can be retired, unset MAILRULES_MASTER_KEY and MAILRULES_MASTER_KEY_FILE and remove master.key from the data directory.

Without the master key, while something still needs it, MailRules does not start and says how many data keys, secrets and copies only it opens. Set the key and start again. If it is gone for good, MAILRULES_NEW_MASTER_KEY=true lets MailRules start for once without it: what only it opened stays unreadable and is entered again, as in If the master key is lost.

Commands such as mailrules accounts need the build’s module too, with the same settings as the daemon.

Tell your inbox what to do. Then stop thinking about it.