Separate synchronization state from configuration

[tadfisher]

I use nix to manage all of my software configurations. What it does is generate immutable configuration files in a store, then symlinks those files to the correct locations for software that cannot be configured to look in arbitrary locations for config files.

I have a problem doing this for Gmailieer, because Gmailieer modifies .gmailieer.json after each pull with ephemeral state:

• last_historyId
• lastmod

Perhaps these could be stored in a separate file, or an "immutable" configuration could be provided?

An alternative is to wrap gmi with CLI configuration options, but I'd rather generate config files.

[gauteh]

I think you could store a file with those fields missing and they would get their default values read if they are missing in the file. Note that you also need the auth tokens (which are renewed at intervals) to avoid having to re-auth.

[gauteh]

Closing untill further activity.

[NickHu]

I'd also like to chime in on this; I'm looking into writing a home-manager module for gmailieer, but am running into the same problem.

I think you could store a file with those fields missing and they would get their default values read if they are missing in the file.

What did you mean by this?

[gauteh]

If you remove some values from the configuration/state file they will be set to default when gmailieer starts. So you can store the static part of the file and copy it into the repo, gmailieer will then complete it on first start. At least in theory.

[NickHu]

The way nix works is that it would create a symlink from ~/.mail/gmail/.gmailieer.json to /nix/store/blahblahblah/.gmailieer.json, and /nix/store is mounted read-only. So it is really quite essential for the configuration and state to be separate files in this case; that is, gmailieer needs to not panic when reading a read-only configuration file.

[gauteh]

Gmailieer definetely needs a writable config file at the moment. But lots of application do, surprised that nix doesn´t support that?

[NickHu]

Sure it does, and in fact gmailieer is already packaged in nix. What we're talking about here is having nix also manage the configuration too, so that a user could just write a couple lines into their nix config and have it all set up pretty much automatically. In order to do that, configuration needs to be read-only.

For an example of what I'm talking about, if I write the lines

home.programs.notmuch.enable = true;

(assuming I have set up my email in the config too) then it autogenerates ~/.notmuch-config, which becomes a symlink to a file in the read-only nix store which looks like

# Generated by Home Manager.

[database]
path=/home/nick/.mail

[maildir]
synchronize_flags=true

[new]
ignore=.uidvalidity;.mbsyncstate
tags=unread;inbox

[search]
exclude_tags=trash;deleted

[user]
name=Nick Hu
other_email=<redacted>
primary_email=<redacted>

If I want to change the exclude_tags, I can set home.programs.notmuch.search.excludeTags to something else in the same configuration (once this is merged). The idea is that you can have a relatively small configuration file which creates a specification of your entire working environment, and then you run home-manager switch and it among other things

1. installs the programs you specify (notmuch, gmailieer, etc.);
2. also configures them (rather than you having to laboriously manage your dotfiles in a git repo or something, which is not very robust between different systems and doesn't age well);
3. allows you to rollback to previous generations (in case you made a mistake, it's all still there in the nix store);
4. gives you confidence in the reproducibility of your builds (because the store path /nix/store/zwcv2v5szrj413l9mb4hb7fdam6grwx4-gmailieer-0.10 contains enough information to know what the versions of all the dependencies and the expressions used to build them are - effectively zwcv2v5szrj413l9mb4hb7fdam6grwx4 is a hash of the dependency graph used to build the package, and impurities are removed by the build environment).

I'm only touching on what nix the package manager can do here, but hopefully this gives you an idea of what I'm trying to achieve.

[gauteh]

Hm ok, well I dont see why we cant do that with gmailieer too. Would be happy to review / guide PR. Requires backwards compatability, but that should not be a problem since we can just check the config file for the values if the state file is missing.

[NickHu]

Ok great, I'll work on it soon I think. I think this can be done by basically

1. making gmailieer load two files, the config file and a state file;
2. making gmailieer able to be called from any directory (probably not necessary but would be nice) and more error robust in this sense;
3. thinking up a way to make the initial setup non-interactive (especially the OAuth flow - maybe another mode could be provided to use Google App Passwords instead?).

[tadfisher]

@NickHu Before you do all that work, see https://github.com/tadfisher/home-manager/tree/gmailieer

[NickHu]

@tadfisher Nice! Now if we make all the changes I outlined to gmailieer the work's basically all done. Will you upstream your gmailieer module to home-manager?

[tadfisher]

Yep, I'm planning on it. I can rip out all that config-updating code if you can make the changes.

[NickHu]

@gauteh @tadfisher I have opened a pull request in #105 which makes the changes I outlined (except non-interactive auth).

[gauteh]

I do not think non-interactive auth is possible. The auth tokens expire after a couple of weeks if they are not updated (you run gmi), in which case you have to re-do gmi auth.

[gauteh]

Closing for now, not possible to do non-interactive auth. Happy to review code for this.