himalaya: notmuch-backend not built despite feature set true

[jhvst]

Describe the bug

Seemingly, cargoBuildFlags = [ "--features=notmuch-backend" ]; has to be set for notmuch feature to be enabled.

However, even after this is set, and when notmuch works for a given mailbox, opening himalaya shows no results.

Steps To Reproduce

Steps to reproduce the behavior:

1. create an overlay in which withNotmuchBackend is set to true
2. there will be an error saying that notmuch is not a valid backend
3. create an overlay that sets cargoBuildFlags, and it now 2. disappears, but no mail is shown

Expected behavior

After enabling the feature flag, I would expect notmuch to be built, specifically, I don't expect 2. to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Connected to https://github.com/nix-community/home-manager/issues/2888

Notify maintainers

@soywod @toastal @yanganto

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-darwin"`
 - host os: `Darwin 22.5.0, macOS 13.4`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.13.3`
 - channels(root): `"nixpkgs"`
 - channels(juuso): `"darwin, home-manager, nixpkgs"`
 - nixpkgs: `/Users/juuso/.nix-defexpr/channels/nixpkgs`

[jhvst]

Here is also my mailbox configuration and the overlay which I use to get the notmuch to be built: https://github.com/jhvst/nix-config/commit/bc47023050df9e35e7b467d142443079df566c9b

[jhvst]

And here is my home-manager patch I use to enable the feature in home-manager: https://github.com/nix-community/home-manager/compare/master...jhvst:home-manager:master

[soywod]

The problem is more complex than I thought. It looks like there are multiple issues:

Notmuch database path expansion

The notmuch database path was not expanded correctly (~, env vars), which I fixed on master.

.notmuch folder

Let's say you have a Maildir++ folder: ~/Mail where the inbox is at the root level (cur folder) and other folders are Maildir++ folders (starting with a dot):

├── .Archives
│   ├── cur
│   ├── new
│   └── tmp
├── cur
├── .Drafts
│   ├── cur
│   ├── new
│   └── tmp
├── .Junk
│   ├── cur
│   ├── new
│   └── tmp
├── new
├── .Notes
│   ├── cur
│   ├── new
│   └── tmp
├── .notmuch
│   └── xapian
├── .Sent
│   ├── cur
│   ├── new
│   └── tmp
├── tmp
└── .Trash
    ├── cur
    ├── new
    └── tmp

and that you configure notmuch this way:

[database]
path=~/Mail

Notmuch creates a .notmuch folder at the given path. The problem is that this folder is considered a Maildir++ folder, which leads to sync errors. I also fixed this issue on master by discarding the .notmuch folder.

Empty result

At this point (with the same maildir structure mentioned earlier), himalaya list returned an empty table for me also. The reason is that, behind the scene, the notmuch backend uses the -f|--folder as a notmuch search folder: attribute. So: in order to list emails from my inbox, I need to give an empty string so it refers to the current directory: himalaya list -f "". To list emails from my trash folder (.Trash): himalaya list -f .Trash.

I see that you are using mbsync, and if I remember well mbsync does not respect the Maildir++ standard. Mailboxes are just regular folders (not starting by dot), even the inbox. So technically you could try himalaya list -f Inbox. Let me know. Feel free to open a dedicated issue by sending an email at ~soywod/pimalaya@todo.sr.ht.

Nixpkgs derivation

I was able to enable the notmuch backend using the REPL:

$ cd ~/code/nixpkgs
$ nix repl

nix-repl> :l .
Added 19227 variables.

nix-repl> :b himalaya.override { withNotmuchBackend = true; }

This derivation produced the following outputs:
  out -> /nix/store/in5hfbxiribn8vv1wfp9xlm9plgdvl5v-himalaya-0.7.3

So it should work from your configuration:

let himalaya = pkgs.himalaya.override { withNotmuchBackend = true; };

Home manager

Could notmuch be added if both mbsync and notmuch are enabled for a given account?

I would say that checking notmuch is enough, since you do not necessary use mbsync with notmuch (you could use OfflineIMAP, or himalaya sync).

---

Issues are not directly related to nixpkgs. CLI-related issues will be resolved for the next release (because fixes are already on master), and home manager already have an open issue for supporting notmuch, so I guess we can close the issue?

[jhvst]

Thanks for the insightful comment! I'm just starting to do terminal email, so I expected that part of this problem might be my configuration error, hence why I shared the Nix files.

I'm OK to close this issue. I will investigate and re-open in the correct forum (sourchehut seemingly) should there be any bugs.

Off-topic: for Maildir++, what client would you recommend?

[soywod]

Thanks for the insightful comment! I'm just starting to do terminal email, so I expected that part of this problem might be my configuration error, hence why I shared the Nix files.

It was very useful, thank you! I was able to fix the home manager package to support notmuch: https://github.com/nix-community/home-manager/compare/master...soywod:home-manager:himalaya-v0.8.0. Once I release the v0.8.0 I will open the PR.

Off-topic: for Maildir++, what client would you recommend?

Himalaya :D it supports Maildir and Maildir++. It also support synchronization (like mbsync). If you have an IMAP account then it get synchronized locally in a Maildir++ folder.

[jhvst]

Himalaya :D it supports Maildir and Maildir++. It also support synchronization (like mbsync). If you have an IMAP account then it get synchronized locally in a Maildir++ folder.

Thanks, I will try it with notmuch once the new release lands on Nix.

[soywod]

New release in progress: https://github.com/NixOS/nixpkgs/pull/235855