Aspell returns an empty dictionary list

01mf02 commented 7 years ago

Issue description

When requesting a list of available dictionaries from the aspell API, it searches for its dictionaries in the path where aspell was built, but not where the dictionaries actually are, thus always returning the empty list. The API unfortunately also seems to ignore the ASPELL_CONF environment variable when building the list of available dictionaries (see #29429). Note that it seems that the dictionaries are still found when requested explicitly. Therefore, I'm not sure whether this is not a bug in aspell. One workaround might be to build aspell with dict-dir being set to /run/current-system/sw/lib/aspell, but this might be a kind of hacky solution (and possibly only work with aspell-with-dicts?).

This also impacts programs that depend on aspell, such as enchant and Claws Mail.

Steps to reproduce

Put something like (aspellWithDicts (ps : with ps; [ cs de en fr it ])) into your systemPackages (in configuration.nix).
Download and unpack spell-test.tar.gz.
Run nix-build in the spell-test directory.
Run result/aspell-test to see what happens when you use the default API -- no dictionaries should be found.
Run result/aspell-test fix to see what happens when you set the right dictionary path (via the aspell API) before using the default API -- more than zero dictionaries should be found.

On my system, the output is:

[michi@nixos:~/spell-test]$ export | grep ASPELL_CONF
declare -x ASPELL_CONF="dict-dir /run/current-system/sw/lib/aspell"

[michi@nixos:~/spell-test]$ result/aspell-test 
dict-dir = /nix/store/qhb8mq3cl1r6w5pvsr8p9zawkwl538bn-aspell-0.60.6.1/lib/aspell
Dicts: 0

[michi@nixos:~/spell-test]$ result/aspell-test fix
Fixing dict-dir ...
dict-dir = /run/current-system/sw/lib/aspell
Dicts: 54

Technical details

System: NixOS: 17.09beta647.20e1833914 (Hummingbird) Nix version: 1.11.14 Sandboxing enabled: false

FRidh commented 7 years ago

~I thought it was fixed with 20e1833914b0594e53a1e59c8f4ed2d86ac544f6?~

OK, will have a look at this later.

01mf02 commented 7 years ago

For other readers: FRidh's commit at https://github.com/NixOS/nixpkgs/commit/20e1833914b0594e53a1e59c8f4ed2d86ac544f6 fixed one aspect -- aspell finding an explicitly requested dictionary. This bug is about aspell not retrieving a list of all available dictionaries.

I already dug into the aspell source code, but it was quite intricate. Relevant parts for directory finding seem to be in common/config.cpp as well as in common/file_data_util.cpp. The actual list of dictionaries seems to be constructed in ModuleInfoList::fill from common/info.cpp.

The bug in aspell seems to be that loading specific dictionaries loads the correct directory value from ASPELL_CONF (via the function read_in_settings), while loading the list of dictionaries does not consider ASPELL_CONF. I have created a patch that also reads ASPELL_CONF settings when loading the list of dictionaries. On my quick aspell-test, this builds the dictionary list correctly.

The patched version of aspell is here. Unfortunately, I could not test it on my whole system to replace the default aspell, because rebuilding my system on my VM will probably run out of memory.

01mf02 commented 7 years ago

If you can tell me a simple way to build only the necessary system parts for, say, Claws Mail with the patched aspell, then I am of course willing to test that.

FRidh commented 6 years ago

Could you open a PR with your proposed change?

01mf02 commented 6 years ago

Hm, in the meanwhile, ba4cefe4ae70cfb9121281f508ed5d9fa9dbc662 seems to have addressed in an elegant way the issue I have raised. I cannot test it right now though, because I currently do not have a NixOS installation at hand. I hope I'll get around to test it later this week.

01mf02 commented 6 years ago

Okay, I tested the new aspell with ba4cefe4ae70cfb9121281f508ed5d9fa9dbc662 now. My test in the OP still returns no dictionaries. However, reading the commit message of ba4cefe4ae70cfb9121281f508ed5d9fa9dbc662, this is actually to be expected:

[...] in some cases, when used as dependency by another derivation, the passed dictionaries will be missing. However, invoking aspell directly should be fine.

Aspell is used as a dependency by many other derivations, so that fix pretty seriously limits the usability of aspell. Furthermore, ba4cefe4ae70cfb9121281f508ed5d9fa9dbc662 adds a patch to load the right dictionary directories, but does actually not enable the patch when building aspellWithDicts. I do not understand why ... Any idea how to proceed here? Unfortunately, the committer of the fix, Rostislav Beneš, seems not to be on GitHub ...

FRidh commented 6 years ago

@01mf02 when we have tools that have (optional) run-time dependencies, there are basically two options one can take:

Write the derivation so that during run-time it searces PATH and the Nix profiles and whatever other env variables
Record absolute paths to all dependencies, so that everything is pulled in.

You can install aspell, and its dictionaries separately, and aspell can function this way by scanning the profiles for the dictionaries. This is an implementation of solution 1). The aspellWithDicts function does 2).

Which approach are you interested in, and what is not working for that approach? Looking at the two implementations, it seems aspellWithDicts doesn't set data-dir.

stale[bot] commented 4 years ago

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

Search for maintainers and people that previously touched the related code and @ mention them in a comment.
Ask on the NixOS Discourse.
Ask on the #nixos channel on irc.freenode.net.

nixos-discourse commented 8 months ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/aspell-dictionaries-are-not-available-to-enchant/39254/1

nbCloud91 commented 2 days ago

This is still an issue, when I use the aspellWithDicts package, upon doing normal checks like:

aspell --lang=en_US --encoding=utf-8 check tmp.error

it works. But when I run it to create a master file needed for recoll

 aspell --lang=en_US --encoding=utf-8 create master /home/username/.recoll/aspdict.en.rws

it starts to look for dict files in the directory where it does not exist

  Error: The language "en_US" is not known. This is probably because: the file "/nix/store/1amvdri3kwnv0bhz3bx0ix1zmrkrhip8-aspell-0.60.8.1/lib/aspell/en_US.dat" can not be opened for reading.

Is there any workaround?

NixOS / nixpkgs