Closed nijel closed 6 months ago
chardet is not required in aeidon/gaupol, it's imported only under the aeidon.encodings.detect
function and guarded with if aeidon.util.chardet_available():
. I think I put in the setup-aeidon.py
's install_requires
for convenience. Do I understand correctly that there's no opt-out type depedencies, i.e. ones that would be installed by default but that you could opt out of with some syntax?
I think I was a bit eager to make dependencies optional back when writing these. Encoding auto-detection is something that probably 95% of users want, since they're downloading random subtitle files from the internet and can't really know the encoding.
If you do pip install aeidon
you end up with chardet
. You can manually uninstall it, but pip will then complain about unmet dependencies.
Anyway, I've created https://github.com/otsaloma/gaupol/pull/223 to migrate to charset-normalizer, please review it if this is something you want.
While memory profiling Weblate I've noticed that 2+ MB is consumed on
chardet
module which we directly don't depend on.The only reverse dependency for
chardet
isgaupol
in our case. Everybody else seems to have switched tocharset-normalizer
instead, which is a maintained, faster and low memory footprint alternative.I'm willing to contribute a pull request, but first I'd need to know which direction you prefer. Two approaches I can see:
charset-normalizer
, but it supports only Python 3.5 and newer while aeidon supports older Python versions, so it would mean raising the bar. I don't think this should be an issue these days.chardet
from required dependencies, moving it to extras.