configuring `all` modules: catching AttributeErrors on missing blocks?

[purarue]

Something I've been thinking about for a while now -- currently in the process of splitting my mail module to have an all.py file

So previously, there was no class mbox, it was the nested:

class mail:
    class imap:
        mailboxes = []
    # would be added for new module:
    # class mbox:
    #   mailboxes = []

So, the all.py imports from both imap.py and the new mbox.py: https://github.com/seanbreckenridge/HPI/blob/ffbae2767b8c11e2b093b8cf28a941b0a8dfa4f6/my/mail/all.py

Then, when running the my.mail.all doctor, which imports from both:

✅ OK  : my.mail.all                                       
❗      - stats:                      computing failed
   Traceback (most recent call last):
     File "/home/sean/Repos/HPI-karlicoss/my/core/__main__.py", line 267, in modules_check
       res = stats()
     File "/home/sean/Repos/HPI/my/mail/all.py", line 45, in stats
       return {**stat(mail)}
     File "/home/sean/Repos/HPI-karlicoss/my/core/common.py", line 454, in stat
       res = _stat_iterable(fr)
     File "/home/sean/Repos/HPI-karlicoss/my/core/common.py", line 486, in _stat_iterable
       count = ilen(eit)
     File "/usr/lib/python3.10/site-packages/more_itertools/more.py", line 496, in ilen
       deque(zip(iterable, counter), maxlen=0)
     File "/home/sean/Repos/HPI-karlicoss/my/core/common.py", line 469, in funcit
       for x in it:
     File "/home/sean/Repos/HPI/my/mail/all.py", line 34, in mail
       yield from unique_mail(
     File "/home/sean/Repos/HPI/my/mail/common.py", line 158, in unique_mail
       yield from unique_everseen(
     File "/usr/lib/python3.10/site-packages/more_itertools/recipes.py", line 413, in unique_everseen
       for element in iterable:
     File "/home/sean/Repos/HPI-karlicoss/my/core/source.py", line 45, in wrapper
       res = factory_func(*args, **kwargs)
     File "/home/sean/Repos/HPI/my/mail/all.py", line 27, in _mail_mbox
       from . import mbox
     File "/home/sean/Repos/HPI/my/mail/mbox.py", line 24, in <module>
       class config(user_config.mbox):
   AttributeError: type object 'mail' has no attribute 'mbox'

That imports like:

from my.config import mail as user_config

class config(user_config.mbox)

which is what causes the error - since the user doesnt have an mbox defined on their class mail in their config.

They may have not set it up yet, or they may just not want to use it

In the latter case, theres no way to configure this without modifying the all.py file and removing the source. That is a solution, but I think it would be nice to catch this AttributeError and send a warning to add the module to their disabled_modules or something instead?

That way it doesn't fatally error when they're using the all.py -- the so-called entrypoint to the whole module

This only happens when there are no additional dependencies for a module. If there were, import_source would correctly catch the import error and return the default

I guess at the top of mbox could do:

user_config = getattr(user_config, "mbox", object)

but feels not mypy friendly and pretty hacky

Can create a PR if catching the AttributeError sounds good -- would probably try and do regex to match it more specifically, to prevent catching unrelated errors

[purarue]

This would also benefit my.browsing -- as that currently uses a nested config and requires you to setup both active_browser ane export blocks in order to use the all.py file

[purarue]

After changes from this PR

If you directly with doctor it'll at least warn you to check the docs:

$ hpi doctor -S my.mail.mbox       
❌ FAIL: my.mail.mbox                                       loading failed
You're likely missing the nested config block for 'mail.mbox'.
See https://github.com/karlicoss/HPI/blob/master/doc/SETUP.org#private-configuration-myconfig or look for an example in the docs
   Traceback (most recent call last):
     File "/home/sean/Repos/HPI-karlicoss/my/core/__main__.py", line 242, in modules_check
       mod = importlib.import_module(m)
     File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
     File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
     File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 883, in exec_module
     File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
     File "/home/sean/Repos/HPI/my/mail/mbox.py", line 24, in <module>
       class config(user_config.mbox):
   AttributeError: type object 'mail' has no attribute 'mbox'

If you use it behind import_source, it warns you and continues:

 $ hpi doctor -S my.mail.all
✅ OK  : my.mail.all                                       
  /home/sean/Repos/HPI-karlicoss/my/core/source.py:57: UserWarning: Module my.mail.mbox (_mail_mbox) could not be imported, or isn't configured properly
    medium(f"Module {module_name} ({factory_func.__qualname__}) could not be imported, or isn't configured properly")
  /home/sean/Repos/HPI-karlicoss/my/core/source.py:58: UserWarning: If you don't want to use this module, to hide this message, add 'my.mail.mbox' to your core config disabled_modules, like:

  class core:
      disabled_modules = ['my.mail.mbox']

    warn(f"""If you don't want to use this module, to hide this message, add '{module_name}' to your core config disabled_modules, like:
You're likely missing the nested config block for 'mail.mbox'.
See https://github.com/karlicoss/HPI/blob/master/doc/SETUP.org#private-configuration-myconfig or look for an example in the docs

If you add it to disabled modules, works without warning, as expected:

[ ~ ] $ hpi doctor -S my.mail.all
✅ OK  : my.mail.all

[purarue]

solved by #224