Use proper maubot dependency management system when one becomes available

[HarHarLinks]

I tried to install this bot in the way that maubot plugins are usually installed. That is:

1. since you don't provide the plugin prebuilt, i build and upload it mbc build maubot-events -u based on the v0.2.0 tag
2. in the maubot web ui, create a new instance with some existing client and the org.fedoraproject.maubot.events plugin
3. it fails:

   
   [00:46:57](file:///opt/maubot/maubot/loader/zip.py:203) ERROR   maubot.loader.zip   Failed to load org.fedoraproject.maubot.events v0.2.0
   [00:46:57](file:///opt/maubot/maubot/loader/zip.py:203) ERROR   maubot.loader.zip   Traceback (most recent call last):
   File "/opt/maubot/maubot/loader/zip.py", line 216, in _load
   importer.load_module(module)
   File "/opt/maubot/maubot/lib/zipimport.py", line 280, in load_module
   exec(code, mod.__dict__)
   File "/data/plugins/org.fedoraproject.maubot.events-v0.2.0.mbp/event_helper/__init__.py", line 14, in <module>
   import validators
   ModuleNotFoundError: No module named 'validators'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/maubot/maubot/loader/zip.py", line 201, in load return self._load(reset_cache) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/maubot/maubot/loader/zip.py", line 220, in _load raise MaubotZipLoadError(f"Failed to load module {module}") maubot.loader.zip.MaubotZipLoadError: Failed to load module event_helper 00:46:57 ERROR maubot.server Error in handler 00:46:57 ERROR maubot.server Traceback (most recent call last): File "/opt/maubot/maubot/loader/zip.py", line 216, in _load importer.load_module(module) File "/opt/maubot/maubot/lib/zipimport.py", line 280, in load_module exec(code, mod.dict) File "/data/plugins/org.fedoraproject.maubot.events-v0.2.0.mbp/event_helper/init.py", line 14, in import validators ModuleNotFoundError: No module named 'validators'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/maubot/maubot/management/api/middleware.py", line 49, in error return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/maubot/maubot/management/api/instance.py", line 83, in update_instance return await _create_instance(instance_id, data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/maubot/maubot/management/api/instance.py", line 59, in _create_instance await instance.start() File "/opt/maubot/maubot/instance.py", line 362, in start cls = await self.loader.load() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/maubot/maubot/loader/zip.py", line 201, in load return self._load(reset_cache) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/maubot/maubot/loader/zip.py", line 220, in _load raise MaubotZipLoadError(f"Failed to load module {module}") maubot.loader.zip.MaubotZipLoadError: Failed to load module event_helper

[MoralCode]

Thanks so much for reaching out and providing feedback!

Honestly ive sorta been working a little backwards and starting from the development environment at https://github.com/fedora-infra/matrix-bots, so im more familiar with Fedora's way of doing things than the standard maubot way (although I assume they have to be fairly similar)

That said, despite this being my first time using maubot, I do use a very similar mbc buildstep within the dev environment's Vagrant VM and it seems to work for me.

Based on the error above it seems like the environment you are trying to build the bot in does not have all the python dependencies installed (specifically that error seems to indicate that validators is missing. If you run pip install -r requirements.txt and then retry your build command, does it work?

I'd love to learn whether there's a more "standard maubot" way of specifying dependencies that doesn't require this extra step.

[HarHarLinks]

Here is the maubot development manual: https://docs.mau.fi/maubot/dev/getting-started.html I tried adding validators to the maubot.yaml dependencies, but that does not work either. I believe that building the plugin needs to build any dependencies that are not part of the default maubot distribution into the plugin package.

Also, I am running maubot itself in a container (https://docs.mau.fi/maubot/usage/setup/docker.html). No easy installing random python packages.

[MoralCode]

thankfully i wasnt using the validators package for much anyway (other than domain name validation for matrix IDs) so i replaced this with essentially copy pasting the regex from the source code of that package

Can you check out the validators-fix branch to see if that helps fix things?

[HarHarLinks]

Now I get the same error for the next library. I think we need to solve how to properly package the plugin.

[MoralCode]

i also looked at the docs for the dependencies config entry and it said this (emphasis mine)

dependencies - A list of Python modules and their version ranges that the plugin requires. This is currently not used, but in the future maubot will offer to automatically install dependencies when uploading a plugin.

so i think i just need to do a better job of documenting the dependency requirements of this bot, at least until maubot supports this config item (right now it just seems like it is a non-functional equivalent to requirements.txt embedded into maubot.yaml for possible future use.

I don't really see any other "official"/"proper" way to package these dependencies so the best option may just be to do it ad-hoc for now.

Would it help if i created a docker container that was based on the maubot official one you said you were using, but with the correct python dependencies installed for this plugin?

[HarHarLinks]

Looking at https://plugins.mau.bot I see some plugins that appear to manage dealing with dependencies. On the other hand the install instructions for https://github.com/SAPUCC/inviterbot?tab=readme-ov-file#installation support what you have been saying, that is the need to install the deps in the maubot server('s container). I did what the latter recommends and the plugin now loads.

[MoralCode]

sounds good!

I'll update this issue to become an "FYI, use a proper maubot dependency management tool when one becomes available in maubot" issue. If theres an upstream issue in maubot to add this feature, I'd love to link it!

[HarHarLinks]

https://github.com/maubot/maubot/issues/10

[HarHarLinks]

For now, what could be done is to explain the situation in the installation instructions. It could go so far as to even provide a prebuilt container via ghcr.io that comes with the required dependencies. I've set up the same in a private repo for myself for now.

[MoralCode]

For now, what could be done is to explain the situation in the installation instructions. It could go so far as to even provide a prebuilt container via ghcr.io that comes with the required dependencies. I've set up the same in a private repo for myself for now.

This is done! The dockerfile sample you sent me is included in the repo