Make elapi bill-teams plugin with elapi installation optional

[alexander-haller]

In GitLab by @mhxion on Jun 9, 2024, 17:30

So far, when elAPI is installed via Pipx/Pip, i.e., pip/pipx install elapi, it is installed with bill-teams plugin. Running elapi/elapi --help will show bill-teams under the "Plugins" category, or running elapi bill-teams will show the sub-commands offered by bill-teams.

Since, elapi bill-teams is mostly useful to URZ and specific to the use-cases of URZ, we propose bill-teams plugin be an optional plugin when installing elAPI.

We leverage the "optional dependencies" feature of PyPI, and define a keyword which can be passed to elapi[<keyword>], and when it is passed, Pipx/pip will install elapi with bill-teams-specific optional dependencies. And, bill-teams plugin will see the optional dependencies are present/installed, and enable itself. If those optional dependencies are not present, bill-teams will disable or hide itself from the "Plugins" category. To clarify with an example:

$ pipx install elapi[uhd-urz]  # The keyword "uhd-urz" chosen as an example. We have not decided on the exact keyword yet.
....
# This will install elapi with bill-teams plugin enabled.
$ elapi bill-teams
<Shows bill-teams sub-commands as usual>

$ pipx install elapi  # How most people will install elAPI
# This will install elapi with bill-teams plugin disabled or hidden from the plugins list.
$ elapi
# This will not show bill-teams under "Plugins" category
$ elapi bill-teams
...
Error: No such command 'bill-teams'.

In PyPI documentation's words, "elapi[uhd-urz] effectively becomes a variant of elapi." These "optional dependencies" specific to bill-teams are: tenacity (that we use for retries), python-dateutil (that we use for calculating complex date-times). In other words, when elAPI is installed without the uhd-urz keyword, tenacity and python-dateutil are not installed at all. This also reduces the size of installed elAPI. Users only get what they need. Note, the codebase for plugin bill-teams still lives inside elapi either way, it's when tenacity and python-dateutil are not installed, bill-teams just disables itself (of course bill-teams's functionalities would not work without those dependencies anyway).

This solution will not restrict us from making bill-teams a standalone plugin.

Benefits

1. In the future, if we ever decide to make bill-teams a standalone plugin (which it will be possible once our third-party plugin support is complete), we will still be able to install elapi with our chosen keyword pipx install elapi[uhd-urz]. I.e., this solution is future proof.
2. We will be able to add more plugins to elapi[uhd-urz] that are specific to our use-cases without worrying about the public impact.
3. Reduces elapi's package size when installed without the keyword.

Pitfalls

1. We will need to update our Ansible deployment to include pipx install elapi[uhd-urz]. Our current configuration pipx install elapi==1.0.7.dev2 will continue to work.
2. Some other user will still be able to see/read or access bill-teams (e.g., if they already have those dependencies: tenacity and python-dateutil installed). This should not be a big issue, as most users will likely not encounter this exact case.

Resource

This package[<keyword>] standard is typically used for this very exact purpose (almost!). Most packages use this to enable additional features to their main package. In our case, even when bill-teams is available to an outside user (they could have installed elapi with elapi[uhd-urz]), bill-teams plugin will remain not-very-useful to them. See some examples on how other packages (e.g.,apache-airflow, splinter) utilize this standard below:

1. https://stackoverflow.com/a/60307740

[alexander-haller]

In GitLab by @mhxion on Jun 9, 2024, 17:36

I have already tried 50% of the implementation (though there's nothing new to implement, it's mostly just updating our package metadata), and it works as expected. When one of the optional dependencies explained above is missing, or if this proposal is approved, and elapi is installed via pipx install elapi (without keyword), will show:

[alexander-haller]

Great stuff - you have to walk me through some questions in our Meeting but I like the direction!

Adapting our own Ansible deployment is always an option - and a pretty simple one atm. It will never block an useful change as far as I'm concerned.