Open gethvi opened 1 year ago
As I said in another communication channel: Thank you for investigating this topic, it was on the wishlist for many years. Previously the blocker were deficiencies in Python packaging tools (the namespace packages themselves weren't sufficient, I don't remember what exactly was the culprit).
Do we want to allow one bot to be composed of multiple packages, or do we require that every bot must be at most in one package? Latter would guarantee that no other package puts data in a bot's own directory? It could reduce the complexity. For developers it could reduce the complexity of maintenance, and for the users/admins the complexity of usage and understanding the system.
After some discussion and testing, the following steps are necessary to turn IntelMQ package into a namespace package. This is supported since Python 3.3.
This is for allowing other third party (namespace) packages (such as the discussed
intelmq-shadowserver
) to tap intointelmq
namespace and provide bots.packages=find_packages()
topackages=find_namespace_packages()
__init__.py
files:intelmq/__init__.py
intelmq/bots/__init__.py
intelmq/bots/*/__init__.py
intelmq/bots/*/*/__init__.py
Basically any subpackage that is possibly sourced from multiple dirs needs to have it's
__init__.py
removed.This is how the experimental namespace package
intelmq
dir tree looks (note the removed__init__.py
files, other files are omitted):This is how the dummy namespace package
intelmq-shadowserver
dir tree looks like (no__init__.py
files either):Having both installed it is possible to do:
This feature is still under consideration, however I wanted to note down my initial experiment and what would need to be done.