Failing resolution of interface implementations when interface and implementation sit in different files

[tomvanschaijk]

The following works perfectly when it's all sitting in one file:

from abc import ABC, abstractmethod
from antidote import implements, inject, interface, world

@interface
class ITask(ABC):
    @abstractmethod
    def do_stuff() -> str:
        pass

@implements(ITask)
class ConcreteTask(ITask): 
    def do_stuff(self) -> str:
        return "ok"

@inject
def f(task: ITask = inject[ITask]) -> ITask:
    return task

some_task = f()
assert some_task is world[ITask]
print(some_task.do_stuff())

however, from the moment I separate this out into different files, for example main.py:

from abc import ABC, abstractmethod
from antidote import implements, inject, interface, world
from example.interface_task import ITask

@inject
def f(task: ITask = inject[ITask]) -> ITask:
    return task

some_task = f()
assert some_task is world[ITask]
print(some_task.do_stuff())

example.interface_task.py:

from abc import ABC, abstractmethod
from antidote import interface

@interface
class ITask(ABC):
    @abstractmethod
    def do_stuff() -> str:
        pass

example.implementation_task.py:

from example.interface_task import ITask
from antidote import implements

@implements(ITask)
class ConcreteTask(ITask):
    def do_stuff(self) -> str:
        return "ok"

and I run main.py, i get the error " Exception has occurred: SingleImplementationNotFoundError No single implementation could match SingleOf(<class 'example.interface_task.ITask'>). File ".....__main__.py", line 22, in some_task = f() ^^^ antidote.lib.interface_ext.SingleImplementationNotFoundError: No single implementation could match SingleOf(<class 'example.interface_task.ITask'>). "

Which is quite annoying of course. I'm working on a bigger application using an architecture as follows:

In which the application layer will have a folder with abstractions, while the concrete implementations will sit in an outer layer, as an implementation detail. If I can not put interface and implementation in different files, the whole idea obviously falls apart. No doubt I'm doing something wrong here, but I can't figure out what. Most of the examples place everything in a single or 2 files, but that's is not going to be viable in a bigger system.

[Finistere]

Hello! Unfortunately, Antidote doesn't do any magic here to discover implementations. The module of ConcreteTask must be imported before you use the interface. You'll need to either load it explicitly with an import ... or eventually by traversing the file tree and importing them with importlib if you have all of your implementations in a predefined module/folder.

[tomvanschaijk]

Thank you for the quick reply. I have to say that's a pity. I really like the library in terms of how relatively unobtrusive it is in your code base (as much as probably possible with a 3rd party library), support for marking interfaces and implementations and so forth. Also, the fact you are not dragging along a container through the different parts of the application is a BIG plus, cause that is something I definitely do not like in a framework like dependency injector. In an architecture like I described in the picture, you would instantiate a container in your outer infrastructure layer (like the api), and you would need to have references to it in your inner layers, requiring references from application and domain to the implementations on the edge, which is exactly what you want to avoid. So all libraries doing this already can be disregarded for this kind of architecture.

I'm currently looking into Kink, and what I want to achieve works there without having explicit references. The thing is: your library allows for scopes like singleton/transient/scoped, which is a necessity in my book. I'm afraid my search has to continue. I have to say, for bigger projects, especially when doing DDD, Python seems to be a struggle. It's quite the handy little glue language and you can get something simple running quickly. For bigger projects though, it's quite the endeavour to tackle all bases. I'm relatively inexperienced for bigger projects using Python, as I have a c# & c++ background, but I must say it's shocking how difficult it seems to be to have "nice things" that are straightforward in many languages I worked with. The one single thing I would change in antidote if I was a dictator, would be the same capability as Kink does, where you can just simple wire up stuff centrally. You basically add registrations to a central container, BUT: you don't need to drag that container along. You just decorate a function (@inject) and it inserts the dependencies. Being honest: antidote does it better than kink, cause where antidote does

@inject
def f(task: ITask = inject[ITask]) -> ITask:
    return task

some_task = f()

kink requires this:

@inject
def f(task: ITask) -> ITask:
    return task

some_task = f()

So in your code, you have the default argument for task being injected because of inject[ITask]. In kink, you don't have that, you just call the function without arguments, sending your IDE into a little bit of a complaint session, and the reader of the code into confusion mode. Quite ugly.

Sorry for the long rant (which isn't really a rant). I just wanted to let you know: awesome library, it's 99% of what I want, and there's a LOT in there that's superior to pretty much any python DI framework I've seen so far, including dependency-injector. I just can't really use it for my use-case. Possibly I'm missing something, I'll go over the docs once more ;-)

Come to think of it, the only addition for the scenario I'm after to work, would be to enable world to be editable, instead of just yielding results. Something like:

world[ITask] = CustomTask()

Possible adding a marker to it to mark the registration as transient/scoped/singleton. To be called in a central function somewhere, some bootstrapper you could call in main. That way, that central location will be responsible for wiring things up. You could even have several locations where you would do this, in different parts of the application, so that dependencies for each layer are taken care of.

This way, you could register dependencies not only by adding decorators, but also explicitely in a bootstrap action. For bigger applications, having a centralised location to do this is quite nice, as it gives you a clear overview. Injecting dependencies can then just work as is. In my mind, that would make it the de facto DI library for python.

[Finistere]

It's quite the handy little glue language and you can get something simple running quickly. For bigger projects though, it's quite the endeavour to tackle all bases. I'm relatively inexperienced for bigger projects using Python, as I have a c# & c++ background, but I must say it's shocking how difficult it seems to be to have "nice things" that are straightforward in many languages I worked with.

I agree. I wouldn't use Python for big projects. The only thing that Antidote could make better here is to provide a method like Pyramid's include example which does the recursive imports for you. Something like:

from antidote import world

# Import all implementations.* modules
world.include('.implementations')

That's something I could add.

But there's no other way around the import, there's no compilation step that would "traverse" all source files. There are different ways to hide it, but at some point, you need to know which modules contain the possible implementations and import them if they weren't already.

You actually can just do:

@inject
def f(task: ITask = inject.me()) -> ITask:
    return task

some_task = f()

@inject is smart enough to infer what you need from the type annotation. The injection must be explicit though. You can create your own wrapper to make it implicit though.

Come to think of it, the only addition for the scenario I'm after to work, would be to enable world to be editable, instead of just yielding results. Something like:

Antidote tries as much as possible for dependencies to be "trackable". It ensures you can always find where they're actually defined. Making world editable goes against that. The interface system is the only exception to this rule. It's only a bit trackable by searching for all text occurrences of @implements(ITask) which isn't great. So yes it could also be an exception for this.

But, your proposal doesn't really change the fundamental problem of your applications needing to be aware of CustomTask existence. Your assignment to world must be executed and its module imported to be taken into account. So at this point, ensuring that world[ITask] = CustomTask() is executed is very close to ensuring that CustomTask is imported at all. The only difference is that an IDE will complain about an unused import, hence why I wanted to add something similar to Pyramid, doing the imports through a method for you.

To be called in a central function somewhere, some bootstrapper you could call in main. That way, that central location will be responsible for wiring things up. You could even have several locations where you would do this, in different parts of the application, so that dependencies for each layer are taken care of.

This would also work for imports. If you have a central location setting up things, doing the recursive imports there isn't really different IMHO.

I just wanted to let you know: awesome library, it's 99% of what I want, and there's a LOT in there that's superior to pretty much any python DI framework I've seen so far, including dependency-injector.

Thanks! :)

[tomvanschaijk]

I see what you mean. Fair point for sure. Actually, the last code snippet is basically what I wanted to achieve, but I got the error no implementations were found. Again, I may be doing something wrong. The thing is, the api "project" I'm running holds the concrete implementation ConcreteTask for example, and the application folder that it references only holds the ITask interface. Just a matter of being able to replace any implementation easily (like swap out a mongodb db for a postgrsql db for example) and have no issues in the application logic part, since it just uses the interface that will have to get implemented by a concrete class in the persistence layer.

[Finistere]

Again, I may be doing something wrong.

You're not, I'm not sure I got the point across. There's no magic way to do this in Python. Until you import a module, Python isn't aware of its content and not even its existence in the first place. Antidote registers implementations through @implements decorator. So at import time usually. There are only two ways to solve it:

• You import the modules containing the implementations individually, and explicitly.
• You load all modules within a folder, more or less the following: https://stackoverflow.com/questions/1057431/how-to-load-all-modules-in-a-folder.

Antidote could simplify the latter by providing the utility function for you as Pyramid does. But even in this case, Antidote would at least need to know where to look for modules. Every dependency injection library will have the same fundamental problem as soon as you separate the implementation from the interface.

While it doesn't change the "import" problem, you could also use a function as an intermediate for registering all implementations:

def register_implementations():
    # Either import ...
    # Or define locally
    @implements(ITask)
    class ConcreteTask(ITask):
        def do_stuff(self) -> str:
            return "ok"

Just FYI, in case it might help, dependencies can be isolated in different catalogs (world is just a static catalog):

from antidote import Catalog

def register_implementations(catalog: Catalog):
    @implements(ITask, catalog=catalog)
    class ConcreteTask(ITask):
        def do_stuff(self) -> str:
            return "ok"

There's one caveat though here, ITask must be defined in the same catalog as the implementation. Implementations cannot cross catalog boundaries.

Example: https://antidote.readthedocs.io/en/stable/changelog.html#id2 Reference: https://antidote.readthedocs.io/en/stable/reference/core.html#antidote.core.Catalog

[Finistere]

I'm closing the issue. Feel free to re-open it if necessary. :)