Does this library have support for qualifiers?

[flisboac]

Hello! I've been reading the documentation, and so far I'm really liking the design of this library! I'm thinking of using it in one of my projects, but I'm not sure if there's any qualifier mechanism in place that I can use. From what I see in the documentation, there's no such functionality, neither in decorators nor in type annotation.

Qualifiers allow for "tag-based" selection of implementations for the same base interface. See Java's CDI for a proper definition. (I think haps also have qualifiers, but the library itself is not as good or well documented as antidote)

Is the tags thing a qualifier? This did not become very clear to me when reading the docs, because there's not much written for tags. I don't know if tag is some sort of name alias to the service, or if multiple services can have the same tag. Can someone confirm?

If not, would it be possible to implement a QualifierProvider or something similar? I reckon I'd need to be able to depend on the other default providers, because a QualifierProvider would be more of an injection filter than a provider itself (considering that it must find all candidates, and exclude from the injection those that are not "qualified").

EDIT: Follows a "quasi"-proposal sort of example:

An example

```py from abc import ABC, abstractmethod from typing import Annotated, Final, List, Optional, TypeVar from dataclasses import dataclass from antidote import Service, ABCService, Provide # # This does not exist, and I'm not sure what substitute there can be for it! # from antidote import qualified, Qualifier, Qualified # Imagine Qualifier implemented as such, with the added restriction that # derived classes must be frozen dataclasses or similar. # It could be a string as well, although checking qualifier hierarchy would not # be possible, or at least not as easy/straightforward. # class Qualifier(ABC): ... # # As an example, suppose I have decorations declared somewhere, indicating the # stage of a data processing pipeline. # @dataclass(frozen=True) class ForEtl(Qualifier): ... # or `ForEtl: Final = "etl"` class ForEtlExtract(ForEtl): ... # or `ForEtlExtract: Final = "etl/extract"` class ForEtlTransform(ForEtl): ... # or `ForEtlTransform: Final = "etl/transform"` class ForEtlLoad(ForEtl): ... # or `ForEtlLoad: Final = "etl/load"` # # Parameter namespace services for each stage. # As more services are added (i.e. as modules are included to the DI container, through imports # or some other mechanism), the DI will add them as candidates. # TParamType = TypeVar('TParamType') class ParameterNamespace(ABCService): @abstractmethod def get(self, name: str, type: Optional[TParamType]) -> Optional[TParamType]: ... @qualified(ForEtl()) # or `@qualified("etl")` class EnvParameterNamespace(ParameterNamespace): PREFIX: Final = 'APP_ETL_' def get(self, name: str, type: Optional[TParamType]) -> Optional[TParamType]: ... @qualified(ForEtlExtract()) # or `@qualified("etl", "etl/extract")` class CloudParameterNamespace(ParameterNamespace): PREFIX: Final = '/app/etl/' def get(self, name: str, type: Optional[TParamType]) -> Optional[TParamType]: ... @qualified(ForEtlTransform(), ForEtlLoad()) # or `@qualified("etl", "etl/transform", "etl/load")` class CustomParameterNamespace(ParameterNamespace): PREFIX: Final = 'param://app/etl/' def get(self, name: str, type: Optional[TParamType]) -> Optional[TParamType]: ... # # A service specialized on abstracting parameter retrieval. # Iterates through each namespace, and returns the first value it has found over all injected namespaces. # class StageParameters(ABCService): @abstractmethod def get(self, name: str, type: Optional[TParamType]) -> TParamType: ... class ExtractStageParameters(StageParameters): def __init__(self, parameter_ns: Annotated[Provide[List[ParameterNamespace]], Qualified[ForEtlExtract]]): self._parameter_ns = parameter_ns # Injected: EnvParameterNamespace, CloudParameterNamespace def get(self, name: str, type: Optional[TParamType]) -> TParamType: ... class TransformStageParameters(StageParameters): def __init__(self, parameter_ns: Annotated[Provide[List[ParameterNamespace]], Qualified[ForEtlTransform]]): self._parameter_ns = parameter_ns # Injected: EnvParameterNamespace, CustomParameterNamespace def get(self, name: str, type: Optional[TParamType]) -> TParamType: ... class LoadStageParameters(StageParameters): def __init__(self, parameter_ns: Annotated[Provide[List[ParameterNamespace]], Qualified[ForEtlLoad]]): self._parameter_ns = parameter_ns # Injected: EnvParameterNamespace, CustomParameterNamespace def get(self, name: str, type: Optional[TParamType]) -> TParamType: ... # # Then I create a transaction script, or command runner, for each step. # class ExtractStageRunner(Service): def __init__(self, parameters: Provide[ExtractStageParameters]): self._parameters = parameters def run(self): timeout = self._parameters.get('timeout', type=str) # Do X ... class TransformStageRunner(Service): def __init__(self, parameters: Provide[TransformStageParameters]): self._parameters = parameters def run(self): timeout = self._parameters.get('timeout', type=str) # Do Y ... class LoadStageRunner(Service): def __init__(self, parameters: Provide[LoadStageParameters]): self._parameters = parameters def run(self): timeout = self._parameters.get('timeout', type=str) # Do Z ... ```

[Finistere]

Hello, glad you like it! :)

Indeed there's nothing similar to what you request currently. There was in a previous version of antidote a "tag" mechanism which had similar behavior to what you're asking. I removed it because the design at the time wasn't really up to the "standard" of antidote and it didn't feel necessary enough. It's still possible to implement it through a Provider as mentioned and probably a stateful factory. Neither are as friendly as exposing a proper API for it though obviously.

Since you ask for this and #32 has a similar need I'm considering reworking/adding this functionality back though. I'm unsure how the API should work to ensure that when asking for a specific qualifier ForEtl you actually only retrieve objects of a specific type. There should be no surprises. Typically in your API example, Antidote would need to infer it from the signature and check at runtime the correctness, like @implementation does, or use it as a filter. Both are complex behaviors for an API.

You talk about qualifier hierarchy in your code example, is it something you actually need? From a different perspective, your example looks like what I'd call a simple plugin system. There's a ParameterSource interface and variants of it which should be used depending on the context which could look like this:

from typing import *

from antidote import inject, Plugin, service

# Used as an interface
class ParameterNamespace(Plugin):
    def get(self, name: str):
        raise NotImplementedError()

# Define the qualifiers
EXTRACT = ParameterNamespace.qualifier()
TRANSFORM = ParameterNamespace.qualifier()
LOAD = ParameterNamespace.qualifier()

@EXTRACT
@service
class EnvParameterNamespace(ParameterNamespace):
    pass

@TRANSFORM
@service
class CloudParameterNamespace(ParameterNamespace):
    pass

@TRANSFORM
@LOAD
@service
class CustomParameterNamespace(ParameterNamespace):
    pass

@inject([ParameterNamespace.matching(TRANSFORM)])
def transform_parameters(parameter_namespaces: List[ParameterNamespace]):
    pass

@inject([ParameterNamespace.all()])
def all_parameters(parameter_namespaces: List[ParameterNamespace]):
    pass

Would a similar API solve your issue? Or am I missing something?

As a side note, just for your information, in this case, intuitively I'd use a Constants class to act as an interface for the configuration:

class ExtractStageParameters(Constants):
   # specifying `int` here will force the type coercion (auto_cast feature)
    timeout = const[int]()

    @inject({"parameter_namespaces": ParameterNamespace.matching(TRANSFORM)})
    def provide_const(self, name: str, arg: Any, parameter_namespaces: List[ParameterNamespace]) -> Any:
        # Find first matching parameter or raise an error.
        for p in parameter_namespaces:
            value = p.get(name)
            if value is not None:
                return value
        raise RuntimeError()

See https://antidote.readthedocs.io/en/stable/recipes.html#configuration.

EDIT: Added @service in code example. On the top of my head, those are orthogonal issues: how to provide the plugin (factory, service) and define the plugin. In all cases this is just a draft I came up with.

[flisboac]

Thanks for the comprehensive answer!

Regarding your suggestion, I think it looks really good! Tying a qualifier to the interface it qualifies is a nice touch, and it makes complete sense. You lose a bit of flexibility, but in turn can ensure that the interface is respected among all implementations. Some part of antidote's code can then verify if the qualifiers come from one of the implementation's base classes, at decoration time. It'll also remove any doubts regarding where a qualifier can be applied.

I would only ask for it to be possible to match more than one qualifier (i.e. ParameterNamespace.matching(TRANSFORM, LOAD), or something like that). That was not uncommon for me in codebases I worked in the past, and I see that it may become a necessity in the codebases I'm working on today. The logic would be for it to select implementations that match at least one of the requested qualifiers.

Regarding the qualifier hierarchy... It was a loose and badly formulated suggestion, I admit. I had some wild ideas about it (like parameterization, and/or using type hierarchy to verify qualification), but it's not necessary at all. I'll refrain from elaborating on it any further, so that we don't lose time. :P

I don't really have a strong opinion regarding the naming for this new functionality. If that Plugin functionality merges in, I'll be more than happy! But I do think it would eventually rub users the wrong way, or give them a confusing initial perception. A plugin is a mechanism that most people consider to be much more complex than just qualifying an injection. Generally you talk about registry, extension points, and whatnot. They may start looking for something that's never been supposed to be there (e.g. may regard it as extending Antidote itself). Can't the Service ABC be extended to support qualifiers instead (because a Plugin is just an ABCService, and qualifiers won't introduce that much functionality to justify an entirely new injectable type, I think)?

In any case, if I can have something resembling qualifiers, it'll seal the deal on the library for me, and I'll be able to pitch it to my coworkers! :D

But out of curiosity... How would I implement that QualifierProvider? As a stateful provider, is it possible to guarantee a provider's order of execution in the Container's internal algorithm? Does a stateful provider run after stateless ones, or something like that?

P.S.: Thanks also for the clarification on Constants. It didn't even cross my mind that I could use it, especially in this manner.

[Finistere]

I agree. You're right, Plugin isn't a good name. In all cases this API will probably be in experimental package or something similar, I don't want to expose it as stable yet.

For now I'm considering using Component instead. I'm also not really satisfied of polluting the class namespace with several methods specific to Antidote... Using an nested class as namespace could be a solution or not requiring inheritance at all like in the second example. If you have an opinion, I'm all ears!

API designs

```python from typing import * from antidote import world, Component, service # Used as an interface class Object(Component): def method(self) -> Any: raise NotImplementedError() # Define the qualifiers FIRST_TAG = Object.Def.qualifier() SECOND_TAG = Object.Def.qualifier() @FIRST_TAG @service class FirstObject(Object): pass @SECOND_TAG @service class SecondObject(Object): pass world.get(Object.Get.qualified_by(FIRST_TAG)) world.get(Object.Get.all()) ``` ```python from antidote import component class Object: pass @FIRST_TAG @service class FirstObject(Object): pass FIRST_TAG = component(Object).qualifier() world.get(component(Object).qualified_by(FIRST_TAG)) ```

In the meantime, here is an implementation of qualifiers using a Provider which is a stable API. See https://antidote.readthedocs.io/en/stable/extend.html. If you need this feature now for production code, that would be my suggestion:

from __future__ import annotations

import inspect
from typing import Hashable

from antidote import inject, service, world, Provide
from antidote.core import Provider, Container, DependencyValue

class AnyQualifier:
    pass

# Everything is thread-safe by default.
@world.provider
class QualifierProvider(Provider[AnyQualifier]):
    def __init__(self):
        super().__init__()
        self.__qualified_dependencies: dict[Qualifier, list[Hashable]] = dict()

    # Used for test isolation
    def clone(self: QualifierProvider, keep_singletons_cache: bool) -> QualifierProvider:
        provider = QualifierProvider()
        provider.__qualified_dependencies = {
            qualifier: dependencies.copy()
            for qualifier, dependencies in self.__qualified_dependencies
        }
        return provider

    # Used to ensure that no one else provides this
    def exists(self, dependency: Hashable) -> bool:
        if isinstance(dependency, Qualifier):
            return dependency in self.__qualified_dependencies
        if isinstance(dependency, MultiQualifier):
            return all(qualifier in self.__qualified_dependencies
                       for qualifier in dependency.qualifiers)
        return False

    # Does the actual job of returning dependencies
    def provide(self, dependency: AnyQualifier, container: Container) -> DependencyValue:
        # exists() is always called before provide()
        if isinstance(dependency, Qualifier):
            dependencies = self.__qualified_dependencies[dependency]
        else:
            # sanity check
            assert isinstance(dependency, MultiQualifier)
            dependencies = [
                dep
                for qualifier in dependency.qualifiers
                for dep in self.__qualified_dependencies[qualifier]
            ]
        return DependencyValue(
            value=[container.get(d) for d in dependencies]
        )

    def register(self, qualifier: Qualifier, dependency: Hashable):
        # sanity check
        if not isinstance(qualifier, Qualifier):
            raise TypeError(f"qualifier must be a Qualifier, not a {type(qualifier)}")
        # Ensure no one else provides this dependency already.
        if qualifier not in self.__qualified_dependencies:
            self._assert_not_duplicate(qualifier)
        self.__qualified_dependencies.setdefault(qualifier, []).append(dependency)

#
# This part is a bit complex for the sake of having a nice API & multi qualifier support
#
class Qualifier(AnyQualifier):
    @inject
    def __call__(self, cls: type, provider: Provide[QualifierProvider] = None):
        # need a cleaner way to express to mypy that provider is not required.
        assert provider is not None
        assert isinstance(cls, type) and inspect.isclass(cls)  # sanity check
        # provider MUST NOT be given inside __init__() as this will break test isolation.
        # One should never cache a provider.
        provider.register(self, cls)
        return cls

    def __or__(self, other: MultiQualifier | Qualifier) -> MultiQualifier:
        if isinstance(other, MultiQualifier):
            return MultiQualifier(other.qualifiers + [self])
        if isinstance(other, Qualifier):
            return MultiQualifier([other, self])
        raise TypeError("other must be a list of Qualifier or a Qualifier, ")

class MultiQualifier(AnyQualifier):
    def __init__(self, qualifiers: list[Qualifier]):
        self.qualifiers = qualifiers

    def __or__(self, other: MultiQualifier | Qualifier) -> MultiQualifier:
        if isinstance(other, MultiQualifier):
            return MultiQualifier(other.qualifiers + self.qualifiers)
        if isinstance(other, Qualifier):
            return MultiQualifier(self.qualifiers + [other])
        raise TypeError("other must be a list of Qualifier or a Qualifier, ")

#
# User code
#

# Define the qualifiers
EXTRACT = Qualifier()
TRANSFORM = Qualifier()
LOAD = Qualifier()

class ParameterNamespace:
    def get(self, name: str):
        raise NotImplementedError()

@EXTRACT
@service
class EnvParameterNamespace(ParameterNamespace):
    pass

@TRANSFORM
@service
class CloudParameterNamespace(ParameterNamespace):
    pass

@TRANSFORM
@LOAD
@service
class CustomParameterNamespace(ParameterNamespace):
    pass

@inject([TRANSFORM])
def transform_parameters(parameter_namespaces: list[ParameterNamespace]):
    return parameter_namespaces

print(world.get(TRANSFORM))
# [CloudParameterNamespace, CustomParameterNamespace]

print(world.get(EXTRACT | LOAD))
# [CustomParameterNamespace, EnvParameterNamespace]

[Finistere]

But out of curiosity... How would I implement that QualifierProvider? As a stateful provider, is it possible to guarantee a provider's order of execution in the Container's internal algorithm? Does a stateful provider run after stateless ones, or something like that?

There's no guarantee on the ordering of the providers, because there's no need to. They're meant to handle strictly separate dependencies. Providers manage a set of dependency declarations, not the actual instances/dependency values. The latter are handled by the Container. This makes adding new kinds of dependencies relatively easy. Everything in Antidote relies on a Provider underneath. One only needs to specify three methods and usually don't need to worry about thread-safety:

• clone(): return a copy, used for test isolation. (I probably should have named it copy())
• exists(): used to know whether this dependency is managed by the provider
• provide(): does the actual job of creating the instance/dependency value

Regarding stateful vs stateless, I think you're mixing with "probably a stateful factory" I said in the first comment. Stateful factories are supported in Antidote, but it would have been a big hack to use them for this. It was a mistake to say this. It's really the job of a Provider here.

[flisboac]

First of all, thank you immensely for the Qualifier implementation using providers! I'll lean on it until true qualifiers are implemented! :D

I think the second proposal is better because it's less invasive. Even though it doesn't follow the library's philosophy of preferring base classes, I think it is for the better. But if we go the base class route (ABC or not; or even a mix of the two proposals), perhaps a @qualifier decorator could also be offered, to be used when base-class conflicts arise (much like the relationship between Service, ABCService and @service).

I also think component is the perfect name for what you're going for in the second proposal. As a matter of fact, Component is the name for the "injectable" concept in Spring, that is, anything that's injectable, and can be detected by the DI container. This does not correspond completely to the same concept in Antidote (because e.g. Service, ABCService, etc., would not be contemplated by that component helper), but it's a close enough to warrant the name. Some other DI frameworks also use this terminology (I've seen it being thrown out in CDI discussions, etc).

Well, what if component was a decorator instead? For example:

from antidote import component, Component, ABCComponent

# To allow for such an usage, `component` will most probably
# be a callable object.

# You can do this...
@component
class Object:
    pass

# ... Or, alternatively:
class Object(Component):
    pass

# ... Or, alternatively:
class Object(ABCComponent):
    pass

# Then, `component.of(Object)` is only allowed if:
# 1. `Object` extends either `ABCComponent` or `Component`; or
# 2. `Object` is decorated with `@component` (in which case it will have
#     some metadata associated with it, and stored either globally, or
#     embedded in the class itself).
FIRST_TAG = component.of(Object).qualifier()

@FIRST_TAG
@service
class FirstObject(Object):
    pass

world.get(component.of(Object).qualified_by(FIRST_TAG))

---

By the way... What do you think of parameterized qualifiers? A parameterized qualifier would be much like a NamedTuple, in that:

1. It is frozen by the time it's created;
2. It can have fields, and its values must only contain stably-hashable objects (probably only literals, or hashable-frozen objects);
3. Instances of qualifiers are compared like tuples, i.e. field by field, using ==.
4. Hashing is done with all fields.
5. OR-ing would generate a QualifierSet (or QualifierSpec?), which is just a set of qualifiers. You would use qualifier_a | qualifier_b in a qualified_by method (which can receive either an individual qualifier, or a QualifierSet.

By allowing for parameterization, we avoid the need for users to create a multitude of qualifiers for the same "qualifier kind." A decorator could create a frozen dataclass for us, and enforce invariants, constraints and/or implementation details.

For example:

from antidote import component, qualifier

# The previous example snippet does not conflict with this one.
# You could, in theory, keep the idea of creating "qualifier tokens"
# with `component.of(Object).qualifier()`.
# These would be only equal to themselves, as expected.
# Qualifier types, as opposed to tokens, would be compared in a
# smarter way, i.e. the `is` operator is not applicable.

@component
class LogSink:
    pass

# Instances of `LogSinkOn` are immutable, and compare like a tuple.
# For a first release, the decorator can do some validation to ensure 
# that attributes are decorated only with built-in types.
@component.qualifier(LogSink)
class LogSinkOn:
    name: str

@LogSinkOn("prd")
@LogSinkOn("stg")
class SyslogLogSink:
    pass

@LogSinkOn("stg")
@LogSinkOn("dev")
class ConsoleLogSink:
    pass

world.get( component.of(LogSink).qualified_by(LogSinkOn("prd") | LogSinkOn("stg")) )

[Finistere]

Thank you very much for your comments, it helps a lot to put into perspective what Antidote does and what it should do. :)

I think the second proposal is better because it's less invasive.

I agree. Service and ABCService might be too invasive and complex in hindsight. It's currently only strictly needed for parametrization of the service (https://antidote.readthedocs.io/en/stable/recipes.html#parameterized-service-factory) which could be done in a similar way parametrized[MyService](...) instead of MyService.parameterized(...) which may provide better typing experience with PEP612. I don't see an alternative for Factory though, the @ operator would be missing otherwise. And adding it after class creation is not explicit.

Obviously, any changes would be done in a backward-compatible way.

@component
class Object:
    pass

While decorating would probably be of no use for the implementation it's probably a good idea to require it to enforce explicit code.

Can't the Service ABC be extended to support qualifiers instead (because a Plugin is just an ABCService, and qualifiers won't introduce that much functionality to justify an entirely new injectable type, I think)?

I'm intuitively against because for two reasons:

1) I think they're orthogonal issues, @component in this issue is for me about providing certain implementations for a specific interface based on some conditions. Somewhat similar to what @implementation tries to do. 2) a component might be provided by a factory or another kind of dependency, it's not necessarily a service as defined in Antidote.

Maybe @contract / @interface would be better instead of @component?

By the way... What do you think of parameterized qualifiers?

My main problem with it is worse developer experience: searching for the usage of LogSinkOn("prd") can only be done through text search. And with multiple arguments, different ordering, etc.. it can quickly become very hard to search for. Searching for the usage of a variable TRANSFORM is a lot cleaner and safer with any IDE.

So simple parameterized qualifiers have worse DX. And the only usage I see of complex ones is when Antidote is used as a foundation for another framework. I'm aware of a need to choose one (or multiple) components based on some conditions and/or priorities for a framework. So IMO Antidote needs here to provide a very generic approach to provide logic for selecting one/multiple implementation(s) for an interface for the complex cases. It would be in the middle ground between the full flexibility of a Provider and the "standard" API (service, factory...) meant for ease of maintenance.

Also, if there are other things you like or not in Antidote, I'd love to hear it! Thank you again for taking the time to respond with detailled answers!

[flisboac]

Hello, @Finistere ! I'm truly sorry for taking so long to answer... :(

About the parametrized[MyService](...) idea, I fully agree with you, and like the syntax. And for me at least, if such a change lands, it would be really great! I think subclassing should be entirely on the hands of the developer, and we should force as little of it as possible. But how hard would it be for us to implement such functionality? Would we need to rewrite how class lookup works, or something? Or was it just to elaborate more on my comments, and nothing of the sort is planned?

Here be digressions

What I find confusing about ABC classes is that they may mess with `isinstance`. For example, I find this quite strange: ```py from abc import ABC class SomeService(ABC): pass class SomeServiceImpl(SomeService): pass service = SomeServiceImpl() print(isinstance(service, ABC)) # ==> True (but why, though?) ``` A much better idiom I've seen is: ```py from abc import ABCMeta class SomeService(metaclass=ABCMeta): pass class SomeServiceImpl(SomeService): pass service = SomeServiceImpl() print(isinstance(service, ABC)) # ==> False (much better!) ``` ... But metaclasses are scary stuff, and none of it matters because you won't go around checking if something is an instance of `ABC` anyway (you'd do that only when the object is a class, and via `issubclass`)!

---

I think @component acting as a marker is a good thing too, to indicate that the class/language element somehow makes part of, or is integrated with, the DI mechanism. I'd say it's even more evident than subclassing, just because decorators are visually much more distinct, and not as common for normal (user) classes (specifically; except for dataclasses, ofc).

About items 1 and 2... Fair enough. But again, if the importance of subclassing is somehow downplayed in the implementation of indirect providers, that makes even more sense.

About the @contract/@interface suggestion, I think it's a better name scheme. I'd personally go for @interface, it makes a lot of sense in my mind, coming from staticlang world. :D

---

Now, about parameterized qualifiers:

My main problem with it is worse developer experience: searching for the usage of LogSinkOn("prd") can only be done through text search.

That's a really good point.

Speaking outside Antidote and Python in general, for a bit... Some languages and frameworks have some IDE support or some tooling for qualifier discovery and search, and it truly helps with finding e.g. named entities, named services, etc., without having to declare a qualifier token for each possible combination of parameters in an application (which may be infeasible without parameterization).

All of that search intelligence is nice and all, but I admit that tracing some variable's usage across a code base is a much easier way to achieve a very similar effect, and it works today. AND that's something the user can do, too; we don't need to enforce qualifier tokens (unless it becomes painful to implement parameterization on Antidote's side, which I don't think is necessarily the case). In fact I think it'd be much better for the user to have some flexibility (i.e. a more feature-rich qualifier implementation), because Antidote could be used as the foundation for something bigger (e.g. an internal framework).

Follows yet another suggestion:

from antidote import component, qualifier

@component
class LogSink:
    pass

@component.qualifier(LogSink)
class LogSinkOn:
    name: str

# "Tokenization" by hand
# Usage of these tokens (instead of instantiating a new qualifier)
# could be enforced by convention, internally (e.g. code reviews).
LogSinkOnDev = LogSinkOn("dev")
LogSinkOnStg = LogSinkOn("stg")
LogSinkOnPrd = LogSinkOn("prd")

# Optionally, `component.qualifier` could augment the class
# with a `freeze` method, to prevent further instantiation.
LogSinkOn.freeze()

@LogSinkOnPrd
@LogSinkOnStg
class SyslogLogSink:
    pass

@LogSinkOnStg
@LogSinkOnDev
class ConsoleLogSink:
    pass

world.get( component.of(LogSink).qualified_by(LogSinkOnPrd | LogSinkOnStg) )

If that's not enough... As a compromise, and abandoning the parameterization idea, could we give the user the option to create custom qualifiers (i.e. non-tokens) in the form of Python Enums? In this case, component.qualifier would add a __call__ to the enum class, so that elements could be used as decorators as well. For example:

from antidote import component, qualifier, world
from enum import Enum

class EnvironmentType(Enum):
    DEV = 'dev'
    STG = 'stg'
    PRD = 'prd'

@component
class LogSink:
    pass

@component.qualifier(LogSink)
class LogSinkOn(Enum):
    # Here I use another enum as `value` (which in turn has `str` as value),
    # but by being an enum, `LogSinkOn` could be customized to have as many
    # fields as necessary in its value, and/or provide custom properties in the
    # enumeration class for easy access (i.e. without passing through the 
    # `value` property, e.g. `LogSinkOn.DEV.field` instead of
    # `LogSinkOn.DEV.value.field`).
    DEV = EnvironmentType.DEV
    STG = EnvironmentType.STG
    PRD = EnvironmentType.PRD

    # As with any enum, they could have custom properties as well.
    @property
    def env_name(self): return self.value.value

@LogSinkOn.PRD
@LogSinkOn.STG
class SyslogLogSink:
    pass

@LogSinkOn.STG
@LogSinkOn.DEV
class ConsoleLogSink:
    pass

world.get( component.of(LogSink).qualified_by(LogSinkOn.PRD | LogSinkOn.STG) )

In any case, my aim is to use Antidote as a base for an internal framework, hence why I was thinking about qualifier parameterization. :p Use cases like that have already appeared in some of our plannings.

About implementing some middle-ground functionality for customized injection condition, it would be great too. I will try to think about something.

---

P.S.: While I was writing this, a question came to my mind. How does Antidote discovers injectables? Does it look into __subclasses_? (Like... What if I try to inject an "interface", but do not import (either directly or indirectly) any implementation?)

[Finistere]

Sorry, it's taking me a long time to respond now... ^^" Unfortunately won't have the time to work on Antidote until February. I'll come back to you at that time with a more detailed answer. Here's a short answer to some of your points:

About the parametrizedMyService idea, I fully agree with you, and like the syntax. And for me at least, if such a change lands, it would be really great! I think subclassing should be entirely on the hands of the developer, and we should force as little of it as possible. But how hard would it be for us to implement such functionality? Would we need to rewrite how class lookup works, or something? Or was it just to elaborate more on my comments, and nothing of the sort is planned?

I started implementing it already. :) Service classes should probably be deprecated, they just don't bring anything. I wanted them for easier type-safety and thought they would be better for Antidote's maintainability. But I was wrong and it hurts the developer experience... I also think that I should embrace protocols a bit more.

In any case, my aim is to use Antidote as a base for an internal framework, hence why I was thinking about qualifier parameterization. :p Use cases like that have already appeared in some of our plannings.

Antidote being used as a building block for a framework is something I want to improve, so I'm definitely coming back to you on your proposals :)

P.S.: While I was writing this, a question came to my mind. How does Antidote discovers injectables? Does it look into _subclasses? (Like... What if I try to inject an "interface", but do not import (either directly or indirectly) any implementation?)

Antidote doesn't need to discover them currently, because the API forces you to import them. For @factory and @implementation you need to specify the function, the source, creating the dependency when using it:

For factory for example:

from typing import Annotated
from antidote import factory, world, inject, From

class Database:
    pass

@factory
def load_db() -> Database:
    return Database()

from antidote import world, inject

world.get(Database @ load_db)
# with typing hints
world.get[Database](Database @ load_db)
world.get[Database] @ load_db

@inject([Database @ load_db])
def f(db: Database):
    pass

@inject
def f(db: Annotated[Database, From(load_db)]) -> Database:
    return db

Source: https://antidote.readthedocs.io/en/stable/reference.html#antidote.factory.factory

So I'd like to keep as much as possible this design for features. If not possible, I'd probably need to add something like Pyarmid's config.include (https://docs.pylonsproject.org/projects/pyramid/en/latest/api/config.html#pyramid.config.Configurator.include) to let people import dependency definitions. I probably wouldn't use __subclasses_(), it's not explicit enough. Also tying Antidote's dependency chain to the inheritance chain is probably also not for the best.

[flisboac]

@Finistere

Unfortunately won't have the time to work on Antidote until February. I'll come back to you at that time with a more detailed answer.

Understandable, not a problem at all, my friend. I can wait! :)

I started implementing it already. :)

You mean the decorator-based Service (in deprecation of Service as a mandatory base class)? If so, that's great news! Regarding the scope of this issue, I didn't even request that, but I really welcome the change anyway! :D

Antidote doesn't need to discover them currently, because the API forces you to import them.

I understand. My question was more directed towards the indirect (no pun intended) provider.

And now for even more digressions, now regarding indirect providing

Reading the documentation again, and some of the source-code, I now have a better understanding of the overall mechanism used by Antidote. Not only the factory provider, but also the indirect provider, requires you to specify a factory function of sorts. I believe it must work this way because it's not possible to guarantee that the implementation classes are loaded (as in imported from modules and made available to the injector/app) unless the app import them at some point. Those factories guarantee that, and shifts the selection of implementation candidates to the user. It's a really smart decision, and I'm quite impressed. Such a design forces your injection site (e.g. your app/lib) to indirectly depend on a predefined set of implementations. In a way, you still have the benefits of dependency injection, but the lack of service discovery makes implementing auto-provided library services a bit harder. I'll try to illustrate what I mean. #### About multi injection Back to the logger domain. As an example, suppose I have a "core" library declaring an interface `LoggerSink`. Then I may want to offer to users different implementations of a log sink, that will be injected by means of some indirect factory. An implementation would look like this:

Code snippet: On library "my_logging_core" (with basic logging framework definitions)

```py # # On library "my_logging_core" (with basic logging framework definitions) # from abc import ABCMeta, abstractmethod from typing import Callable, List, Optional, Sequence, Set, Tuple, Type, TypeVar, Union from typing_extensions import TypeAlias from antidote import world, Service, Provide class LogSink(Service, metaclass=ABCMeta): """Sends logs somewhere.""" @abstractmethod def log(self, message: str) -> None: ... class Logger(Service, metaclass=ABCMeta): """Orchestrates log sinking.""" def log(self, message: str) -> None: ... TIndirect = TypeVar('TIndirect', covariant=True) TIndirectFactory: TypeAlias = Callable[ ..., Union[ TIndirect, List[TIndirect], Tuple[TIndirect], Set[TIndirect], ], ] TIndirectFilter: TypeAlias = Callable[[TIndirect, TIndirectFactory], bool] def multi_inject( indirect_type: Type[TIndirect], *factories: TIndirectFactory, # Not necessary at all, but a possibility when: Optional[TIndirectFilter] = lambda: True, required: Optional[Union[bool, int]] = False, ) -> Sequence[TIndirect]: expected_typename = indirect_type.__name__ def injector(): selected: List[TIndirect] = [] for factory in factories: candidates = world.get[indirect_type](indirect_type @ factory) if not isinstance(candidate, (list, tuple, set)): if not isinstance(candidates, indirect_type): typename = type(candidates).__name__ raise ValueError( f'Multi-injection candidate must be an instance of "{expected_typename}", got "{typename}".' ) candidates = [candidates] for candidate in candidates: if when(candidate, factory): selected.append(candidate) # The assertions could be better, but should give an idea of what I mean if required is True: assert len(selected) > 0, \ 'Multi-injection of "{expected_typename}" is required, but no suitable candidate was found' elif isinstance(required, int): assert len(selected) >= required, \ 'Multi-injection of "{expected_typename}" is required,' \ ' but not enough suitable candidates were found (requires at least {required})' return tuple(selected) return injector ```

Code snippet: On library "my_logging_remote" (with sinks for some external logging service/api)

```py # # On library "my_logging_remote" (with sinks for some external logging service/api) # from antidote import implementation, Service from my_logging_core import LogSink class CloudwatchLogSink(Service, LogSink): ... class LogstashLogSink(Service, LogSink): ... # Don't want to complicate the snippet any more than it already is. :x # Please imagine here some sort of configuration injection, and all that. @implementation(LogSink, permanent=True) def remote_log_sink_inject() -> Sequence[LogSink]: return (CloudwatchLogSink(), LogstashLogSink()) ```

Code snippet: On library "my_logging_os" (with sinks specialized on services in the local OS)

```py # # On library "my_logging_os" (with sinks specialized on the local OS) # from antidote import implementation, Service from my_logging_core import LogSink class SyslogLogSink(Service, LogSink): ... # Don't want to complicate the snippet any more than it already is. :x # Please imagine here some sort of configuration injection, and all that. @implementation(LogSink, permanent=True) def os_log_sink_inject() -> Sequence[LogSink]: return (SyslogLogSink(),) ```

Code snippet: On library "my_logging_fd" (with sinks specialized on file descriptors)

```py # # On library "my_logging_fd" (with sinks specialized on file descriptors) # from antidote import implementation, Service from my_logging_core import LogSink class ConsoleLogSink(Service, LogSink): ... class RotatingFileLogSink(Service, LogSink): ... @implementation(LogSink, permanent=True) def fd_log_sink_inject() -> Sequence[LogSink]: return (ConsoleLogSink(), RotatingFileLogSink()) ```

Code snippet: On library "my_logging" (the entrypoint for the logging framework)

```py # # On library "my_logging" (the entrypoint for the logging framework) # from typing_extensions import TypeAlias, Annotated from antidote import implementation, Service, From from my_logging_core import LogSink, Logger from my_logging_remote import remote_log_sink_inject from my_logging_os import os_log_sink_inject from my_logging_fd import fd_log_sink_inject class DefaultLogger(Service, Logger): """Orchestrates log sinking.""" def __init__(self, sinks: Sequence[LogSink]): self._sinks = sinks def log(self, message: str) -> None: for sink in self.sinks: sink.write(message) all_log_sinks = multi_inject( fd_log_sink_inject, os_log_sink_inject, remote_log_sink_inject, ) ProvidedAllLogSinks: TypeAlias = Annotated[Sequence[LogSink], From[all_log_sinks]] @implementation(Logger, permanent=True) def default_logger(sinks: ProvidedAllLogSinks) -> Logger: return DefaultLogger(sinks) ProvidedLogger: TypeAlias = Annotated[Logger, From[default_logger]] ```

Code snippet: On a final app (i.e. actually using the "logging framework")

```py # # On a final app # from antidote import inject from my_logging import ProvidedLogger @inject def main(logger: ProvidedLogger): # ... do some work logger.log('done!') # would log to many different sinks (or less) main() ```

(I haven't tested this code, but I suppose it should work. Please correct me if I'm wrong!) That is all completely fine for smaller apps, domains or teams. But as those increase in size or scope, or are further broken down, maintaining such a tight grip of dependencies at the library level (in my example, through `all_log_sinks`, or similar mechanisms) can become increasingly harder. As a library or framework author, the more freedom you can give the user, the better. I agree we should not suggest bad practices, or overwhelm the user with options, but in this case specifically, a better service discovery may allow those authors to loosen coupling even more, and allow more decentralization whilst maintaining a sane baseline (in the form of interfaces, and similar elements). Concerning this scenario, one could say that the app developer already have the freedom he needs, but that freedom comes at the cost of unnecessary complexity. For example, he could reuse the `multi_inject` mechanism to select exactly which sinks he wants to use, and then construct the `Logger` himself:

Code snippet: On a final app with customized log sink factories

```py from antidote import inject, world from my_logging import DefaultLogger from my_logging_core import LogSink, Logger, multi_inject from my_logging_os import os_log_sink_inject from my_logging_fd import fd_log_sink_inject app_log_sinks = multi_inject( fd_log_sink_inject, os_log_sink_inject, ) @inject def main(): # Note that I cannot reuse the default logger factory, because I want to use a # different set of sinks than the defaults provided by the statically specified # indirect provider: # >>> logger = world.get[Logger](Logger @ default_logger) # <<< (Not possible!) # Because of that, we need to construct a logger by hand, or make a new factory. # This requires us to have knowledge of some implementation, and hurts # testability in a way (because dependency is now fixed). # The need for using world inside a function that's in principle auto-wired # does not look very good (also, the same can be said about `default_logger`). logger_sinks = world.get[LogSink](LogSink @ app_log_sinks) logger = DefaultLogger(sinks) # Now, the rest of the app follows as before. # ... do some work logger.log('done!') # would log to 4 different sinks (or less) main() ```

This use case could be handled better, and by Antidote itself. What if we had a concept in Antidote for a collection of provider elements (Service, Component, Factories, Indirects, etc), much in the same vein as [Modules in NestJS](https://docs.nestjs.com/modules) or [Modules in AngularJS](https://angular.io/guide/architecture-modules)? The idea is for a library author to provide said collection, and it's up to the user to select which of those collections he wants to include in the auto-injection mechanism. I would call such a collection a **catalog** (as in service catalog, etc), so that it does not conflict with Python's concept of a module. Also, I think catalog is a better description of the intention of that functionality. NOTE: The same argument could be applied to single-element (but likewise indirect) injections too. #### ... And yet another proposal! A catalog could be declared as a simple Python module with some well-known exported, non-dunded properties. As of now, from their NestJS and Angular concept counterparts, the only property we should care about is `exports` and `imports`. `exports` is mandatory, and would specify the injectables that will be considered when no injection is provided by the other providers. Injectable type could be deduced by how the element is decorated (e.g. `@implements`, inheriting `Service`; possibly with some helper to determine which kind of element it is. `imports` would be used for when a library needs to include the injectables of another from inside this catalog mechanism. It would incur some degree of indirection, but nothing that could be e.g. easily followed in any IDE by simple CTRL+click, etc. Also, `imports` is entirely optional, and would just add the imported catalogs' exports (transitively) into the set of possible injections. (I guess this would be slower than the current mechanism, but not by much, as long as the chain is not too long, and the candidate set is not too big; perhaps some of the work can be optimized by pre-calculating the entire candidate set on Cython?) This would allow Antidote to keep the design philosophy of "explicit is better that implicit," because it would be clear for the user (from both library and app perspective) from where the injection candidates are coming from. Expanding on the previous example (supposing we refactor it appropriately):

Code snippet: On module "my_logging_remote.catalog"

```py from .impl import remote_log_sink_inject, CloudwatchLogSink, LogstashLogSink exports = (remote_log_sink_inject, CloudwatchLogSink, LogstashLogSink) ```

Code snippet: On module "my_logging_os.catalog"

```py from .impl import SyslogLogSink, os_log_sink_inject exports = (SyslogLogSink, os_log_sink_inject) ```

Code snippet: On module "my_logging_fd.catalog"

```py from .impl import ConsoleLogSink, RotatingFileLogSink, fd_log_sink_inject exports = (ConsoleLogSink, RotatingFileLogSink, fd_log_sink_inject) ```

Code snippet: On module "my_logging.catalog"

```py from .impl import DefaultLogger, default_logger exports = ( DefaultLogger, default_logger, ) ```

Code snippet: On the final app (default)

It stays the very same way. ```py from antidote import inject from my_logging import ProvidedLogger @inject def main(logger: ProvidedLogger): # ... do some work logger.log('done!') # would log to many different sinks (or less) main() ```

Code snippet: On the final app (with custom log sink providing)

```py from antidote import inject from my_logging_fd import catalog as fd_catalog from my_logging_os import catalog as os_catalog from my_logging import catalog, ProvidedLogger @inject(catalogs=(catalog, fd_catalog, os_catalog) def main(logger: ProvidedLogger): # ... do some work logger.log('done!') # would log to many different sinks (or less) main() ```

Note that this is all mostly about indirect providing. By this design, perhaps a new `CatalogProvider` could be implemented, to cover the case when a provider is not immediately offered by the other providers. This would keep compatibility with the current injection mechanism, and offer an alternative for when a greater degree of control of indirection is needed. Libraries can have as many catalogs as needed, or offer some catalog factory to customize provisioning (much in the same way the very common [`Module.forRoot` pattern in Angular](https://angular.io/guide/singleton-services#the-forroot-pattern) works). Note that this is on top of the qualifiers for which this issue is for. Catalogs would be there to provide entire sets of injectables, and in some way filter them, at the container level (i.e. coarser control). Qualifiers would be used to slice and/or filter injection at the injection point level (i.e. more granular control). Also, contrary to modules in NestJS or Angular, there would be no isolation between catalogs, in the sense of internal vs external injectables (i.e. no "privates"), as I guess it would be quite hard to implement something like that and it would also not bring as much benefit. One disadvantage of this approach is that I don't see an obvious way to alter `world` to make it also work with catalogs when injecting: - Could the concept of sub-container be introduced (i.e. like `clone()`, but using parent setup to look up injections, hence propagating the catalog set downstream)? - Or should `world.get()` also accept catalogs, and each injection would be distinct from each other in terms of lookup? - And in that case, what if I have to programmatically inject something? (NestJS solved this with injections of so-called [ModuleRefs](https://docs.nestjs.com/fundamentals/module-ref), which in this proposal would be something more akin to `ContainerRef`s, and return a customized "world") Those questions prevented me from trying to implement a new Provider. I needed to check the feasibility of such an idea, hence why I'm walltexting (I'm sorry) :x ... It took me literal hours to write all this, and I don't even know if the example I gave is the best or more succint one I could give... But if needed, I can elaborate further. What do you think? Is this a good idea? If not, please do tell!

---

In any case, thank you very much for being so helpful and responsive! It counts a lot!

[pauleveritt]

But I was wrong and it hurts the developer experience... I also think that I should embrace protocols a bit more.

I spent the weekend trying to embrace protocols for the third time in my registry/injector thingy. It's just painful. Places in Python that want a Type....subprotocols that add attributes then are flagged as not compliant. It just seems too early to use Protocols without pain.

As a zope.interface guy, I still lament the old decision not to use them.

[flisboac]

Thanks for the insights, @pauleveritt !

Places in Python that want a Type

At first, I'd expect a Type[SomeProtocol] to be acceptable, and a substitute for Type[SomeClass] where SomeClass implements SomeProtocol.... But someone could argue that a better type annotation could be Callable[..., SomeProtocol], so that you don't tie your value's typing to a specific concrete type (which is what Type denotes), but as you said, so many places in Python do require Type specifically. It's quite sad. :(

subprotocols that add attributes then are flagged as not compliant.

Well, that's especially sad, and I see no reason why it wouldn't be accepted. Perhaps they need to implement that support in, but there're so many static analysis tools, and we'd need to wait for all of them to converge to some similar behaviour, or have said behaviour properly specified in some PEP (I don't know if that's the case) (and perhaps wait for at least some of them to catch up).

---

On paper, protocols are a good idea, and a better fit than strict class matching. On a smaller scale, and for very specific cases, they work very well. But the feature itself still feels incomplete.

For declaring injectable classes, specifically, I think the decorator route is a better one, as you don't need to specify any subclass or class format at all. You're just marking the class for injection and/or autowiring. Matching types and type formats will mostly happen at injection time (except when e.g. caching type hierarchy for faster injections), and in those cases protocols are not really a good fit, because verification is too loose, and takes a bit longer (because it's essentially duck typing; and all of this implies @runtime_checkable, of course, which cannot be properly enforced by the DI library, AFAICT).

[Finistere]

Hello @flisboac, thanks a lot for your comments and proposals, it's really helpful. And I really am grateful for the time you spent already!

I agree on the need on Antidote providing tooling for service discovery instead of hard-wiring everything. It's something @pauleveritt also brought up to me. Also big thanks to you too @pauleveritt, we already had a lot of conversations, it's really inspiring. :)

Catalog idea looks good to me. So if I understand it correctly:

• A catalog would be in Antidote current terminology something like a sub-container on which another container or world itself would rely on.
• A catalog defines a public set of dependencies which can be used by upper catalogs.
• Within a dependency you need to be able to access the catalog/container you're in (ModuleRef).

Does this summarize correctly your catalog proposal? All of this seems possible to me. Antidote did have something similar for some time, but I deemed it too complex and had other priorities at the time.

Is it necessary to be able to declare catalogs at @inject-level though ? The catalog idea for me seems to be global definition. You declare, within your current container/catalog, on which sub-catalog/container you want to rely on.

On the qualifier topic, I think there are two cases that need to be separated:

• a sealed qualifier, where all possible qualifiers are defined in one place like an enum
• a "open" qualifier, no freeze() method at first sight as at that point you either freeze everything through world.freeze() or not at all IMHO.

So each of your example matches one of those. For the API itself, I need to play a bit with it to get it a feeling for it but looks good at first sight!

As a final note, you don't need to worry about having a working implementation / using antidote feature in your examples. What I really care most about is having a good developer experience/API, so having example on how it should or could behave and look like is the most important thing. :)

---

Regarding the topic of protocols I expect to see issues when Antidote will support some form of service discovery. At that point I'm not sure whether I want Antidote to actually enforces types or how far. In the same spirit I intend to remove the type check in world.get[T](dependency) as there isn't a clean way to handle it with anything more complex than T being a class. And because IMHO those type checks would be better done at dependency definition level.

---

I'm currently working on simplifying Antidote, deprecating in backward-compatible manner, to focus on decorators and being more type-safe/static-typing friendly. So stuff like this for the 1.1:

# behaves better with static-type checking, it won't complain about a missing argument
@inject
def f(db: Database = inject.me()):  
    ...

@inject
def f(db: Database = inject.me.from_(factory)):
    ...

@inject
def f(db = inject.from_(factory).get(Database)):
    ...

@inject
def f(db = inject.get(Database)):
    ...

# Replacing `Database @ factory` syntax for plain english
@inject([from_(factory).get(Database)])
def f(db):
    ...

world.from_(factory).get(Database)

And deprecating Service in favor of @service, etc... I intend to finish it probably next week or so. And after this I'm tackling the qualifier, catalog, service discovery topics! Still other things to improve, such as all the lazy dependencies but they'll be done a bit later.

And again, thanks a lot for all the insights @pauleveritt and @flisboac!

[Finistere]

Hello @flisboac,

I'm currently considering the following API for qualifiers:

from enum import Enum

from antidote import interface, world

@interface
class LogSink:
    pass

class LogSinkOn(Enum):
    DEV = 'dev'
    STG = 'stg'
    PRD = 'prd'

@interface[LogSink].when(qualifiers=[LogSinkOn.PRD, LogSink.STG])
class ConsoleLogSink:
    pass

world.get(interface[LogSink].all(qualifiers=[LogSinkOn.PRD]))
# Ensuring there is one and only one matching.
world.get(interface[LogSink].single(qualifiers=[LogSinkOn.PRD]))

I would probably limit the qualifier values to anything except strings/ints as they're cached by Python and I'd only use is to check qualifier equality, forcing one to actually use the same object. I would also force the decoration of the interface with @interface even though it probably wouldn't do anything. It would make it explicit that Antidote can provide this interface.

I'm considering implementing this top of a core API a "predicate" API (using Pyramid terminology, subject to change), suggested by @pauleveritt.

from antidote import Qualifier

@interface[LogSink].when(Qualifier(LogSinkOn.PRD), Qualifier(LogSink.STG))
class SyslogLogSink:
    pass

world.get(interface[LogSink].all(Qualifier(LogSinkOn.PRD)))

Qualifier would be a "predicate", meaning a constraint specifying when a dependency can be used for an interface. I would like developers such as yourself to be able to develop their own predicates. Antidote would only provide keyword arguments, such as qualifiers for the predicates provided out of the box for type-safety. interface[LogSink].single would retrieve only one candidate and fail if there are more. When multiple predicates are used, a weight system could be used to determine the priority of the candidates.

This would allow one to extend Antidote with simple custom selection logic. I'm unsure which kind of predicates Antidote would support at first sight. I'm thinking of two currently:

• predicate that evaluates a certain condition, specifying whether the current candidate can be used or not.
• predicate that given another predicate of the same type would determine whether the candidate matches, a bit like Qualifier would work.

As you've definitely had a lot of experience with different dependency injection frameworks in different languages, I'd love to hear your opinion! Do you see yourself using such an API? Do you have use cases that would profit from it or on contrary not fit within this logic? Too complex/too simple?

Note: the qualifier API would come before the predicate API, with the keywords it's easy to add and would so be available quicker.

[flisboac]

Hello, @Finistere ! Thanks for all the kind words, and all the attention you've been giving to this issue and more!

Regarding the catalog idea, yes, that's a good summary. I'd also add the possibility for a dependency to be resolved through the ModuleRef, with a flag to enable searching the dependency outside the catalog (ie. app and world-level) -- which is inspired by the same functionality in NestJS (in NestJS parlance, that is strict: false), and that I did use in more than one occasion.

The advantage of this approach is that:

1. Dependencies local to the catalog always have priority.
2. It makes users less dependent on interfacing with world when searching for a dependency dynamically.
3. It makes it possible to do an "app-level" dependency search, in case a specific set of catalogs are selected by your app's entrypoint.

In my initial proposal, the idea was for world to have "globals" (global-level dependencies), and nothing changes in regards to how it's implemented today. Whereas catalogs would implement "scoped" dependencies, much in the same way how modules in Python work (ie. you need to import dependencies to use them, and imports are local to the importing catalog; with optional caching, perhaps).

Is it necessary to be able to declare catalogs at @inject-level though ? The catalog idea for me seems to be global definition. You declare, within your current container/catalog, on which sub-catalog/container you want to rely on.

In regards to the @inject decorator, the catalog selection would only apply to the entrypoint (ie. your main function). I don't think injection points in constructor/function parameters need this at all.

This would lessen the amount of dependencies the DI mechanism need to register and traverse, and it makes the whole mechanism a bit more scalable. Tests could be smaller and focused on specific catalogs, for example. Or the app may offer multiple entrypoints, each of them with their own catalog selection.

Otherwise, the only thing you would need, at the app-level, is to import the catalog, and then antidote would automagically include it, which is not very explicit and would raise some warnings (e.g. flake8 complaining the import is not used). Also, all entrypoints would see all catalogs as well, which defeats the purpose (as far as I understand it).

But I admit this could be an immense change to the codebase, in which case opting for a static config approach would be the only way. I also don't know how it'd play out with the Providers mechanism (do we extend the Provider API to allow for the catalog implementation? How to integrate existing providers with this supposed new means of resolution?). But again, this is just an idea; if it's not feasible, I will understand, and it's not a problem at all, antidote is already plenty helpful as it is!

---

Regarding the topic of protocols I expect to see issues when Antidote will support some form of service discovery. At that point I'm not sure whether I want Antidote to actually enforces types or how far.

I think there's still value in type-checking, as long as we keep it to concrete and ABC types only. Type-checking protocols equates to duck-typing, which for me is a battle already lost at the start.

Other DI libraries/frameworks can inject dependencies based on either a type or a name. I think protocols are a better fit for those named dependencies, because for protocols to work, the user would need to have prior knowledge of a dependency's API/format, in which case a protocol is the best way to type it out. Now, the name could come from either the parameter name, or Annotated, or some other mechanism, but in either of those cases Antidote doesn't seem to support this use case (named dependency) yet, so this point is moot.

So I would:

1. Avoid to use Protocols, and/or warn the user when a dependency is provided in terms of protocols (if that's possible). Or
2. We could keep isinstance matching when injecting non-protocol classes, but avoid it altogether when the dependency is given in terms of Protocols at the injection point (in which case the user would also need to explicitly tell which classes implement the protocol at declaration time, via e.g. @implements, or something similar).

As a side note, in CDI and some other Java-based DI mechanisms, the dependency's name is some sort of qualifier. @Named in CDI has special meaning, and is a required annotation for classes to be included in the DI mechanism. If a name is not provided, the name of the class is assumed to be the name of the dependency (with the first character lowercased). Matches are done either by name or type, and there's type checking to ensure that a dependency has the right type for the injection point. When injecting an interface, any class that implements that interface becomes a candidate, and you can optionally also provide qualifiers to narrow down selection (e.g. @Named, or some other @Qualifier annotation).

---

behaves better with static-type checking, it won't complain about a missing argument

Man, that's the one thing that I found a bit odd when I first tried Antidote! It's much more natural this way.

But I will admit that I found the Dependency @ origin idiom quite charming, and succint, and I'll miss it. :(

P.S.: that underline in from_ is a bit weird. Perhaps of would be better? But that's just my bikeshedding. Also, I'm pissed at the fact that Python took such an useful word from us! >:(

---

I would probably limit the qualifier values to anything except strings/ints as they're cached by Python and I'd only use is to check qualifier equality, forcing one to actually use the same object. I would also force the decoration of the interface with @interface even though it probably wouldn't do anything. It would make it explicit that Antidote can provide this interface.

That's very reasonable.

@interface[LogSink].when(qualifiers=[LogSinkOn.PRD, LogSink.STG])

I'd use a different decorator, if possible, because the way it is declared, it seems like the decorated class is an interface, which is not the case. It's an implementation.

What about @injectable or @implements instead? e.g.

@injectable[LogSink].when(qualifiers=[LogSinkOn.PRD, LogSink.STG])
class ConsoleLogSink:
    pass

# OR

@implements[LogSink].when(qualifiers=[LogSinkOn.PRD, LogSink.STG])
class ConsoleLogSink:
    pass

IMHO, it reads a bit better, and avoids confusion.

---

I'm considering implementing this top of a core API a "predicate" API (using Pyramid terminology, subject to change), suggested by @pauleveritt.

That's an amazing idea, and it never occurred me. I like it! I just don't know how it would be implemented (e.g. would we pass a context object to the def __call__(self, ctx) -> bool method of a predicate? If so, what could go into it?), but I see it as the best way to make a more generic filtering mechanism.

Although I'd me more keen on naming the predicates as adjectives or adjective phrases, as they would qualify whatever they're annotating/decorating. For example:

from antidote import QualifiedBy

@implements[LogSink].when(QualifiedBy(LogSinkOn.PRD), QualifiedBy(LogSink.STG))
class SyslogLogSink:
    pass

world.get(interface[LogSink].all(QualifiedBy(LogSinkOn.PRD)))

I think it reads a bit better, like a full phrase. What do you think?

In any case... If you have the possibility to put predicates on both ends, how would you match a predicate with another?

One idea I had would be for predicates to be checked against one another; this way, the class hierarchy would be responsible for establishing how to check values of its type. For example, QualifiedBy could have a matches method (enforced by some Antidote interface or protocol) that makes an "includes" of the input set against the implementation's qualifier set. If true, the dependency is properly qualified.

Problem is if we give more than one predicate, and need to select considering the whole qualifier set. Imagine the user passes two QualifiedBys. the correct way for a verification to occur is for both sets to be joined and then verified. In this case, perhaps a simplify method could optionally be provided as well: if two or more qualifiers are present, simplify could be called to join them into a single predicate. In QualifiedBy's case, that would mean all qualifier sets joined into a single predicate.

OR, we could limit the predicate set to have only one instance per class hierarchy (i.e. no two predicates' types should be a subclass of each other, or something along those lines).

Note that I'm just expanding on the possibility of implementing qualifiers in this new mechanism. The same scenario could apply in some other use case.

Also, what would happen if we try to filter a dependency with a predicate type that's not provided in the implementation class? Do we skip said dependency? (My gut feeling is that yes, it must be skipped, because it would have no means to match that predicate).

---

Thanks so much for everything, @Finistere !

[Finistere]

In my initial proposal, the idea was for world to have "globals" (global-level dependencies), and nothing changes in regards to how it's implemented today. Whereas catalogs would implement "scoped" dependencies, much in the same way how modules in Python work (ie. you need to import dependencies to use them, and imports are local to the importing catalog; with optional caching, perhaps).

I'm not entirely sure to understand your vision. For me, off the top of my head, I would do something like this:

# essentially world
class Container:
    catalogs: list[Catalog]

    def get(self, dependency: object) -> object:
        # provider logic
        # ...

        for catalog in self.catalogs:
            try:
                return catalog.get(dependency)
            except DependencyNotFoundError:
                pass

        raise DependencyNotFoundError(dependency)

# API

catalog = world.new_catalog()
with catalog:  # replaces world within this context, similar to world.test.new()
    # declare dependencies, eventually through their import
    ...

# add to a mother catalog
mother_catalog.add_catalog(catalog)
# or to world directly
world.add_catalog(catalog)

I also don't know how it'd play out with the Providers mechanism (do we extend the Provider API to allow for the catalog implementation? How to integrate existing providers with this supposed new means of resolution?). But again, this is just an idea; if it's not feasible, I will understand, and it's not a problem at all, antidote is already plenty helpful as it is!

It would be orthogonal to the provider mechanism IMO. Providers define how dependencies are constructed and how they should be cached (scope). A Catalog would have its own Providers. I don't know yet if the global Container should be a Catalog itself or not, but they're definitely similar.

with a flag to enable searching the dependency outside the catalog (ie. app and world-level)

I'm unsure how to make this for now. Getting them through world would work, but specifying at @inject time that you actually want to rely on the world is a bit more tricky. I suppose I could add inject.from_world.me and the FromWorld annotation. FromWorld could also work in @inject(dependencies={'x': FromWorld(...)}).

Otherwise I really like the approach of local-first, global if requested!

---

I think protocols are a better fit for those named dependencies

I agree. Named dependencies in Python are a disaster for me as you wouldn't necessarily have the type... In Java, it's a different story. Thanks for the explanation!

Now, the name could come from either the parameter name, or Annotated, or some other mechanism, but in either of those cases Antidote doesn't seem to support this use case (named dependency) yet, so this point is moot.

Antidote supported using argument names and it could be added through a wrapped @inject & Provider, but well. I don't recommend it. :)

We could keep isinstance matching when injecting non-protocol classes, but avoid it altogether when the dependency is given in terms of Protocols at the injection point (in which case the user would also need to explicitly tell which classes implement the protocol at declaration time, via e.g. @implements, or something similar).

I was leaning to this conclusion, thanks for your input!

---

But I will admit that I found the Dependency @ origin idiom quite charming, and succint, and I'll miss it. :(

So you would prefer something like this?

@inject
def f(db: Database = inject.me @ factory):
    ...

@inject
def f(db = inject.get(Database) @ factory):
    ...

To be honest the two reasons I have currently to use from_ are:

• It's plain english, I'm so used to dependency @ origin that I'm not sure whether it's actually that obvious as an idiom. But you obviously expressed the opposite.
• There are two ways to implement it:
  • __rmatmul__(), the current implementation, which requires that I change the factory object which I would like to avoid.
  • __matmul__() which works with world and inject, but specifying dependency @ origin doesn't. I don't see how one would express the dependency in this case: @inject(dependencies={'db': Database @ factory}). Maybe @inject(dependencies={'db': From(factory).get(Database)}) using the annotation From.

P.S.: that underline in from_ is a bit weird. Perhaps of would be better?

I agree it's not great, but it feels more natural in English, to me at least as a non-native speaker, than of.

---

What about @injectable or @implements instead? e.g.

Excellent point, I might prefer @implements(LogSink) as it states that the underlying object should implement the specified interface. With @injectable it feels less explicit.

---

Although I'd me more keen on naming the predicates as adjectives or adjective phrases, as they would qualify whatever they're annotating/decorating. For example:

Agreed, QualifiedBy & qualified_by are better.

In any case... If you have the possibility to put predicates on both ends, how would you match a predicate with another?

I see two predicates kind that could easily be handled:

• condition: defines at import time whether an implementation can be used
• value: if given as query argument, candidate implementation must have this predicate. For those, the predicate defines itself whether the implementation predicate matches the query one.

# Called at import time. Can be injected to retrieve configuration for example.
def condition_predicate():
    ...

# evaluated against each other.
class ValuePredicate:
    def matches(self, other: ValuePredicate) -> bool:
        ...

OR, we could limit the predicate set to have only one instance per class hierarchy (i.e. no two predicates' types should be a subclass of each other, or something along those lines).

Agreed, I think it's simpler. They could be subclasses, but they can't have the same type, at least for ValuePredicates. So no multiple QualifiedBy like in the examples.

Also, what would happen if we try to filter a dependency with a predicate type that's not provided in the implementation class? Do we skip said dependency? (My gut feeling is that yes, it must be skipped, because it would have no means to match that predicate).

Yes, that's also my opinion.

[Finistere]

I don't see how one would express the dependency in this case: @inject(dependencies={'db': Database @ factory}). Maybe @inject(dependencies={'db': From(factory).get(Database)}) using the annotation From.

Actually doing Get(dependency) @ factory doesn't look that bad and could also be used as an annotation. It's probably a good idea to have the same objects for annotation and direct dependencies.

[Finistere]

Still not 100% sure, but I'm considering this instead:

@inject
def f(db: Database = inject.me(source=factory)):
    ...

@inject
def f(db = inject.get(Database, source=factory)):
    ...

@inject([Get(Database, source=factory)])
def f(db):
    ...

world.get(Database, source=factory)

My main problem with @ is the lack of clarity for anyone reading this code without any prior knowledge of Antidote. If the factory has a poor name, it would not be obvious what the @ means. Sure you can check out the factory code and guess what it does, but it's already too much indirection IMHO.

[flisboac]

Hello again, @Finistere !

I'm not entirely sure to understand your vision. For me, off the top of my head, I would do something like this: Ok, that's a perfectly fair API.

I was considering the @inject use case, for when you configure a catalog per app entrypoint (ie. per "main" function), as you would want to have as many as needed. Updating my samples from before:

from antidote import inject, world
from my_logging_fd import catalog as fd_catalog
from my_logging_os import catalog as os_catalog
from my_logging import catalog, ProvidedLogger

aws_ecs_catalog = world.new_catalog()
aws_ecs_catalog.add_catalog(catalog, fd_catalog, os_catalog)

aws_lambda_catalog = world.new_catalog()
aws_lambda_catalog.add_catalog(catalog, fd_catalog, os_catalog)  # could be different

@inject.from_(aws_lambda_catalog)  # could also be `@inject(catalog=aws_lambda_catalog )` ?
def aws_lambda_main(logger: ProvidedLogger): 
    """Specialized for running as a AWS Lambda."""
    # ProvidedLogger comes from main_catalog, or transitively from its (catalog) dependencies.
    # Any dependency neither declared in those catalogs nor in world are not even visible here.
    # ...
    # do some work
    logger.log('done!')  # would log to many different sinks (or less)

@inject.from_(aws_ecs_catalog)
def aws_ecs_main(logger: ProvidedLogger):
    """Specialized for running as a container, eg. ECS/Fargate, etc."""
    # same here
    # ...
    # do some work
    logger.log('done!')  # would log to many different sinks (or less)

# I could have as many "main"s as needed.

# Each should see only the dependencies on the catalog it depends on.
# Dependencies of dependencies would use the catalog's resolution mechanisms.
# ie. an injected service would have its dependencies resolved from the
# scope of the main's catalog, not world; no explicit catalog reference should
# be needed to be specified at the injection point. Antidote knows which catalog
# is the "current", and will use it for resolutions.

# Of course, this is just an alternative to the context manager form,
# for when the needed dependencies are known or can be declared as
# parameter type annotations.

# The sequence format I initially proposed, ie. `catalogs=(...)`
# would do all that catalog creation for you automatically, if all
# you would do is call `add_catalog` (which I think will be a common
# use case for catalogs).
# this catalog would exist only for, or at least for, the duration of the
# function's execution.
# Example:

@inject.from_([catalog, fd_catalog, os_catalog])  # could also be `@inject(catalog=aws_lambda_catalog )` ?
def aws_lambda_main(logger: ProvidedLogger): 
    """Specialized for running as a AWS Lambda."""
    # ...
    # do some work
    logger.log('done!')  # would log to many different sinks (or less)

# All of this would allow me to share entrypoints in a library as well, for example.

---

A Catalog would have its own Providers.

Perfect! That's how I initially imagined them to work. More specifically, my worries were about duplicating efforts when splitting between world and catalogs' provider implementation. I understood initially that there would be distinct implementations for both, but looking at Antidote's code, I understand better what you mean.

What about this, for example, for the indirect provider (src/antidote/_providers/indirect.py)?

@API.private
class IndirectProvider(Provider[Hashable]):

    # ...

    def maybe_transitive_provide(self, dependency: Hashable, container: Container
                      ) -> Optional[DependencyValue]:
        """Searches for dependencies in imported catalogs"""

    # Is this a sufficient implementation?
    def maybe_provide(self, dependency: Hashable, container: Container
                      ) -> Optional[DependencyValue]:
        if not isinstance(dependency, ImplementationDependency):
            return self.maybe_transitive_provide(dependency, container)  # <-- here

        try:
            target = self.__implementations[dependency]
        except KeyError:
            return self.maybe_transitive_provide(dependency, container)  # <-- here

        if target is not None:
            return container.provide(target)
        else:
            # Mypy treats linker as a method
            target = dependency.implementation()
            if dependency.permanent:
                self.__implementations[dependency] = target
            value = container.provide(target)
            return DependencyValue(
                value.unwrapped,
                scope=value.scope if dependency.permanent else None
            )

    # etc...

@API.private
class ImplementationDependency(FinalImmutable):
    __slots__ = ('interface', 'implementation', 'permanent', '__hash')
    interface: type
    implementation: Callable[[], Hashable]
    permanent: bool

    # If None, comes from world; if not None, comes from some catalog.
    # Either that, or `catalog: Catalog`, and world will be a catalog.
    # Should be explicit in case the dependency comes from somewhere else.
    # And, of course, this is only relevant if caching transitive dependencies
    # locally is deemed important. Otherwise, this is not necessary.
    catalog: Optional[Catalog]

    __hash: int

    def __init__(self,
                 interface: Hashable,
                 implementation: Callable[[], Hashable],
                 permanent: bool,
                 catalog: Optional[Catalog] = None):
        super().__init__(interface,
                         implementation,
                         permanent,
                         hash((interface, implementation, catalog)))  # <-- here

    # ...

    def __eq__(self, other: object) -> bool:
        return (isinstance(other, ImplementationDependency)
                and self.catalog is other.catalog  # <-- here
                and self.__hash == other.__hash
                and (self.interface is other.interface
                     or self.interface == other.interface)
                and (self.implementation is other.implementation  # type: ignore
                     or self.implementation == other.implementation)  # type: ignore
                )  # noqa

In any case, each catalog has their own provider instances, as you said. The user can add more providers, if needed (or not; I'm not entirely sure if it's a good idea, though).

Is this what you mean (or anything along those lines)?

---

specifying at @inject time that you actually want to rely on the world is a bit more tricky.

The "flag" thing was specifically about programmatic resolution (ie. using the API, and not type annotations), but I agree the functionality should be provided on both sides.

We could initially assume the following:

• For declarative dependency lookup: If a dependency is annotated with FromWorld, Antidote would ignore catalog search, and go straight to the "world catalog";
• For programmatic dependency lookup: If strict: false and the dependency is not found in the catalog's scope, the "world catalog" will be consulted.

Both of those suggestions mean using world providers' semantics, not the catalog providers' semantics, when a dependency is being searched in world.

But I understand that this may cause some confusion. We may not have provider-type parity amongst all catalogs involved (world included), so some injections may fail if we follow this suggestion.

In this case, perhaps a global "provider registry" could be implemented. Everytime a catalog is created, all registered "default" provider factories are called, and catalogs are constructed using those. (Don't know about the feasibility of such a solution, though; may be too much work for little benefit).

What do you think?

---

I see two predicates kind that could easily be handled

I understand we have those two use cases, but will they have different types?

I thought about the following:

class Predicate(metaclass=ABCMeta):  # Could be a Protocol too
    @abstractmethod
    def matched_by(self, other: Any) -> bool:  # could it be __contains__?
        ...

    @abstractmethod
    def __eq__(self, other: Any) -> bool:
        ...

    @abstractmethod
    def __hash__(self) -> int:
        ...

class QualifiedBy(Predicate):
    def __init__(self, *qualifiers: Qualifier):
        self.qualifiers: Final[Set[Qualifier]] = set(qualifiers)

    def matched_by(self, other: Any) -> bool: 
        if not isinstance(other, QualifiedBy):
            return NotImplemented
        return other.qualifiers.issubset(self.qualifiers)

    def __eq__(self, other: Any) -> bool:
        if not isinstance(other, QualifiedBy):
            return NotImplemented
        return self.qualifiers == other.qualifiers

    def __hash__(self) -> int:
        return hash(self.qualifiers)

Because we won't allow "duplicates" (in terms of all(type(a) is not type(b) for a in preds for b in preds if a is not b), or something like that), this should be enough (ie. no grouping is needed). QualifiedBy and similar implementations could be used on both ends, no problem, but the order at which they're compared is important (hence why I changed matches to matched_by, so that the predicate declared in the class can always be the left-hand side of the comparison, as it looks a bit more natural to me.

---

Actually doing Get(dependency) @ factory doesn't look that bad

I like it too!

@inject([Get(Database, source=factory)])

Not bad either!

Regarding the @ operator, I agree with your points. I like the @ because I read it as "at", or "from somewhere". Reminds me of email@domain, which for more seasoned programmers/tech-people it's quite a common idiom, and obvious in meaning. But the english-first approach is more reasonable overall.

Could we offer both APIs? There's benefits to both approaches. But if it's too much work, I'd prefer the word-based one, because it looks more like Python, and less like, idk, Perl. :p (jk, I don't dislike Perl)

[Finistere]

I was considering the @inject use case, for when you configure a catalog per app entrypoint (ie. per "main" function), as you would want to have as many as needed. Updating my samples from before:

I'm not convinced by your example. With aws_lambda_main and aws_ecs_main, you're distinguishing the code path between lambda and ecs, so hiding this specificity behind a catalog & dependency injection doesn't make sense. IMHO either they are the same function and it's the caller that defines the catalog or the catalog selection is inside their body like:

def aws_lambda_main(): 
    """Specialized for running as a AWS Lambda."""
    with aws_lambda_catalog:
        common()

def aws_ecs_main():
    """Specialized for running as a container, eg. ECS/Fargate, etc."""
    with aws_ecs_catalog:
        common()

@inject
def common(logger: ProvidedLogger = inject.me()):
    logger.log('done!')  # would log to many different sinks (or less)

But I do see the need to change, at runtime, the catalogs used by world. If I understand your need correctly, it's the possibility to override within a certain context the dependencies exposed by world.

Perfect! That's how I initially imagined them to work. More specifically, my worries were about duplicating efforts when splitting between world and catalogs' provider implementation. I understood initially that there would be distinct implementations for both, but looking at Antidote's code, I understand better what you mean.

I think you misunderstood my pseudo-code, so trying again with a bit more details. :)

class Catalog:
    catalogs: list[Catalog]
    providers: list[Provider]

    def get(self, dependency: object) -> object:
        for catalog in self.catalogs:
            try:
                return catalog.get(dependency)
            except DependencyNotFoundError:
                pass
        # rough logic inside current Antidote's Container
        try:
            return self.singletons[dependency]
        except KeyError:
            for provider in self.providers:
                try:
                    value, singleton = provider.provide(dependency)
                except DependencyNotFoundError:
                    pass
                else:
                    if singleton:
                        self.singletons[dependency] = value
                    return value

        raise DependencyNotFoundError(dependency)

# root catalog singleton
world = Catalog()

With this design, world would be the root catalog. A catalog could hold other catalogs in which it'll search for dependencies and use its providers as last resort. The catalog usage logic might change, but that's at least the interaction I see between a Catalog and Provider. The Container in current Antidote's terminology would become a Catalog which could hold other Catalogs. Providers wouldn't be Catalog-aware at all.

So to summarize the catalog specification you're proposing:

• A catalog defines a public set of dependencies on which other catalogs may rely on.
• For injections, you need to be able to override within a certain context the dependencies exposed by world with a catalog
• When a dependency is declared within a catalog, all injections within the class/function should only use the catalogs dependencies. But, it should be possible to reach for dependencies from world if explicitly told so.

I don't have a precise idea on how the API should look like/behave to be coherent though for the moment. I'll need to play with dummy examples to have an understanding of what's intuitive.

---

I understand we have those two use cases, but will they have different types?

Yes those would indeed have different types, something like ValuePredicate / MatchPredicate and ConditionPredicate in the same spirit as your code.

but the order at which they're compared is important (hence why I changed matches to matched_by, so that the predicate declared in the class can always be the left-hand side of the comparison, as it looks a bit more natural to me.

Good point! I'm not entirely sure we would have exactly the same objects in the query and class definition though. Qualifier query could be for example a "one of the qualifiers", "all of the qualifiers" or "all qualifier of this type". However, the API must feel symmetrical. And for performance, I would intuitively limit a "query predicate" to only access one kind of "class predicate", whether they're actually the same type or not. So it could be something like:

@implements(Logger).when(QualifiedBy(X))

# Maybe support this ?
world.get(interface[Logger].all(QualifiedBy(X))  # X
world.get(interface[Logger].all(QualifiedBy(X, Y))  # X and Y
world.get(interface[Logger].all(QualifiedBy.one_of(X, Y))  # X or Y
world.get(interface[Logger].all(QualifiedBy.instance_of(Enum))

---

Regarding the @ operator, I agree with your points. I like the @ because I read it as "at", or "from somewhere". Reminds me of email@domain, which for more seasoned programmers/tech-people it's quite a common idiom, and obvious in meaning. But the english-first approach is more reasonable overall.

Could we offer both APIs? There's benefits to both approaches. But if it's too much work, I'd prefer the word-based one, because it looks more like Python, and less like, idk, Perl. :p (jk, I don't dislike Perl)

You're giving me other arguments not to use @, Perl. :D Well, another issue I have is consistency with the @:

inject.me(source=factory)
inject.get(Database, source=factory)
Get(Database, source=factory)
world.get(Database, source=factory)

vs

inject.me @ factory
inject.get(Database) @ factory
Get(Database) @ factory
# Can't use world.get(Database) !
world.get[Database] @ factory

And another small inconvenience is that with the @ syntax, you need to do wrap the output with parentheses...

(world.get[Database] @ factory).method()
world.get(Database, source=factory).method()

This is even less obvious when using __call__ or any other operator:

(world.get[Database] @ factory)(arg=X)
world.get(Database, source=factory)(arg=X)

[flisboac]

IMHO either they are the same function and it's the caller that defines the catalog or the catalog selection is inside their body like

I think we may be talking about the same thing, but just disagreeing on syntax. Either that, or there is some difficulty on offering that context-managed injection via @inject I'm failing to foresee.

The context manager syntax makes a lot of sense, and I'm pleased by the API you proposed. The @inject support would be a shortcut (an extra, or syntax sugar) to the user. It's not necessary at all, it's just to avoid the need for writing two functions for each entrypoint, or for when programmatic dependency resolution is not going to be used. It would allow for each entrypoint to be injected with the dependencies they need right away, at annotation level. Also, parameters that are not injected could still be passed along, something that may be useful for standalone functions (like the entrypoints).

The same machinery used to implement the context manager idiom could be used to implement the decorator-based one (unless there's some problem with it, which I'm not seeing).

Regarding my example, the implementations of aws_lambda_main and aws_ecs_main are not the same. Sorry for not making that obvious at first. They have different needs, do different tasks, or inject completely different dependencies. Therefore, their code is what is going to differ, and is what matters, not their injections. Them having the same dependencies is only coincidental. I just didn't bother to put different implementations for them at the time. :(

Folows an updated example:

# The lambda parameters will be passed by AWS Lambda's framework.
# event and context are just just JSON objects.
# With a code this simple, it become very easy to sell the use of a DI library,
# proper IoC and testability.
@inject(source=aws_lambda_catalog)  # copying your proposed new factory syntax here
def aws_lambda_main(
  event,
  context,
  *,
  s3: S3Client= inject.me(),
  sns: SnsClient = inject.me(),
  logger: ProvidedLogger = inject.me(),
) -> None:
  # do some work
  # ie. process incoming event, publish sns message, etc
  # ...
  s3.put_object(event, ...)
  sns.notify({ "type": "...", "message": "work done" })
  logger.info("event processing finished successfully)

# Now, suppose we're going to provide an API as a Fargate container.
# aws_ecs_main is a long-running process, and receives no
# parameters. The aws_ecs_main is called from a script, or from some
# executable module (e.g. `python -m my_lib.main.aws_ecs`)
@inject(source=aws_ecs_catalog)
def aws_ecs_main(
  *,
  cli_args_parser: CliArgsParser = inject.me(),
  api_config: ApiConfig = inject.me(),  # Gets some env-var configs, etc. Is a Constants class
  config_provider: ConfigProvider = inject.me(),
  api_service: ApiService = inject.me(),
) -> None:
  # Also consider that different entrypoints may parse the CLI
  # through different CLI parsers, or may parse the CLI arguments
  # themselves.
  cli_args = cli_args_parser.parse_args()

  config = config_provider.merge_configs(cli_args, api_config)
  api_service.serve(config)

# They both do different things. There may be some overlap in dependencies
# or implementation on them, but that's not what I was arguing for.

As a library author, I'll provide catalogs that can be used by both end-users (lambda and ECS). Catalog selection is primarily the responsibility of the end user (or, of a library provider, transitively).

Another advantage of @inject-decorated functions that they can be called as-is, for simpler use-cases. Now, how this is going to play out when they're called in the context of some other catalog is something I'm not sure myself how to deal with, and may be the reason for some opposition to this idea. When I first wrote the proposal, I was thinking about catalogs overriding whatever was in their context, much in the same way it is done today with world, but without a clone. For example:

# In normal code:

# A new catalog, with some overrides
specialized_aws_ecs_catalog = world.new_catalog()
specialized_aws_ecs_catalog.singleton(...)
# ...

def specialized_aws_ecs():
  with specialized_aws_ecs_catalog:
    # I don't need to specify aws_ecs_catalog here, because
    # the entrypoint would have imported it already.
    # The context manager override would be an exception,
    # not a rule.
    aws_ecs()

#
# Or, in tests:

def test_aws_lambda_main():
  # This context manager would take precedence when providing dependencies.
  with world.test.new_catalog() as test_catalog:
    test_catalog.override.singleton(S3Client, MockedS3Client())  # Or simply `test_catalog.singleton(...)`?
    # ...
    # Would use the MockedS3Client first, and respect
    # dependencies coming from the original aws_lambda_catalog
    # if not present in test_catalog
    aws_lambda_main(test_event, test_ctx)

But your suggested use is perhaps clearer in intent, and better from an Antidote-implementation point of view. Rewriting the previous example, I would have the following:

def aws_lambda_main(
  event,
  context,
) -> None:
  with aws_lambda_catalog:
    aws_lambda_main_impl(event, context)

@inject
def aws_lambda_main_impl(
  event,
  context,
  *,
  s3: S3Client= inject.me(),
  sns: SnsClient = inject.me(),
  logger: ProvidedLogger = inject.me(),
) -> None:
  # do some work
  # ie. process incoming event, publish sns message, etc
  # ...
  s3.put_object(event, ...)
  sns.notify({ "type": "...", "message": "work done" })
  logger.info("event processing finished successfully)

Problem now is that aws_lambda_main lose the indication that it works in terms of dependency injection. I will also need to test both, or have a really small aws_lambda_main implementation and depend on test results of aws_lambda_main_impl (mostly). If a catalog context-manager is able to affect aws_lambda_main_impl even when the one being directly called is aws_lambda_main, then my complaints are unfounded, but I'm not sure about that, E.g.:

with mocked_aws_lambda_catalog:
  # How would I test aws_lambda_main, if what has injection is aws_lambda_main_impl?
  aws_lambda_main()

Well, that said... As a side note...

My specific needs, as of now, are for relatively short executables, parameterized by a combination of CLI arguments, environment variables and configuration values coming from all sorts of external places/services. Each project (a repository; one or more for each team and product) will have an assortment of job implementations of varied types, written as Python functions, executable modules or normal Python scripts. There will be a lot of projects, and a lot of jobs for each project (ie. it's numerous in both aspects). Each job type parses, fetches and merges parameters (CLI, env-vars, etc) in different ways, but some common properties among them are somewhat guaranteed. Each job will execute in a specific environment, or will interface with a specific systems, so a considerable number of services (service classes, etc) will be offered in the form of an internal framework, to standardize those services and some aspects of job implementations.

Catalogs would allow me to extend a baseline with the necessary implementations for each job type (not each job!), without enforcing the entire framework upon all of them (ie. not putting everything in world). Each job would pick the catalogs they need, so that the amount of scanned dependencies could be reduced, and dependencies could be more easily traceable. It's still up to the job implementation to orchestrate how its injected services are used, or implement details not covered entirely by a service (ie. things specific to that job).

I can see most of the smaller or simpler jobs using the short (@inject) form, because some use cases can be abstracted away quite significantly. This is what we were focusing on (ie. simplifying and reducing the amount of code we need to write). But the bigger ones may be complex enough to warrant more code. In both cases, the use of a DI library would do wonders.

A catalog could hold other catalogs in which it'll search for dependencies and use its providers as last resort.

Well, that's the inverse of what I was thinking of, in terms of dependency resolution order.

My catalog proposal was based on my experiences with NestJS modules. I also used it as a base model. Please take a look at this diagram, from their documentation:

My initial idea was for dependencies to be looked up catalog-first. If strict: True, only the catalog and global-level (in our case, world) is looked up. If dependency is not available locally, and strict: False, it would be looked up at application-level (which in our case, would be the application entrypoint's catalog) or at global level (in our case,world), but only ifstrict: false`. Overrides could then be implemented in terms of replacing the entrypoint's catalog with another, wich in turn imports the initial catalog. World could also be a last-level catalog, added by the resolution algorithm automatically.

With providers last, dependencies would always come from world first, if catalogs would import world in some fashion. Perhaps that's why you favor world dependencies to be explicit, but I don't think it's a good idea to force specific catalogs, not even world, because it would become unwieldly rather fast. Provider-last also means that the context-manager will be the only way for dependency overrides to happen -- which I can deal with, but it's a bit limiting.

In general, apart from factories, I don't think injection points (e.g. a type-annotated parameter or property) should bother with where the dependency comes from, at all. The only point at which it is relevant to specify a catalog, in my opinion, is on your application's entrypoint, or on catalog dependencies. The initial catalog selection is obviously important, but in any other point, neither library nor application authors should bother with where an injected parameter comes from.

Up to now, for factories, that degree of specificity was necessary so that importing the injected (implementation) class could be guaranteed. Now, the catalog is doing pretty much the same (guaranteeing that all the classes it offers are loaded), but in a well defined "DI package" that's explicitly selected by the user. That's why I don't think we need to be this specific.

In my understanding, a world dependency is there just so that it is available to all catalogs, regardless of whether they import each other or not. Such a feature should be used judiciously, though, e.g. a framework based on Antidote can provide some DI-managed registry service, but library authors using said framework should focus on providing their dependencies through catalogs.

Even when disconsidering world, if dependencies come from imported catalogs first, a bit of the benefits of having a catalog "scope" is lost, as local dependencies will be ignored in favor of external ones. For long chain of imports, the more front-facing catalogs lose a bit of relevance as dependency providers themselves, and would be mostly relegated to be simple catalog importers.

Also, by establishing catalogs and its inter-dependencies, we may have multiple injection candidates. The idea was to use catalogs not only to group those dependencies, but to also provide specialized ones. This is especially true for indirect providing (ie. when what's being requested is some ABC/interface). If imported catalogs have the preference, I will need to tell explicitly from which catalog each dependency comes from, either with context managers or some other mechanism, when said specializations are to be preferred. It's a level of specificity that I didn't want to enforce users with. It also makes it harder for library authors to provide said specializations, for the same reason. Qualifiers can alleviate this problem, but because of the resolution order, dependencies nearer the entrypoint catalog will have the least preference by default, which can be unintuitive.

I'm sorry for all this wall of text so far. I'll try to be brief going forward. :(

Good point! I'm not entirely sure we would have exactly the same objects in the query and class definition though. Qualifier query could be for example a "one of the qualifiers", "all of the qualifiers" or "all qualifier of this type". However, the API must feel symmetrical. And for performance, I would intuitively limit a "query predicate" to only access one kind of "class predicate", whether they're actually the same type or not. So it could be something like:

So, in a way, predicates have predicates. :D

All in all, I very much like the idea! It's explicit, and leaves the API open for extension in the future.

I think it could be a single class, but having specialized classes is a better call, perhaps (for extensibility, SRP and all that good SOLID stuff).

You're giving me other arguments not to use @, Perl.

(world.get[Database] @ factory).method()

Well, that's it, then. Axe the at!

[flisboac]

A catalog could hold other catalogs in which it'll search for dependencies and use its providers as last resort.

Regarding this again, I spent some time thinking about this resolution order specifically, and on second thought, it may make more sense for provider-last to be the default.

It guarantees a more deterministic resolution, because it will always go for the first provided dependency in the catalog hierarchy, much in the same way as class/module loaders are implemented in most languages. Once the dependency is found, that very same dependency is used again on later resolutions, something that won't be guaranteed if providers are checked first. Now, depending on the size of the catalog hierarchy, I'm not sure if it'll be easy for users to find where the dependency comes from, but from a "dependency loader" point of view, this makes more sense.

So, now I think your approach is the right way to go. But how will overriding work in this case (e.g. during tests)?

[Finistere]

Regarding this again, I spent some time thinking about this resolution order specifically, and on second thought, it may make more sense for provider-last to be the default.

To be honest, I don't have any strong opinions on this matter. In my previous comments, I only tried to have an understanding of what you would like and think about how it would impact the current code & API. I wrote it this way because it seemed natural with the idea of having a catalog overriding its parent dependencies. The providers would contain the dependencies of a specific catalog after all as they do for the Container currently. So having catalogs first means children override the parent catalog.

I'm sorry for all this wall of text so far. I'll try to be brief going forward. :(

On the contrary, I'm happy to have so much feedback. :) Thanks for the detailed example, indeed I've misunderstood it before! I didn't have yet the time to fully digest your comment. I need to take a deeper look at NestJS and play with dummy APIs to get a feeling on this. So I'm coming back on this later this week. :)

Intuitively what's important for me is for the API to be as clear as possible. It should be straightforward to understand which catalog is used. It'd be possible to do runtime inspection with world.debug(), but one shouldn't have to use this. It might be better to not provide context managers syntax at all. And instead only add catalog keyword to @inject, @factory, @service, etc... Anyone could then wrap those decorators with their own if you're using specific catalogs at several places. And adding the possibility to register a catalog to world: allowing libraries to expose specific dependencies and application entry points to add globally the catalogs they need.

In tests, it's fine to have context managers like world.test.clone() because they're usually short-lived or with a clearly defined context (function, class, module, global). But for application code, outside of the entry point, it doesn't sound like a good idea at first sight. And I want to avoid anything that could encourage poor code design. As said though, those are only thoughts nothing more yet. Coming back to you on this soon. :)

By the way, I'm considering renaming @service to @injectable based on earlier comments from you, what do you think?

[Finistere]

But how will overriding work in this case (e.g. during tests)?

For sure, one needs to be able to override dependencies of world or any specific catalog. This shouldn't be hard. Currently, Antidote completely replaces the global container in tests with one that supports overrides. And like catalogs are before providers in my previous example, overrides are executed before any provider. A similar logic would be applied. Catalogs would probably have a "test"-mode in which they would support overrides.

[Finistere]

By the way, I'm considering renaming @service to @injectable based on earlier comments from you, what do you think?

Or maybe @component, re-reading your earlier comments.

Regarding this again, I spent some time thinking about this resolution order specifically, and on second thought, it may make more sense for provider-last to be the default.

Actually, re-thinking about it led me to understand that my example wasn't clear. To me, intuitively child catalogs would be first but parent catalogs would be after providers. My example didn't specify whether catalogs were children or parents.

Anyway, just to let you know, I'll release 1.1 by the end of week and start working on the qualifiers for the 1.2.

[flisboac]

Or maybe @component, re-reading your earlier comments.

Well, the @injectable was supposed to denote a more generic concept, ie. any injectable dependency. @component sounds a bit more specific, like a service (ie. a component of a bigger application). I'm not sure if Antidote should determine or discern what is or what is not a component of your application, so @injectable sounds like a more reasonable description of what Antidote deals with, because it introduces a concept that's less charged with meaning into your domain (e.g. what if a "component" already means something else in some domain? it's a very common word in software development). But @componentis not a bad choice, because for most cases, that's what the injectables will effectively be.

To me, intuitively child catalogs would be first but parent catalogs would be after providers

Just to be sure we're on the same page regarding the terminology... The only mechanism by which you could be able to establish some kind of catalog hierarchy is via importing, right? In this case, the dependent catalog (child) imports its depending catalog (parent). So, in this case, the dependent catalog (child) would provide first, and only then would it provide from parents.

(You could also invert that logic, but well... Which will it be?)

For sure, one needs to be able to override dependencies of world or any specific catalog. This shouldn't be hard.

That's reassuring. I was a bit unsure as to how this could be implemented.

And instead only add catalog keyword to @inject, @factory, @service, etc...

Well, I'd add it only to @inject, because that could denote an application entrypoint, for the reasons I exposed in my previous comment.

@factory and @service are just injectables (dependencies), and they will be part of a catalog, not require one. Only catalogs import catalogs. Consider a catalog like you would a Python module, and it'll make sense. Defining the catalog in an @inject is only valid because the decorated element is outside the dependency injection mechanism (i.e. is not an injectable; it only receives injections).

Just as an example, from NestJS's documentation, this is how you create a module:

import { Module } from '@nestjs/common';
import { CatsController } from './cats.controller';
import { CatsService } from './cats.service';

@Module({
  controllers: [CatsController],
  providers: [CatsService],
})
export class CatsModule {}

Note the decoration there. We could even follow the same idea, and type-validate the new module via some Protocol type (instead of e.g. forcing CatsModule to inherit a Catalog class).

In NestJS, providers can be classes, as long as it is decorated with @Injectable.

Now, your application must run in terms of a "root module". Any module can be a root module. For example:

import { Module } from '@nestjs/common';
import { CatsModule } from './cats/cats.module';

@Module({
  imports: [CatsModule],
})
export class AppModule {}

Note the imports there. In this case, AppModule depends on and imports CatsModule. That's what I'm suggesting we follow as well. (You can even export imported modules, which is super useful to create modules that aggregates functionalities from multiple modules!)

To execute the app, you need to create an "entrypoint": it's either a script, or a function, etc, which instantiates a NestApplication from an initial module:

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
}
bootstrap();

The app.listen is only relevant if you have controllers in your app, which are REST endpoints NestJS automatically exposes via an internally managed express web server. In this case, CatsController would be served locally, at localhost:3000.

For Antidote, the approach should instead be more similar to standalone apps, because Antidote is not a web framework. That means some component of your app should be delegated as the entrypoint; it would then fetch and execute those managed services. For example:

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { SomeTask } from './some/nested/module/some-task.service'

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  const someTask= app.get(SomeTask);
  someTask.run();
}
bootstrap();

bootstrap is the entrypoint here. SomeTask is an injectable class from some module. It's either provided or imported by the "root" (app) module. (It may be hard to find from where SomeTask is being provided, but I think that's for the better, because in most cases this won't be relevant. You should depend on the interface, not the implementation (e.g. in Python terms, SomeTask could be just an ABC). I think just following the trail of module imports (or, in Antidote's case, world.debug) is a good compromise for allowing IoC and improving discoverability.)

What I suggested for @inject was some automation of this "entrypoint" logic. The entrypoint would not be injectable, but it could receive injections. Instead of doing gets, you could just decorate some function's parameters with the types you want to inject, and @inject would do the rest, provided you parameterize it with the catalog you want as a source/root.

Anyway, just to let you know, I'll release 1.1 by the end of week and start working on the qualifiers for the 1.2.

Excellent news! Thank you very much!

[pauleveritt]

Sorry for arriving late. I will say, I love re-reading this ticket. It's all stuff that fits my brain. Various comments...

Regarding sub catalogs, my current registry has nested registries. So:

site_registry = Registry()
request_registry = Registry(parent=site_registry)

At the end of a request, I throw out that registry.

For predicates, I did something called Predico which did predicates for Dectate apps. Each custom predicate could declare a score or say exact match was required. Best match was the one with the best score (and no vetos.) I tried to do a pre-optimized tree data structure for faster lookups, but I suck at that kind of thing.

In some of the discussion above about syntax, I'm having a lot of luck abusing dataclasses.field for this. As in, a LOT of luck. I make it really convenient for people who write providers to package up little custom fields. It's less typing, but it's also the fact that it is domain-specific jargon. I can provide an example if needed.

I'm also doing something called "operators" on injection. Such as:

@dataclass
class Greeter:
   customer_name: str = resource(attr="name")

This shows:

• a custom field resource which asks the injector for a Resource, then...
• ...plucks the name attribute off.

I probably could do better as a fluent style.

For the name component, I...well...hope you don't choose it. 😀I want that name for something that looks like a React component and returns a VDOM (which I have in my system.)

P.S. Eager to work with the new release and syntax.

[Finistere]

Still hadn't the time to (really) read the latest messages on Catalogs, but it's on my list I promise @flisboac. :) I might create another issue for this, but unsure since a lot of messages regarding this topic are in this issue.

@pauleveritt, didn't took a close look at Predico, but will do! For the dataclasses.field it's for another issue. :)

The good news is that I finally have a beta version of the predicates/interface/implementation/qualifier in the linked PR. The PR has two examples, one with interface/implementation/qualifier and another with a custom predicate. If anything feels wrong, I'm all ears!

[flisboac]

As qualifiers support landed in v1.2.0, this issue is in fact resolved.

As suggested by @pauleveritt and @Finistere, going forward, discussions about the catalogs feature should be conducted here.