Question on testing with python-dependency-injector

[asyncee]

Hello and thank you for such a great library.

I have a question regarding of how to test application, that uses python-dependency-injector library.

Lets take simple usecase:

class EmailSender:
    def send(self, email):
        pass

class SmtpEmailSender:
    # implementation with use of smpt library

class EchoingEmailSender:
    # logging / printing to stdout implementation

def notify_users(email_sender, email):
    email_sender.send(email)

In production i want to use SmtpEmailSender, but in tests only EchoingEmailSender. I have configured a container which provides me with production-ready class of EmailSender and using it like:

Services.notify_users(email)

So, notify_users get production-ready dependency injected.

So the question is: how do i switch implementation in tests? Surely i can override this specific dependency, and it will work okay, but what if i have 10 containers with different providers, that is used by application, should i override them in every test i write? I think it can become an error-prone approach.

Thanks.

[rmk135]

Hi @asyncee ,

Well, yeah, there is a solution exactly for what you're asking about. You can override provider - it's one of the main features. Please a look on this example - https://github.com/ets-labs/python-dependency-injector/blob/master/examples/miniapps/mail_service/container.py

In particular, please, pay attention to line #36 and #41. I often do something like this:

class SomeTestCase(unittest.TestCase):
    def setUp(self):
        Container.mail_service.override(Singleton(MailServiceStub))
        self.addCleanup(Container.mail_service.reset_last_overriding)

Link to the documentation - http://python-dependency-injector.ets-labs.org/providers/overriding.html

Hope it helps.

Thanks a lot for you question and feedback. Please, feel free to file more issues when needed - I would be pleased to help.

Thanks, Roman

[asyncee]

Roman, thank you for such a detailed answer and links to documentation.

Currently i can solve my problem by using suggested approach, and for smaller projects it will work great.

But imagine that you are working on a large project with large / frequently changing codebase. At some point of time there are possibility that one can forget to override a provider and tests will go to physical test database or test email sender service instead of mocks.

That may slower tests or introduce different problems, since we can not provide a contract (abstract base class, for example) for a container in a natural way supported by library. Of course we can write wrapper around container and implement methods like get_mail_service_factory or something like that, but it is overkill i think.

What we can do is to introduce the concept of a "Component", that will be composed of few containers and use that to retreive dependencies, something like dagger 2 components in Java.

Actually, while I was writing this message, I came up with the idea that i can manually implement such a global container factory and request container with dependencies from it. In production environment i can register production containers in it, and in test – mocked containers.

I'll close the issue, but it will be great, if you'll write down your thoughts on this.

Thanks!

[rmk135]

Short question, what if we make provider overriding in a global scope? Let's say before running all the tests?

# Somewhere in tests runner...
Container.mail_service.override(Singleton(MailServiceStub))

# Tests
class SomeTestCase(unittest.TestCase):
    def test(self):
        assert isinstance(Container.mail_service(), MailServiceStub)

class AnotherTestCase(unittest.TestCase):       
    def test(self):
        assert isinstance(Container.mail_service(), MailServiceStub)

In example above everything that depends on Container.mail_service will receive a MailServiceStub. Does it make sense?

---

Another case is when you have a container with more of such providers that you want to mock in dev/test environment, you can use feature of overriding containers:

http://python-dependency-injector.ets-labs.org/containers/overriding.html

Please, also take a look on this example - http://python-dependency-injector.ets-labs.org/examples/movie_lister.html

What do you think of this ? https://github.com/ets-labs/python-dependency-injector/blob/fee8530b1e9a58c067e741682a9c2310cfdd98cb/examples/miniapps/movie_lister/app_csv.py#L25-L26

---

You know, I will re-open issue and really ask for more feedback in terms of describing solution that could work for you best way. What you're looking for sounds like something that this library should be making out-of-the-box, so if it does not, it requires improvements :)

Thanks a lot, Roman

[asyncee]

Feature of overriding containers and providers is very useful and absolutely can be and should be used in testing.

But the problem i'm trying to ask you about is a little bit different – i want to find out how can we provide a contract for new developer, that is new to project's codebase, it's class hierarchy and dependencies graph.

By contract i mean something that will constrain developer from producing an errors, in python — ABC's for example.

Something, that will work even if programmer does not know dependency graph, which can be deeply nested.

In more detail, i think about such flows in testing:

Fail story:

1. Developer overrides containers for tests
2. Runs tests
3. Finds out unwanted side effects on his computer / database / redis server / etc, because he did not override .

Success story:

1. Developer configures Component that represents dependency graph with mocked container
2. Runs tests
3. Tests are failing because of missing dependencies, because developer forgot to add needed containers / providers

So, i think that there are should not be any pre-configured containers in testing and should be a way to separate containers configuration between unit tests / integration tests / production.

Although maybe i have too little experience with python-dependency-injector and just don't know best practices yet to avoid such a mistakes.

[rmk135]

Alright, I think I've got it now...

I know that I've put a lot of examples from existing docs, but can I, please, ask you to look onto one more thing? :)

That's it - http://python-dependency-injector.ets-labs.org/examples/bundles_miniapp.html

In particular, I'm taking about this part - http://python-dependency-injector.ets-labs.org/examples/bundles_miniapp.html#run-application

https://github.com/ets-labs/python-dependency-injector/blob/d5ac1474d40ca2b356219aaee08cbe09a53aa91b/examples/miniapps/bundles/run.py#L25-L31

Btw, one important part of this is a Dependency provider:

https://github.com/ets-labs/python-dependency-injector/blob/d5ac1474d40ca2b356219aaee08cbe09a53aa91b/examples/miniapps/bundles/bundles/users/__init__.py#L10-L18

It can make things similar to what ABC does:

https://github.com/ets-labs/python-dependency-injector/blob/d5ac1474d40ca2b356219aaee08cbe09a53aa91b/examples/providers/dependency.py#L47-L62

[asyncee]

Thank you for help, your time and detailed answers, you helped me a lot!

I will try each of suggested approaches.

And last question — i have such components

class Adapters(containers.DeclarativeContainer):
    email_sender = providers.Singleton(SmtpEmailSender)

class UseCases(containers.DeclarativeContainer):
    signup = providers.Factory(SignupUseCase, email_sender=Adapters.email_sender)
    ... more usecases with dependency on Adapters container ...

As you can see there are direct dependency on specific class Adapters. Are there any way to provide Adapters container implementation to UseCases class, just like providers.Dependency, for example?

# Example code
class Adapters(containers.DeclarativeContainer):
    email_sender = providers.Singleton(SmtpEmailSender)

class TestAdapters(containers.DeclarativeContainer):
    email_sender = providers.Singleton(EchoEmailSender)

class UseCases(containers.DeclarativeContainer):
    adapters = providers.Dependency()  # Want to use container as dependency 

    signup = providers.Factory(SignupUseCase, email_sender=adapters.email_sender)

use_cases = UseCases(adapters=Adapters)
# or
use_cases = UseCases(adapters=TestAdapters)

# Another file, views.py
from .containers import use_cases

use_case = use_cases.signup()
use_case.execute()

I do understand that this approach may be implemented with @overrides :)

[rmk135]

As you can see there are direct dependency on specific class Adapters. Are there any way to provide Adapters container implementation to UseCases class, just like providers.Dependency, for example?

Not yet possible, and sounds like a good feature candidate. Thinking of DependenciesContainer provider...

[asyncee]

Yes, exactly, this feature is available in some java libraries by default, so you can have different implementations of DependenciesContainer, with single interface, which is quite useful for consistency and testing.

Thank you very much for your support, i think issue can be closed, if you do not need any more feedback on.

P.S.: i'll be highly waiting for such feature :)

[rmk135]

P.S.: i'll be highly waiting for such feature :)

Will ship it sooner than later... U're welcome to follow the progress in terms of #178.

Your contribution is very valuable and welcome - thank you very much!

Respectfully, Roman