Suggestions For Improving Testing

[NarekA]

Describe your proposal/problem

I have a suggestions that could improve development experience specifically around testing. Forgive me if you've already considered these, but I thought this might be useful feedback.

1. Sharing Deployments Across Tests via Fixture Scopes

   I realize some tests are going to need custom deployments, but many of them can use a shared deployment/flow which is launched once at the beginning of the session. This means we only have to wait for the app to startup once.

   
   @pytest.fixture(scope="session")
   def hub_client(session_mocker):
       with Deployment(
           uses=MyExecutor, protocol="http",
       ) as dep:
           yield dep

2. Run CI Locally

   We can use Tox, Docker Compose, or some orchestration tool to allow us to reproduce CI locally. (If we already have this, I missed it and maybe I can update the docs to make it more clear)

3. Use pytest-xdist to run tests asynchronously

   I believe there have been some efforts to do this, and it might require some refactoring, but I saw a huge reduction in runtime when using multiple workers.

---

Environment

Screenshots

[JoanFM]

Hello @NarekA ,

Thanks for the thought process.

For 1. You are more than welcome to do so wherever you think that the same Deployment can be shared in multiple tests. For 2. It would be nice to have a clear Documentation on how to do so, but it is quite complex and I anticipate a lot of work to do that. And most of the time, the scope of the changes should be limited and thus it should be quite clear which tests are sensitive. For 3. I believe this is already being used in CI

[NarekA]

@JoanFM Thanks for the response. I totally get the dynamic nature of this project, hopefully, I'll be able to help more with Jina and its developer experience.