avocado-framework / avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
https://avocado-framework.github.io/
Other
345 stars 344 forks source link

[RFC] Creating utility/helper library specifically for avocado-misc-tests #4049

Open harish-24 opened 4 years ago

harish-24 commented 4 years ago

Avocado-misc-tests use a lot of code which is repetitive across tests. The obvious solution was to use avocado API, and we are continuing to do so. But there are still quite a few use cases which still cannot go in as a plug-gable avocado API as it is may not be useful outside as a usable functionality, but only for avocado-misc-tests repository.

A basic example would be steps to install packages for each test.

            for pkg in deps:
                if not self.smm.check_installed(pkg) and not self.smm.install(pkg):
                    self.cancel('%s is needed for the test to be run' % pkg)

There should be a few more which can still be moved as helper libraries.

As a reflection of what has been discussed here, requesting to have a helper library specific to avocado-misc-tests so that we can minimize the code redundancy in tests.

@clebergnu @beraldoleal @willianrampazzo Your thoughts here would be helpful.

beraldoleal commented 4 years ago

@harish-24 I think its great you are thinking about move things inside Avocado. +1 here. But in this case/example that you gave maybe we don't need to move right now, because we (mostly @willianrampazzo) are working on a "requirements resolver" that will handle not only package installation but all kind of requirements for a test.

willianrampazzo commented 4 years ago

@harish-24 I think its great you are thinking about move things inside Avocado. +1 here. But in this case/example that you gave maybe we don't need to move right now, because we (mostly @willianrampazzo) are working on a "requirements resolver" that will handle not only package installation but all kind of requirements for a test.

That is right, the initial work on the Requirements Resolver is scheduled to be introduced to the N(ext) Runner architecture, probably on release 83 or 84, after the release of LTS.

There were also some discussions about it here: https://github.com/avocado-framework/avocado/pull/3678

harish-24 commented 4 years ago

@harish-24 I think its great you are thinking about move things inside Avocado. +1 here. But in this case/example that you gave maybe we don't need to move right now, because we (mostly @willianrampazzo) are working on a "requirements resolver" that will handle not only package installation but all kind of requirements for a test.

@beraldoleal @willianrampazzo Good to know. Another example would be copying test binaries/source files from data directory similar to here. What do you recommend here?

willianrampazzo commented 4 years ago

@harish-24 I think its great you are thinking about move things inside Avocado. +1 here. But in this case/example that you gave maybe we don't need to move right now, because we (mostly @willianrampazzo) are working on a "requirements resolver" that will handle not only package installation but all kind of requirements for a test.

@beraldoleal @willianrampazzo Good to know. Another example would be copying test binaries/source files from data directory similar to here. What do you recommend here?

As far as I can think of, this is still a requirement of a file. This case will also be covered by the requirements resolver. You can see all cases that we planned to initially support here: https://avocado-framework.readthedocs.io/en/80.0/blueprints/BP002.html

harish-24 commented 4 years ago

@willianrampazzo Thanks, that looks comprehensive. From the description this means we may not need a setUp going forward. meaning whatever we are achieving through setUp of a test will be replaced by Requirements resolver, right?

If that is the case then I would wait for that to be in place so that this issue can also be satisfied.

willianrampazzo commented 4 years ago

@willianrampazzo Thanks, that looks comprehensive. From the description this means we may not need a setUp going forward. meaning whatever we are achieving through setUp of a test will be replaced by Requirements resolver, right?

This is true for any kind of requirement covered by the Requirements Resolver. The setUp will still be useful to set other requirements not covered by the resolver, for example, a temporary directory.

harish-24 commented 4 years ago

@willianrampazzo Thanks again. I understand lot of work has been put in to make these changes, please reach out for any help. Btw any approx. date estimate we can probably expect it arrive so that we can align accordingly?

willianrampazzo commented 4 years ago

@willianrampazzo Thanks again. I understand lot of work has been put in to make these changes, please reach out for any help. Btw any approx. date estimate we can probably expect it arrive so that we can align accordingly?

We are working hard to have a good LTS on release 82. After that, the plan is to introduce the Requirements Resolver, so we should have an initial work on release 83 or 84. If everything goes fine and we keep the pace of 3 weeks sprint starting on 82, we are talking about 6 to 9 weeks after the release of 81.