Add data provider

[mirekdlugosz]

This is a proposal of new data providing facility. It's far from complete, but it may work for certain cases and I feel it's complete enough to enable discussion about something tangible.

This is fundamental framework work that - once completed - will enable us to tackle DISCOVERY-293. As it turned out, it will also enable to mostly tackle DISCOVERY-284. In other words, with this done, we will be set up to run all API and CLI tests in CI.

Commit 2f62d0f0e42684dd5acb04442b8798640015f9d1 is for illustrative purposes only. I will remove it before opening PR for merge.

The basic assumption is that most of the time, tests want data that is valid, and that can be safely removed afterwards. Sometimes tests want only data (i.e. negative object creation tests), sometimes they want created data (i.e. negative object update tests). Sometimes they want shared dependencies (to save time needed to create them), sometimes they want completely new chain of dependencies (to ensure independence from other tests).

Another important point is that data should be created lazily, only when actually needed. This is large problem with current setup - scans are run automatically when running any test, because some tests might need scan data - even if we deselected them all in current pytest session.

The data provider is meant as a helper for most common use cases, not mandatory solution for all your data needs. You can still create and maintain QPCModels manually, as you did in the past. (Although I do feel that data provider should be the only one responsible for data cleanup, no matter if objects were created manually or not.)

Open questions:

• Should dependencies be created through the same interface as object under test itself? Let's say you are running CLI test for scans. With current implementation, you ask data provider for valid data of scan, but source and credential of this potential scan are going to be created through HTTP API anyway.
• Using littletable for object querying. Alternative projects with similar goals are asq and PyFunctional. I picked littletable because it seems to be the most active right now. But whatever we pick, I'm not super happy that it is becoming the core API camayoc is depending on. On the other hand, I am picking something precisely because I don't want to invent my own iterable querying API.

[codecov[bot]]

Codecov Report

Merging #391 (6027752) into master (eb87cc2) will increase coverage by 2.22%. The diff coverage is n/a.

:exclamation: Current head 6027752 differs from pull request most recent head ad3ac85. Consider uploading reports for the commit ad3ac85 to get more accurate results

@@            Coverage Diff             @@
##           master     #391      +/-   ##
==========================================
+ Coverage   80.73%   82.96%   +2.22%     
==========================================
  Files           5        6       +1     
  Lines         244      317      +73     
==========================================
+ Hits          197      263      +66     
- Misses         47       54       +7     

see 3 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[bruno-fs]

I'm taking some time to digest. sorry for the delay 😅

[mirekdlugosz]

The CI is currently broken due to typing-extensions 4.6.0 (released earlier today) and pydantic: typing-extensions bug , pydantic bug. Locally you can pip install typing-extensions==4.5.0 and all unit tests will pass. I don't want to introduce any workarounds, as pydantic team is working now on releasing a fix, so this should start working again tomorrow or so.

[mirekdlugosz]

@quipucords/development @ruda this is now ready for review and merge. For current CI failure, see comment above.

I'm open to suggestions and comments, but at this point my plan is to merge it sooner rather than later and move all ssh key tests to data provider, so we can reap the benefits. Let the results speak for themselves.

[mirekdlugosz]

@bruno-fs We should adopt poetry, yes. DISCOVERY-298 is about that, sitting at low priority.