search

DMTF / Redfish-Protocol-Validator

The Redfish Protocol Validator tests the HTTP protocol behavior of a Redfish service to validate that it conforms to the Redfish specification.
Other
14 stars 12 forks source link

Test query parameters #46

Open FighterNan opened 2 years ago

FighterNan commented 2 years ago

Is query parameters testing in the roadmap of this repo?

If so, what's the planned workflow? I would imagine we firstly check the |ProtocolFeaturesSupported| on service root, if certain parameters are supported, we pick some resources (or walk through every resource), apply query parameters on them, and finally check the result according to these rules:

  1. only; response shall be exactly the same as querying the resource directly
  2. expand; response shall be exactly the same as querying the list of sub-resources sequentially
  3. select; selected properties and values shall be exactly the same as they are in the original response; revered properties are turned regardlessly
  4. filter; pending..

Any better ideas?

mraineri commented 2 years ago

We currently have that testing done under the Redfish-Usecase-Checkers project here: https://github.com/DMTF/Redfish-Usecase-Checkers/tree/master/query_parameters

mraineri commented 2 years ago

I do wonder if it's worth migrating that testing to the protocol validator though; now that I think about it a bit more, it's not really a "usecase".

@jautor do you have any opinion?

jautor commented 2 years ago

There's certainly some amount of validation we should do in the Protocol-Validator since we can detect the support based on the ProtocolFeaturesSupported. But there may be limits on what we can reasonably test consistently across implementations - so I expect we'd still have more exhaustive (and more flexible) test cases in the use case checker regardless.

mraineri commented 2 years ago

I think given the scope of the existing usecase checker, we can add the same determinism since it also is relying on service root to "discover" what is supported. At this time, there's nothing too exotic in my opinion.

mraineri commented 2 years ago

Migrated issue to the protocol validator (service validator is strictly resource conformance against schema).