[BUG] Extractor pipeline still extracts unnecessary version sets instead of extracting only version set of API being extracted

[JamesProant]

Release version

6.0.1

Describe the bug

When using extractor pipeline, all version sets are being extracted from all of our API's in the APIM instance, instead of only the version set of the specific API being extracted. Feature req-(https://github.com/Azure/apiops/issues/529#issuecomment-2207007896) was declared as solved, but same behavior is happening.

Expected behavior

Only version set of API being extracted should be added to artifacts.

Actual behavior

Every version set from our APIM instance is being extracted and added to artifacts.

Reproduction Steps

Run extractor pipeline for API-A. New branch with artifacts contains every version set for every API in APIM instance.

[github-actions[bot]]

  Thank you for opening this issue! Please be patient while we will look into it and get back to you as this is an open source project. In the meantime make sure you take a look at the [closed issues](https://github.com/Azure/apiops/issues?q=is%3Aissue+is%3Aclosed) in case your question has already been answered. Don't forget to provide any additional information if needed (e.g. scrubbed logs, detailed feature requests,etc.).
  Whenever it's feasible, please don't hesitate to send a Pull Request (PR) our way. We'd greatly appreciate it, and we'll gladly assess and incorporate your changes.

[guythetechie]

We don't limit version sets by API because they can exist independently. A version set can have no API, one API, or multiple APIs.

To only extract specific version sets, update your configuration.extractor.yaml like this:

versionSetNames
- version-set-to-extract

[JamesProant]

Hello @guythetechie

Thank you for your quick reply.

I have tried adding that parameter to the configuration.extractor.yaml file, but it seems to not work properly. In fact whenever I add the parameter "versionSetNames" to the config file, the extractor no longer extracts any api artifacts at all.

Any idea why this behaviour is happening?

[guythetechie]

Sure.

If you put version set names under versionSetNames, we will only extract those version sets. We will also filter APIs and only extract APIs associated with those version sets.

Make sure that the names under versionSetNames match the version set names/IDs, not their display names. Easy way to see the correct names is to remove the version set filter in configuration.extractor.yaml, extract everything, and then look at the folder names in /artifacts/version sets/.

[guythetechie]

To expand on the API/version set filter relationship, we filter APIs like this:

• Is there an apiNames filter in configuration? If yes, get those APIs; if not, get all APIs.
• For each API, check whether it has a version set.
  • If it has no version set, extract it
  • If it has a version set, check whether there is a versionSetNames filter in configuration
  • If there is no filter, extract the API
  • If there is a filter, only extract the API if its version set is in the versionSetNames filter.

[ash9368]

only change i have to do in the configuration.extractor.yaml file to extract the API version or is there anything also i have to do