[BUG] net - resourcemanager - mgmt - ci does not run all steps for all mgmt plane

[m-nash]

Library name and version

Azure.ResourceManager

Describe the bug

We recently updated the ci for mgmt to only run net - resourcemanager - mgmt - ci for all mgmt plane libraries instead of individual ci's due to performance reasons. Commit is here. In the latest run here I noticed that the analyze step is only running for Azure.ResourceManager and NOT all of the other mgmt projects.

Our analyze step is able to deal with * as a ServiceDirectory and auto-expand we probably need to adjust that to treat resourcemanager as a wild card as well and run each mgmt library.

To find all mgmt projects we can use this code here. This excludes Azure.ResourceManager which we probably want to include when running analyze so just remove the extra . at the end of the filter.

Expected behavior

All mgmt plane libraries have their analyze step run

Actual behavior

only resourcemanager library has its analyze step run

Reproduction Steps

run the net - resourcemanager - mgmt - ci

Environment

No response

[m-nash]

For now @ArthurMa1978 please run /azp run net - [ServiceDirectory]- mgmt - ci for each rp one at a time to not clog the build system but still get the coverage

EDIT: actually this doesn't work because it gives the following error. @weshaggard please advise

[weshaggard]

For triggering the pipeline manually run /azp run net - [ServiceDirectory] - mgmt instead.

@sima-zhu once you get the work done to run the core pipeline code gen step in parallel done, can you also please do the same for the resourcemanager pipeline?

[ArthurMa1978]

@sima-zhu can you have an ETA of fixing this issue?

[weshaggard]

After further conversation we did not end up enabling this for the core pipeline because it turns out it is not needed so we removed the * from there. I was thinking we would do the same thing (i.e. not enable this) for resourcemanager however I just realized we disabled all the individual mgmt pipelines from triggering so we probably do still need this support for resourcemanager.

@sima-zhu can you use your work you did for core and try enabling it for resourcemanager pipeline as we do need to check all the mgmt libraries in this case.

[sima-zhu]

Analyze Step, check the box if it supports the project list

• [ ] Sparse-checkout (Java has the same setting
• [ ] Check file case duplicates, currently run all git tree.
• [ ] Spell checking, run PR only, input is changed files in PR.
• [ ] Link verification, run changed markdown files in PR. Run servicedirectory in nightly run.
• [ ] Verify sample metadata, run on servicedirectory/servicedirectoies. Same as sparse-checkout.
• [ ] Verify Readmes, run on Scanned path/Scanned paths. Path is some mutation on servicedirectory.
• [ ] Analyze Path Lengths. Run on servicedirectory only, need to extend to multiple service directories.
• [x] Verify generated code We have already achieved the purpose.
• [x] Build snippet. It can take in the project list as it is msbuild command using service.proj
• [ ] Validate any changes in eng/common. Scan the changes under eng/common For most of the tasks, support service directories is much easier than support project list.

[sima-zhu]

Further categorizations:

1. Taking project list:
   • Verify generated code
   • Build snippet
2. Taking changed files:
   • Check file case duplicates
   • Link verification
   • Validate any change in eng/common
   • Spell checking
3. Taking multiple service directories:
   • Sparse-checkout
   • Verify sample
   • Verify readme
4. Taking single service directory:
   • Analyze Path Lengths

[sima-zhu]

The steps in Build

1. Taking project list:
   • Build and Package: Taking project list
   • Detect API changes: Taking the artifacts from build package steps. Condition on: 1. artifact is newsdk and 2. changed files are in artifacts.
   • Build DocFx Documenttion Site: Same. Depend on the artifacts from build
2. Single service directory:
   • Update package properties with dev version

[sima-zhu]

The steps in Compliance:

1. The job scans against changed files in PR.

[m-nash]

@sima-zhu and @weshaggard The original intention of this issue is now complete. Are we using this issue to track follow up work as well or is it safe to close this?

[weshaggard]

I'd like to keep this open for now because what we have is still in testing from my perspective.

[weshaggard]

@benbp this is a case study when you start thinking about the pipeline changes.

[github-actions[bot]]

Hi @m-nash, we deeply appreciate your input into this project. Regrettably, this issue has remained unresolved for over 2 years and inactive for 30 days, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.