search

Azure / azure-sdk-tools

Tools repository leveraged by the Azure SDK team.
MIT License
108 stars 166 forks source link

Pre-requisites data automation for attestation automation of APEX launch criteria (S360 KPIs) #8225

Open ladonnaq opened 1 month ago

ladonnaq commented 1 month ago

Data Automation Requirements

Data Automation

ladonnaq commented 1 month ago

Reviewed requirements and updated relevant tabs in spreadsheet for engineering.

maririos commented 3 weeks ago

@ladonnaq I have the following questions after looking at the data:

The more I read on the data and the significance of the Not applicable value, it makes me think that the Product is the one that isAPEX . So in reality, the question should be asked during onboarding and saved in the product. When a release plan is created, instead of asking the question, we populate the isAPEX value with the same value as the product. that way you can distinguish release plans that are APEX and the ones that are not.

With that in mind, we can also simplify the rules. For example, if a product onboards and it is in scope for APEX:

Now, looking at the specifics:

Generic question for the APEX launch criteria fields. Using as example the field Mgmt PrivPrev API Readiness Launch Criteria:

Questions for Lifecycle State updates:

ladonnaq commented 2 weeks ago

@ladonnaq I have the following questions after looking at the data:

The more I read on the data and the significance of the Not applicable value, it makes me think that the Product is the one that isAPEX . So in reality, the question should be asked during onboarding and saved in the product. When a release plan is created, instead of asking the question, we populate the isAPEX value with the same value as the product. that way you can distinguish release plans that are APEX and the ones that are not.

With that in mind, we can also simplify the rules. For example, if a product onboards and it is in scope for APEX:

  • If they are only data plane, then all the management plane fields for both launch criteria and lifecycle state will be Not applicable.
  • If they are yes to both, then leave those fields empty and wait until there is a release plan created
  • If they say unsure, then they meet with you, and after meeting with you, you can ask them to onboard again and update the scope so automation can do its thing.

Now, looking at the specifics:

  • With the addition of the isAPEX field in the Release Planner work item, I am noticing that we could then change the behavior of the field Release plan type. Currently automation is always doing APEX <release type>. But we could now start just doing <release type> where the values are Private Preview, Public Preview, GA. We won't be able to go back in time and change previous records though.

Generic question for the APEX launch criteria fields. Using as example the field Mgmt PrivPrev API Readiness Launch Criteria:

  • The document says we should use Not Applicable but the field value is N/A. I want to confirm we are looking at the same thing here. And if so, could you update the requirements so they reflect the exact value we should use per field?

Questions for Lifecycle State updates:

  • GA State, PrivPrev State, and PubPrev State are being automatically populated by automation when the Product work item is created. This information is coming from Service Tree. I see in your requirements that we always override that value. Question is, should we then disable the automation to assign those values during creation time for the Product work item?
  • It looks like the equivalent of Not applicable changes according to the field. @ladonnaq could you please update the requirements so that the value is specific to the field? For example, the value for the fields DataPrivPrevState and MgmtPrivPrevState is Not Applicable (N/A) while for DataPubPrevState is N/A.

  • Requirement: IF Release Plan Type = "Private Preview" AND release plan MgmtScope = "yes" AND the epic-product type DataScope ="yes", THEN check value of DataPrivPrevState. If DataPrivPrevState DOES NOT EQUAL "Incomplete", PrivPrev State = DataPrivPrevState. IF DataPrivPrevState EQUALS "Incomplete", THEN PrivPrev State = "Incomplete"

    • Looks like no matter what the value of DataPrivPrevState is, the result is always PrivPrev State = DataPrivPrevState

    • Why would I check data plane values if I am working on a management plane release?

    • What happens when there is a product that has both management plane and data plane scopes?. For example, take the values:

    • DataPrivPrevState = Incomplete

    • MgmtPrivPrevState = Complete`

    • Release plan scope = Data plane

    • Would then PrivPrevState = Complete? even though there is active work happening in data plane?

    • If the last point is valid, there there is no work to be done when a release plan is completed and the product is both data and management plane as the value won't be changed because of the work that happened in the release plan.

@maririos Let's discuss today during the engagement experience synch meeting. The interim between lifecycle milestone's can be 2 years for some products, so service partners may release new SDK versions after completing Public Preview and before GA. The APEX signoff is associated with the release plan for the lifecycle milestone approval. Once the APEX signoff is complete for the lifecycle milestone, all other release plans targeting that lifecycle would not be "APEX" release plans. This is why I suggested identifying the APEX scope during creation of the release plan. We can go through your other questions as well and make sure that we are all on the same page.