search

Azure / azure-sdk-tools

Tools repository leveraged by the Azure SDK team.
MIT License
110 stars 172 forks source link

SDK Review Meeting prerequisites check feature (proposed for 2H DI semester) #7676

Open ladonnaq opened 7 months ago

ladonnaq commented 7 months ago

Requirement: Trends in SDK Review Meetings shows that some service partners are scheduling SDK reviews far in advance and then either (a) showing up to the meeting and not completing all of the prerequisites (b) cancelling at the last minute which does not give enough time for another team to take that slot

Prerequisites Intro: (First Beta, service team has changed scope significantly, service partner has not been to an arch board session in around 2 years)

  1. API Readiness is complete (which means that API Stewardship Board has signed off on the API version).
  2. Generate the API Views for SDK review. (1 API View be prepared)
  3. Hero/Champion scenario link has been uploaded to the meeting invite.
  4. Write samples for generated library. (Samples written for 1 API View) - Create issue to confirm that samples are added to APIView.

Prerequisites SDK Review - Follow-up Meetings: (Stable SDKs)

  1. API Readiness is complete (which means that API Stewardship Board has signed off on the API version).
  2. API Views are autogenerated from PRs from the language repos.
  3. Hero/Champion scenario link has been uploaded to the meeting invite.
  4. Write samples for generated library.

TODO: Rename the Follow-up meetings to Stable SDK Review. TODO: Determine how to handle service partners who do not use our repos for their SDK code. TODO: Adding non-Tier 1 languages to the data plane SDK release milestone to release planner. TODO: On scheduling side surface the languages that were selected in the release planner.

Cancellation Strategy

### Tasks
- [x] Confirm all requirements for prerequisite are identified.
- [x] Determine which of the prerequisites can be automatically validated.
- [ ] Determine what the cutoff time is for meeting the prerequisites before the meeting is cancelled.  I would suggest 5 business days/1 week to give time for SDK arch board to review artifacts and also time for another team to grab the timeslot if cancelled.
- [ ] Identify any UI requirements.
- [ ] Identify what tools will be impacted (i.e. Release Planner, Scheduling tool, ...).
ronniegeraghty commented 7 months ago

Hi @ladonnaq, I think we need to call out that the Prerequisites for SDK Reviews are different depending on the type (Intro or Follow Up).

Prerequisites:

Introduction SDK Review:

Note: This review is commended before any public preview but is not a hard requirement.

  1. API Readiness is complete (which means that API Stewardship Board has signed off on the API version).
  2. One API View Listing (for an intro meeting this can be taken from the PR to the API Specs repo).
  3. Hero Scenarios (samples that use methods from the API View listing chosen above).

Follow Up SDK Review:

Note: This review is required before a GA release of the libraries.

  1. Introduction SDK Review is complete (Encompassing all the intro meeting requirements).
  2. Generate data plane SDKs
  3. Open pull requests to the language repos for the generated libraries.
  4. API Views for all languages release a GA version. Tier 1 languages are required, but if a team in planning to also release Go or other language libraries they also need to be included. We'd like to require that these libraries are generated from a pull request to the specific language repo. Doing so will help minimize duplication API View listings and the need for any additional API View approvals before release.
  5. Hero Scenarios (these can be exactly the same as scenarios used in the intro meeting, they just need to be updated to reflect any changes made in the libraries' surface area since the intro).

Also, with most things we'll need a way to override these checks for the strange one-off scenarios like:

ronniegeraghty commented 5 months ago

Checking and Reminder Sending Cadence:

When to start sending reminders.

Email Template for reminders:

Hello \<Meeting Scheduler>, Thank you for scheduling an Azure SDK Review for \<Meeting Date>. This is a reminder to add required materials to your meeting at least 2 business days before the scheduled time. This will give our architects enough time to review them and prepare for the meeting. Looks like you are still missing the following required materials:

  • \<List missing materials>
  • \<Intro meeting> An API View listing in at least 1 of the tier 1 programming languages (.NET, Java, JS, or Python). This can be taken directly from your Pull Request to the azure-rest-api-specs repo.
  • \<Intro Meeting> Code Samples (These should use the APIs from an API View you've submitted)
  • \<Stable Release Meeting> API View listings in the following languages: . These API View will be automatically generated when you create a pull request into the Azure SDK Language repos.
  • \<Stable Release Meeting> Code Samples (These can be written in any of the languages you've submitted any API View for)

If these materials are not added within 2 business days of the schedule meeting time, this meeting will be automatically canceled.

If you have any questions about the materials required for you meeting, please reference our Engineering Hub Docs.

Thank you.

ronniegeraghty commented 5 months ago

Also, adding a note. We might want to give admins the ability to override materials requirements for specific meetings. For when an arch board is scheduled for a more abstract conversation instead of a review.

ladonnaq commented 5 months ago

[heart] LaDonna Quinn reacted to your message:

From: Ronnie Geraghty @.> Sent: Wednesday, March 27, 2024 10:06:29 PM To: Azure/azure-sdk-tools @.> Cc: Author @.>; Assign @.> Subject: Re: [Azure/azure-sdk-tools] SDK Review Meeting prerequisites check feature (proposed for 2H DI semester) (Issue #7676)

Also, adding a note. We might want to give admins the ability to override materials requirements for specific meetings. For when an arch board is scheduled for a more abstract conversation instead of a review.

— Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-sdk-tools/issues/7676#issuecomment-2024071789 or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQEFYOQCOF7HKGE4NGLEDXDY2M7GNBFKMF2HI4TJMJ2XIZLTSWBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVIYTGNZUG43DINRYGCSG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRUGI4DKMBUGU2DOMNENZQW2ZNJNBQXGX3MMFRGK3FMON2WE2TFMN2F65DZOBS2YSLTON2WKQ3PNVWWK3TUUZ2G64DJMNZZJAVEOR4XAZNKOJSXA33TNF2G64TZUV3GC3DVMWUTCNZQGU4TEMJYG2BKI5DZOBS2K2LTON2WLJLWMFWHKZNKGIYTGMZQGI2DAMZWQKSHI6LQMWSWYYLCMVWKK5TBNR2WLKRRGM3TINZWGQ3DQMECUR2HS4DFUVWGCYTFNSSXMYLMOVS2UNBSHA2TANBVGQ3TDJ3UOJUWOZ3FOKTGG4TFMF2GK. You are receiving this email because you authored the thread.

Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

ladonnaq commented 5 months ago

[heart] LaDonna Quinn reacted to your message:

From: Ronnie Geraghty @.> Sent: Wednesday, March 27, 2024 10:05:21 PM To: Azure/azure-sdk-tools @.> Cc: Author @.>; Assign @.> Subject: Re: [Azure/azure-sdk-tools] SDK Review Meeting prerequisites check feature (proposed for 2H DI semester) (Issue #7676)

Checking and Reminder Sending Cadence:

When to start sending reminders.

Email Template for reminders:

Hello , Thank you for scheduling an Azure SDK Review for . This is a reminder to add required materials to your meeting at least 2 business days before the scheduled time. This will give our architects enough time to review them and prepare for the meeting. Looks like you are still missing the following required materials:

If these materials are not added within 2 business days of the schedule meeting time, this meeting will be automatically canceled.

If you have any questions about the materials required for you meeting, please reference our Engineering Hub Docshttps://eng.ms/docs/products/azure-developer-experience/develop/sdk-review?tabs=what-to-bring.

Thank you.

— Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-sdk-tools/issues/7676#issuecomment-2024070357 or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQEFYOQ5BQVMFDMVWR7C7ILY2M7CFBFKMF2HI4TJMJ2XIZLTSWBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVIYTGNZUG43DINRYGCSG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRUGI4DKMBUGU2DOMNENZQW2ZNJNBQXGX3MMFRGK3FMON2WE2TFMN2F65DZOBS2YSLTON2WKQ3PNVWWK3TUUZ2G64DJMNZZJAVEOR4XAZNKOJSXA33TNF2G64TZUV3GC3DVMWUTCNZQGU4TEMJYG2BKI5DZOBS2K2LTON2WLJLWMFWHKZNKGIYTGMZQGI2DAMZWQKSHI6LQMWSWYYLCMVWKK5TBNR2WLKRRGM3TINZWGQ3DQMECUR2HS4DFUVWGCYTFNSSXMYLMOVS2UNBSHA2TANBVGQ3TDJ3UOJUWOZ3FOKTGG4TFMF2GK. You are receiving this email because you authored the thread.

Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

maririos commented 5 months ago

Also, adding a note. We might want to give admins the ability to override materials requirements for specific meetings. For when an arch board is scheduled for a more abstract conversation instead of a review.

ohh this is interesting. Could you expand more on this? how will it work?

ronniegeraghty commented 5 months ago

ohh this is interesting. Could you expand more on this? how will it work?

My thought was that, there could be a check box in the admin tool that said something like:

[ ] Exempt from materials requirements

And if checked the meeting does get check for it's materials, set reminders, or auto canceled. One example is the meetings that arch board has been having around Aspire. There aren't always libraries prepared since the architects are giving pre-implementation advice to the Aspire team.

ladonnaq commented 5 months ago

[heart] LaDonna Quinn reacted to your message:

From: Ronnie Geraghty @.> Sent: Wednesday, March 27, 2024 11:03:52 PM To: Azure/azure-sdk-tools @.> Cc: Author @.>; Comment @.>; Assign @.***> Subject: Re: [Azure/azure-sdk-tools] SDK Review Meeting prerequisites check feature (proposed for 2H DI semester) (Issue #7676)

ohh this is interesting. Could you expand more on this? how will it work?

My thought was that, there could be a check box in the admin tool that said something like:

[ ] Exempt from materials requirements

And if checked the meeting does get check for it's materials, set reminders, or auto canceled.

— Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-sdk-tools/issues/7676#issuecomment-2024125460 or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQEFYOULGV74QIYSO55PXLDY2NF5TBFKMF2HI4TJMJ2XIZLTSWBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVIYTGNZUG43DINRYGCSG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRUGI3TONBWGI2TKNNENZQW2ZNJNBQXGX3MMFRGK3FMON2WE2TFMN2F65DZOBS2YSLTON2WKQ3PNVWWK3TUUZ2G64DJMNZZJAVEOR4XAZNKOJSXA33TNF2G64TZUV3GC3DVMWUTCNZQGU4TEMJYG2BKI5DZOBS2K2LTON2WLJLWMFWHKZNKGIYTGMZQGI2DAMZWQKSHI6LQMWSWYYLCMVWKK5TBNR2WLKRRGM3TINZWGQ3DQMECUR2HS4DFUVWGCYTFNSSXMYLMOVS2UNBSG43TINRSGU2TLJ3UOJUWOZ3FOKTGG4TFMF2GK. You are receiving this email because you authored the thread.

Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

maririos commented 5 months ago

And if checked the meeting does get check for it's materials, set reminders, or auto canceled. One example is the meetings that arch board has been having around Aspire. There aren't always libraries prepared since the architects are giving pre-implementation advice to the Aspire team.

So the flow will be:

Is this correct?

ronniegeraghty commented 5 months ago

Is this correct?

Yes. We can add a thing to the reminder email that says:

If you feel these materials don't apply to your review meeting, please contact the Azure SDK Architecture Board Help Desk(azsdkarch-help@microsoft.com).