search

Azure / azure-sdk-tools

Tools repository leveraged by the Azure SDK team.
MIT License
109 stars 166 forks source link

TypeSpec data related to interval between start and completion of tasks, milestones, and overall release plan #7769

Open MrJustinB opened 4 months ago

MrJustinB commented 4 months ago

Related GitHub issues:

Most of metrics requested in related GitHub issues is not accessible via current engagement data model. We can revisit when we begin planning for Selenium to determine feasibility of obtaining the data.

Meeting notes: • TypeSpec data dashboard: Justin and Mariana will work on creating a dashboard that shows the average and outlier times for service teams to go through the TypeSpec pipeline, as requested by Mario, the TypeSpec PM. We discussed a visual timeline with major milestones along the timeline and average time to get to each milestone and then show the outliers for each (i.e. which service teams were slower and which were faster) so that Mario can follow up to understand challenges faced. • Data plane vs management plane: Mario wants to see the distinction between data plane and management plane services in the dashboard, to understand if there is a significant difference in their TypeSpec experience. • TypeSpec adoption: Mario is less interested in tracking the adoption of TypeSpec, since it is now mandated for all service teams, but he would like to see how many services have moved to TypeSpec as a secondary metric. Follow-up tasks:

MrJustinB commented 4 months ago

Mario: "Thanks Justin, one other thing that just occurred to me – it would be good to differentiate between greenfield and brownfield services in the averages."

maririos commented 4 months ago

Ideas on what we should do:

ladonnaq commented 3 months ago

@MrJustinB @maririos I believe that you are using this issue to track the work that you are doing for Mario for TypeSpec dashboard. I have added a table to track the metrics and information needed. I wanted to sync on the approach and todos so we capture for now and next semester.

image.

maririos commented 3 months ago

I wanted to sync on the approach and todos so we capture for now and next semester.

This work is done from eng side and we have sent it to Mario who can now modify it and own it. I don't expect Eng Team to do any other work in this dashboard

maririos commented 3 months ago

@JonathanCrd what do you think?

MrJustinB commented 3 months ago

I wanted to sync on the approach and todos so we capture for now and next semester.

This work is done from eng side and we have sent it to Mario who can now modify it and own it. I don't expect Eng Team to do any other work in this dashboard

That was not the expectation. My expectation is that we would own the dashboard and enhance over time. Mario's recent enhancement request is to be able to show a visual trend i.e. how are the time going up/down over time.

maririos commented 3 months ago

Eng exp team can def own it. I am saying Eng team gave all the data needed and now for visualization, we can have PM team work on that. We can talk more about this tomorrow in our sync.

JonathanCrd commented 3 months ago

I would like to keep the reports separate and only include usage-related metrics in the current report. This will help to keep the data schema simple for the purpose of each report, especially considering the complexity of the relational database schema we have in Dataverse and all the information coming from ADO.

It is also the tool most frequently used by service partners for tracking their upcoming new product/service releases and also the tool we use to track SDK releases.

@ladonnaq are you talking about ADO or PowerBI? I wouldn't recommend using a PowerBI report for partner teams to trach their progress, since that is the goal of the Release Planner app.

Eng exp team can def own it. I am saying Eng team gave all the data needed

@maririos totally, we can create and update the data model in PowerBI as needed.

However, not all data is already calculated in the release planner tool. For example, in the TypeSpec report I created as part of this issue, I connected the Dataverse and ADO tables together to get stuff like service name, release plan info, if it is typespec or not, etc. But we didn't have the time each team spent on each task. I had to create a new column and write a formulate to calculate this. For some insights, new columns or quick calculations will need to be created from the data that we, the eng team, will provide to the dashboard.

mario-guerra commented 2 months ago

@maririos, @AngelosP, @MrJustinB, @ladonnaq would it be helpful to have a sync meeting to discuss the current state and clarify expectations one what the TypeSpec team wants to see? CC @allenjzhang for awareness.

ladonnaq commented 1 month ago

Hello @mario-guerra, I sent you an email but also adding a comment to this GitHub issue. Can you look at the data in the report and the example visual that Angelos created? We can continue to improve this dashboard but I want to make sure that you (and the TypeSpec) find the data valuable. Also, we should only be looking at the 2024 data. The MVP Release Planner and data model was not complete for non-APEX scenarios until beginning of Feb 2024.