search

apache / incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://devlake.apache.org/
Apache License 2.0
2.61k stars 530 forks source link

[Feature][DORA?] Support trunk-based development in Lead Time calculation #5269

Closed Feggah closed 6 months ago

Feggah commented 1 year ago

Search before asking

Use case

In the Accelerate book (one of the main books about measuring performance and DORA metrics), the authors explain about some technical practices (Chapter 4) to increase performance.

One of the capabilities that impact on Continuous Delivery (thus, performance) is using trunk-based development as opposed to long-lived feature branches.

This means that, teams could try to use trunk-based development to increase their performance, and should be able to see the difference that it is making to their performance.

As DevLake is a project in the scope of DORA metrics, it is important to support trunk-based development when measuring velocity metrics (in particular, Lead Time).

Description

Reading the documentation DORA - Median Lead Time for Changes, I noticed that if a commit is not associated with any Pull Request, it won't be taken into consideration to calculate the Lead Time: image

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

yumengwang03 commented 1 year ago

Thanks for submitting this feature request! We will take this into discussion.

Feggah commented 1 year ago

@hezyin Could you point me where this feature should be implemented? Is it part of a plugin?

I'm willing to help implement this feature, I just need some general guidance because I'm not familiar with DevLake deeper technical details

klemen-df commented 1 year ago

Hi

we use Azure Devops (board, pipelines) and Azure Repos. We also use trunk based development.

I integrated devlake (latest beta version of 0.17) with our account and I get metrics for 'deployment frequency' and 'Change failure rate' but nothing for 'Median lead time for changes'.

I wanted to open a bug but then found this - so this is related to us using trunk based development?

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 1 year ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

klemen-df commented 1 year ago

@yumengwang03 - any news on this?

klesh commented 1 year ago

@KlemenDanfoss @Feggah Thanks for following up on the issue. I was not familiar with the trunk-based development model, so I did some research on it, seems like it doesn't have PRs to represent a Code Change in the process. So, If I understand it correctly, should we treat each Commit just like PR and calculate the Lead Time based on it?

Feggah commented 1 year ago

@klesh Yes, DevLake would be able to calculate Lead Time with commits made directly into the main branch

klemen-df commented 1 year ago

Does it make any difference if that commit is done through a PR?

klesh commented 1 year ago

@Feggah How do we define the Lead Time for Trunk-Based development? When are the start and end times?

Feggah commented 1 year ago

It is similar to how it works with PRs. It will be the difference between the first commit time and the deployment time. The first commit would be considered the first commit that hasn't been deployed yet.

image

It is the same as how it works today, but just removing the PR requirement from the equation.

In practice for trunk-based development, almost all commits triggers a deployment because the commits are going to be pushed directly into the main branch.

klesh commented 1 year ago

@Feggah Thanks for the explanation. Just to clarify, if a developer pushes multiple commits at once, it would be considered one Change, correct? and we should take the First Commit of the Change for Lead Time calculation, correct?

Feggah commented 1 year ago

we should take the First Commit of the Change for Lead Time calculation, correct?

Yes, that's correct.

if a developer pushes multiple commits at once, it would be considered one Change, correct?

What do you mean by Change? Is it a concept different from Commit?

klesh commented 1 year ago

@Feggah Oh, A change is a PR in my mind, and a PR could contains multiple commits to make a Change, I thought First Commit represent the first one of a change/PR.

Feggah commented 1 year ago

I see, so in the context of not using PRs a Change would be something similar to "git push", right?

So if someone pushes 5 commits at the same time, the First Commit that is going to be used as the start time.

klesh commented 1 year ago

@Feggah Yes, that is what I have in mind. Does it make sense to you?

Feggah commented 1 year ago

Yes! 🙂

klemen-df commented 1 year ago

@klesh is this (also) related to #5996 ?

klesh commented 1 year ago

@KlemenDanfoss No, not really. For #5996 , the problem is that issues are missing.

Startrekzky commented 1 year ago

Hi @Feggah @KlemenDanfoss , may I know your

I'm gathering more contexts to think of a proper solution for this issue. Thank you in advance.

Feggah commented 1 year ago

team size

I'm not currently in the same team that I was before, but the tech department of the company was ~2000 employees. My team specifically had 8 people.

and why you chose the trunk-based development model?

My team wasn't doing trunk-based development. We were offering DevLake as a solution in the company, and one of the teams that were using it requested support for trunk-based, so I can't help much in answering this question.

and what Git tool(s) do you use?

Git CLI and GitHub.

klemen-df commented 1 year ago

Hi @Feggah @KlemenDanfoss , may I know your

  • team size
  • and why you chose the trunk-based development model?
  • and what Git tool(s) do you use?

I'm gathering more contexts to think of a proper solution for this issue. Thank you in advance.

Hi,

klesh commented 1 year ago

Hi, @Feggah , I'm proposing a solution for your use case, would you like to see if it can solve your problem or if anything is missing? Thanks.

Feggah commented 1 year ago

@klesh Where can I see it?

klesh commented 1 year ago

@Feggah Sorry, https://github.com/apache/incubator-devlake/issues/6103

Feggah commented 1 year ago

I see, probably this would work indeed @klesh.

I will cc some people here who are probably more involved in the need for the feature and could add more context if needed

cc @dgmorales @cnatan @pimguilherme @renatocmaia

klesh commented 1 year ago

@Feggah that is very thoughtful, thanks

github-actions[bot] commented 11 months ago

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

klemen-df commented 11 months ago

@klesh , anything new here?

klesh commented 11 months ago

@KlemenDanfoss The #6103 is the most promising solution to the problem, you may share your thoughts, vote the issue there.

github-actions[bot] commented 9 months ago

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 6 months ago

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.