Closed matroscoe closed 1 year ago
Hey @matroscoe, thanks for raising the issue with such information!
It will be simpler to continue on Discord though, we just created a dedicated private channel and invited you to it. We are currently looking at the data to understand what happened.
See you on Discord
Sorry if this is not the correct repository to report this on but I figured a GitHub issue is easier to follow than a discord thread.
When I rebase I am not seeing the expected behavior where the
feature branch
has its benchmark it is being compared to reset to the latest value of in our casedevelopment
e.g. in my development branch this is the latest value that was merged in as seen from within codspeed:![image](https://github.com/CodSpeedHQ/pytest-codspeed/assets/102177474/b986693d-0935-49f6-ac1a-87b98e206122)
when I go to:![image](https://github.com/CodSpeedHQ/pytest-codspeed/assets/102177474/a3f53739-a057-4ace-b65f-a95a6de36963)
https://codspeed.io/{org}/{github-repo}/benchmarks
I see this:In my latest feature branch I see:![image](https://github.com/CodSpeedHQ/pytest-codspeed/assets/102177474/ca3ab225-ad70-49f2-930b-c6b1b3224c24)
my concern is that there is something different going on either in the benchmark comparison or in how the new "default" is run because all my feature branches show roughly the same performance drop which ties to the "latest benchmark" from
https://codspeed.io/{org}/{github-repo}/benchmarks
but I wouldn't expect to see so much variability from myfeature
branch to mydevelopment
branch.