Open thibaudcolas opened 3 years ago
Hi @thibaudcolas, thanks very much for your feedback!
We definitely would love to weaponize this capability to help open source maintainers. I think you're already thinking along the same lines like this:
Re: lint packages like prettier
, I created #4 to discuss. e.g. it's not necessarily a bad thing if a lint package release breaks things - if it was intended. I've been wanting to enable Renovate to "lint fix" such packages though so interestingly that would not show up as a failure if Renovate ran prettier --write
as a post update command. Overall the users would be better with auto-updated source files after a Prettier update although it would obscure our data a little if so.
A related challenge is identifying packages where a low confidence score doesn't mean "don't merge". e.g. there are many cases where if a release has broken a lot of people but not you, then you should still be cautious. But there are cases like prettier
where the advice may be "doesn't really matter if it breaks others as long as it doesn't break you".
Well that’s all very exciting, I’m looking forward to see what you’ll build from there!
Nice work on the merge confidence feature! This is very interesting and feels like there’s lots that could be done with this data like you show in your blog post.
From the perspective of a package maintainer – I think I would be very interested in having access to the merge confidence data for my packages’ updates. Is this something you’ve considered? For example, as a maintainer of draftjs_exporter, when I make a release I would be interested in knowing whether users’ test suites are passing with this new release, whether people have managed to merge the upgrade, etc. Projects like Prettier, or any and all linters generally, could also be projects where it’s very interesting to know how much breakage there is with each release, as the breakage is somewhat inevitable.
Currently there isn’t really a good way to do this at scale in the package management ecosystem. Some package repositories have quantitative info about adoption in the form of "download statistics" for packages, but that’s about it. Some projects publish alpha/beta/RC releases in the hopes of collecting feedback from users, but that’s all very ad-hoc and qualitative rather than quantitative. Merge confidence feels like it could automate this feedback loop.