Closed confused-Techie closed 1 year ago
In my ideal design in my head, the download microservice would speak both Cirrus CI and GitHub Releases. It could branch its logic based on "ARM Linux/Apple Silicon macOS? --> get from Cirrus", "other stuff? --> get from GitHub Releases rolling".
So that the download microservice still works for resolving Apple Silicon and ARM Linux payload URLs. Otherwise the service is dropping our ARM builds.
(EDIT: And I surely don't want to try and yeet the Cirrus Rolling binaries onto the GitHub Releases Rolling, that's too much shenanigans for me. Although arguably on the same level and degree of shenanigans as having the download microservice speak to two binary hosts with their own JSON APIs, but eh...)
As for the PR as-is, I haven't taken a close look, but it's nice if this is working already for GitHub Releases.
@DeeDeeG You do have a really good point there.
Although what if we take a slightly different approach and instead add scripts to the end of builds on cirrus to upload those to the github releases? Since you can see in the PR the GitHub API is much simpler to work with, and would be nice to have a proper visual archive of all our rolling releases.
Although I do get if that's difficult to do and we have to add back in the capabilities to work with Cirrus. Since that wasn't something fully considered during the drafting of this PR. Sure this PR will still look for ARM and Silicon binaries, but I had forgotten that we had no plans to make them available here
:shrug: It's a little bit down to what we can dream up, design and implement in the next few days. Not trivial, theoretically doable. I'm sure we can add back more stuff if some of it gets left on the cutting room floor in order to get something out the door in time. Something is better than nothing, to be sure.
Edit: Not to downplay efforts, I think we're doing pretty darn well in response to this situation, as always being volunteers in our spare time. Including this PR seems to have taken some real work to it already. Kudos to all that's working already in this and thanks for this and all the other efforts!! We're gettin' through this.
Appreciate you! But very true, we are limited by time.
Although over on the GitHub Actions PR on the main repo, I put all the logic of uploading artifacts into it's own NodeJS script, not relying on much else, so it may be totally possible to get Cirrus to run that same script after the fact, I'll actually take a try at implementing that soon, since if so it'd be rad to get this all functioning without to many more logic changes lol
Posting my test results, as always for this microservice's PR's:
My old test-find-link.js
from before (see code snippet below) totally works with the new download microservice code. So, this seems to be a functional drop-in replacement for the old code. (Just using GitHub Releases instead of Cirrus's API.)
// This is test-find-link.js
const utils = require("./utils.js");
(async () => {
console.log(await utils.findLink("arm_linux", "linux_deb"));
})();
node ./test-find-link
to test all possible binary assets (click to expand):Note that there are some ok: false
results where I gave incorrect parameter combinations, or where there was a genuine and legitimate reason to return a 404, which worked as intended and as written, even without doing anything with the stuff I mentioned in my review comments.
In summary: I didn't find any real-world failures of the microservice logic with this testing. Handling bad input parameters and handling missing assets were also good and both working as intended in these tests.
@DeeDeeG Addressed everything you commented on, if you'd like to do one more review so we can push this. Apparently our changes to cirrus have in fact already broken the download microservice
Thanks a ton @DeeDeeG, appreciate you and all your work on this PR as well.
Gonna go ahead and merge, then push to production!
This PR removes our
download
microservices interaction with CirrusCI, instead now collecting rolling releases frompulsar-edit/pulsar-rolling-releases
repository.These changes relate to the necessary changes needed to be made, detailed in
pulsar-edit/pulsar#685
.And should resolve any and all issues related to the download microservice, to ensure uninterrupted functionality of our download microservice.