Update to Gradle 8; resuscitate CI

[MrDOS]

This resolves https://github.com/NeuronRobotics/nrjavaserial/issues/226, resolves https://github.com/NeuronRobotics/nrjavaserial/pull/227, and resolves https://github.com/NeuronRobotics/nrjavaserial/issues/232.

These changes specifically avoid making any meaningful changes to built artifacts. That is, this whole PR is just fixing the build process, and not changing what gets built. @madhephaestus and I talked waaay back in early 2021 about automating the release process; this isn't that. I intend to open a follow-up PR to address fully automated releases after tackling the outstanding stability issues raised in https://github.com/NeuronRobotics/nrjavaserial/issues/197, https://github.com/NeuronRobotics/nrjavaserial/issues/201, and https://github.com/NeuronRobotics/nrjavaserial/pull/211.

I've confirmed that the newly-built native libraries pass a simple smoke test (enumerate ports) on Windows 10 on x86_64 and Linux 6.1.0/glibc 2.36 on ARM64. I'll test x86_64 macOS later today. They also work on Mac OS X 10.9 on x86_64. And FreeBSD 13.2 on ARM64.

I don't intend to commit the natives built by CI into this branch as they haven't functionally changed.

[madhephaestus]

I have created a Repo that can publish a compiled release directly to Sonatype. If you would like to try to incorporate a release system too i can all the CI secrets to make it work. (or perhaps archive this project to move it to the active CommonWealthRobotics project that I keep up to date with release secrets).

The main reason to have a proper release in CI would be to have artifacts for all platforms generated from the tagged source, guaranteed, instead of having to commit and collect the artifacts into version control.

Thoughts?

https://github.com/CommonWealthRobotics/mujoco-java/blob/main/.github/workflows/release.yml

[MrDOS]

Thanks for taking a look at this so quickly!

I have created a Repo that can publish a compiled release directly to Sonatype.

Ah, cool, you're using the Gradle Nexus Staging plugin. I wanted to try its apparent successor, the Gradle Nexus Publish Plugin, which looks like basically the same thing, just actively maintained.

the CI secrets

When last we left off, you had provided OSSRH credentials as action secrets (OSSRHUSERNAME and OSSRHPASSWORD). I was crossing my fingers that those were still valid.

I obviously don't have the private key that you used to sign previous releases, but I don't think Sonatype/OSSRH care about that as much as they care about the release being signed by somebody. To that end, I generated a new signing keypair for developers@nrjs.org, publicized it, and stored the ASCII-armoured key and its password in another pair of secrets (SIGNINGKEY and SIGNINGPASSWORD). Then in the Gradle build script, useInMemoryPgpKeys should pass those through to the signing plugin, were we ever to invoke the sign task or any of the publication tasks:

https://github.com/NeuronRobotics/nrjavaserial/blob/ced315c31c9762e0d3844479c0e44c65eb451dce/build.gradle#L241-L242

Again, haven't tried this yet, but… 🤞 Should be straightforward to invoke the publication and close/release tasks when the build workflow is triggered by a new version tag.

perhaps archive this project to move it to the active CommonWealthRobotics project

🤷‍♂️ I'm fairly ambivalent. My biggest concern would be making sure issue and PR history gets moved, and URLs get redirected appropriately. I think GitHub takes care of all that when repository ownership is transferred from one organization to another.

instead of having to commit and collect the artifacts into version control

Agreed, drives me up the wall. Would desperately love to stop dragging binaries around in Git. I've thought about writing a Gradle task to go off and fetch native libraries from the latest release or the latest CI build; that way, users still don't necessarily need a C compiler to work on the Java portion of the project.

[madhephaestus]

Given the support of CI building of binaries, can we do a test release out of the PR branch?

I am happy to approve this if release should be a separate PR, but it would be nice to do have the whole build update.

[MrDOS]

instead of having to commit and collect the artifacts into version control

I went down a rabbit hole for the last few days looking into writing a custom Gradle task to retrieve the native libraries from CI builds. Seems very doable! Just a bit big to include in this PR, so I'll submit that separately. The fly in the ointment is the FreeBSD arm64 library, which wasn't previously supported by CI. (I guess the rule going forward will be: no new platform support without extending the CI build. Which seems fair enough.) I've undertaken a textbook-definition yak shave and have added arm64 support to my FreeBSD cross-compilation container. Consuming that new container isn't a big change, and it's relevant to the other work being done here, so I've added that to this PR (7312c5c6cbc6ac5244ee982af70cf634484d045c).

if release should be a separate PR

I'm kind of leaning in that direction; this PR has gotten quite large. But I also want to see the release automation work done too, and I also want to have confidence that it doesn't require doubling back on these changes at all. So I'm going to do that work in a separate PR, but I'll base it on this PR, and I won't merge this one until we've proven that releasing can work.

[madhephaestus]

Awesome! I totally support this strategy. I will look into setting up all the secrets here so we can push out the release, and look to the future, once ci is stable, to move to the new project. With the secrets in place, we can test releases in the pr branch before merge.

On Wed, Aug 30, 2023, 9:32 PM Samuel Coleman @.***> wrote:

instead of having to commit and collect the artifacts into version control

I went down a rabbit hole for the last few days looking into writing a custom Gradle task to retrieve the native libraries from CI builds. Seems very doable! Just a bit big to include in this PR, so I'll submit that separately. The fly in the ointment is the FreeBSD arm64 library, which wasn't previously supported by CI. (I guess the rule going forward will be: no new platform support without extending the CI build. Which seems fair enough.) I've undertaken a textbook-definition yak shave and have added arm64 support to my FreeBSD cross-compilation container https://github.com/MrDOS/freebsd-cross-build. Consuming that new container isn't a big change, and it's relevant to the other work being done here, so I've added that to this PR (38aa085 https://github.com/NeuronRobotics/nrjavaserial/commit/38aa085ac6edec17b18fc778a3589aed7da21137 ).

if release should be a separate PR

I'm kind of leaning in that direction; this PR has gotten quite large. But I also want to see the release automation work done too, and I also want to have confidence that it doesn't require doubling back on these changes at all. So I'm going to do that work in a separate PR, but I'll base it on this PR, and I won't merge this one until we've proven that releasing can work.

— Reply to this email directly, view it on GitHub https://github.com/NeuronRobotics/nrjavaserial/pull/238#issuecomment-1700198796, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJSKRUPZTN6LMHOEEHQDBTXX7SUXANCNFSM6AAAAAA36OYTUI . You are receiving this because you were mentioned.Message ID: @.***>

[madhephaestus]

Hey all! I did some work on publishing and was able to add all the keys needed to publish directly from the CI build. The keys i added are for the whole project, so if yall wanted to incorporate the changes in the linked PR, we could begin publishing from CI FINALLY!

https://github.com/NeuronRobotics/JCSG/pull/49

[MrDOS]

Amazing! I'll try to take a look soon.