wipfli commented 1 year ago

Similar to #584, we also want an Android release...

wipfli commented 1 year ago

The last one was a bit more than 1 year ago: https://github.com/maplibre/maplibre-gl-native/releases/tag/android-v9.5.2

wipfli commented 1 year ago

The android release workflow seems to only have a manual trigger:

https://github.com/maplibre/maplibre-gl-native/blob/a405d7a9b33563ae6d3aabd55f075976b0219b24/.github/workflows/android-release.yml#L3-L4

wipfli commented 1 year ago

I am tempted to just run this one and see where it crashes...

wipfli commented 1 year ago

Hm I did not find any documentation on how to run the release. So let's just try...

wipfli commented 1 year ago

https://github.com/maplibre/maplibre-gl-native/actions/runs/3660524523/jobs/6187766501

wipfli commented 1 year ago

It seems to be waiting for a self-hosted ubuntu runner.

Requested labels: self-hosted, ubuntu-20.04
Job defined at: maplibre/maplibre-gl-native/.github/workflows/android-release.yml@refs/heads/main
Waiting for a runner to pick up this job...

Do we even still have self-hosted ubuntu runners?

wipfli commented 1 year ago

587 makes the android-release.yml workflow run just on the normal github runners.

wipfli commented 1 year ago

Next attempt: https://github.com/maplibre/maplibre-gl-native/actions/runs/3660654651/jobs/6188035491

wipfli commented 1 year ago

I think this will crash at the firebase test step.

wipfli commented 1 year ago

How were the release notes generated in the past? Like here: https://github.com/maplibre/maplibre-gl-native/releases/tag/android-v9.5.2, is this just taken from the platform/android/changelog.md file if that one exists?

wipfli commented 1 year ago

Hm that file did not get updated it seems... https://github.com/maplibre/maplibre-gl-native/blob/2fcd4f8dea87a43efd561128396dfec08f463a70/platform/android/CHANGELOG.md?plain=1#L1-L12

wipfli commented 1 year ago

git log --oneline -- src include platform/android gives this:

2b26632ee Add missing header guards (#543)
d129432ce Bump semver from 7.3.7 to 7.3.8 in /platform/android (#530)
53207c456 Removing unused versions sdk (#515)
58543a791 Fixes potential NaN when calling `NativeMapView::nativeMoveBy` (#501)
b567d466e Upgrade Gradle from 3.6.3 to 3.6.4 (#456)
39574ed23 (tag: node-v5.0.1-pre.0) Upgrade libs and remove Jetifier (#218)
1310ce298 Fix android ci workflows (#476)
d209c4f5b Bump to JDK 11 in android CI and generate Gradle Wrapper (#474)
485422634 Bump ejs from 3.1.7 to 3.1.8 in /platform/android (#470)
e147b11fd Bump semver from 5.7.1 to 7.3.7 in /platform/android (#461)
f4f7232b0 Bump pixelmatch from 4.0.2 to 5.3.0 in /platform/android (#460)
793a5193f Bump esm from 3.1.0 to 3.2.25 in /platform/android (#463)
3e98f60ef Migrate examples in android TestApp to Kotlin (#416)
c72c468ff Fix typo in geo.cpp (#412)
1341f33ba Add ClientOptions to be able to pass around client name and version (#365)
acdce04a2 Use maplibre docs for more links (#354)
6bbeb9fd9 Use the MapLibre style spec docs website (#353)
52b5f02ed Fix render tests (#351)
5f679b55b Refresh iOS & Android build docs
9cd70bfe7 Qt build improvements and documentation (#277)
a1f3dd263 Bump ejs from 2.7.4 to 3.1.7 in /platform/android (#299)
eca80dbc0 Replace Mapbox with MapLibre in README titles (#297)
7e133e3bd fix shiftY calculation typo (#285)
32ed70c95 [msvc] Fix warnings, mainly casting to smaller types (#270)
539d81a3b Rename MaptilerFileSource to MBTilesFileSource (#198)
87bb49250 Fix street label appearance while animating near zoom level threshhold (#267)
d9a22ba2c Changed missed MapBox reference to MapLibre (#253)
6db27f50a Implement map projection functionality (#254)
ea234edf6 chore: rename master -> main in CI and scripts (#246)
cfcbc62fa Feature - Bring back node support (#217)
b7a42e997 Remove obsolete CI configurations (#219)
b8b2b80a1 Bump lodash from 4.17.19 to 4.17.21 in /platform/android (#195)
381e43fe3 (tag: ios-v5.12.2, tag: android-v9.5.2) Updating changelogs for release ios-v5.12.2 and android-v9.5.2

wipfli commented 1 year ago

The build step in android-release.yml did not succeed:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':MapboxGLAndroidSDK:compileReleaseKotlin'.
> Could not resolve all artifacts for configuration ':MapboxGLAndroidSDK:releaseCompileClasspath'.
   > Could not download okhttp-3.12.3.jar (com.squareup.okhttp3:okhttp:3.12.3)
      > Could not get resource 'https://repo.maven.apache.org/maven2/com/squareup/okhttp3/okhttp/3.12.3/okhttp-3.12.3.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/squareup/okhttp3/okhttp/3.12.3/okhttp-3.12.3.jar'.

            > Read timed out
   > Could not download gson-2.8.6.jar (com.google.code.gson:gson:2.8.6)
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
      > Could not get resource 'https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.8.6/gson-2.8.6.jar'.
Use '--warning-mode all' to show the individual deprecation warnings.
         > Could not HEAD 'https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.8.6/gson-2.8.6.jar'.
See https://docs.gradle.org/6.4.1/userguide/command_line_interface.html#sec:command_line_warnings
            > Read timed out

wipfli commented 1 year ago

Hm I wonder why the release build fail. Currently, we have at lease three workflows that potentially build android:

android-ci.yml (last successful run 2 years ago)
android-ci-pull.yml (last successful run today)
android-release.yml (last successful run 1 year ago)

wipfli commented 1 year ago

The main build steps in android-ci-pull.yml are:

https://github.com/maplibre/maplibre-gl-native/blob/2fcd4f8dea87a43efd561128396dfec08f463a70/.github/workflows/android-ci-pull.yml#L114-L124

Which does not the package build step we have in android-release.yml:

https://github.com/maplibre/maplibre-gl-native/blob/2fcd4f8dea87a43efd561128396dfec08f463a70/.github/workflows/android-release.yml#L100-L101

which currently fails

wipfli commented 1 year ago

Maybe make apackage is something we do in android-ci.yml?

thehoneymad commented 1 year ago

@wipfli is up doing stuff I see. :D

wipfli commented 1 year ago

make apackage works in docker on my ubuntu laptop with the following steps:

docker run --rm -it ghcr.io/maplibre/android-ndk-r21b
git clone https://github.com/maplibre/maplibre-gl-native.git --recursive --depth 1
cd maplibre-gl-native/platform/android/
make apackage

So maybe the timeout above was just a network problem with the GitHub runner...

wipfli commented 1 year ago

Re-running the release action: https://github.com/maplibre/maplibre-gl-native/actions/runs/3660654651/jobs/6192548596

wipfli commented 1 year ago

This time the make apackage step worked. The release failed now at "generate javadoc" with the following error message:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':MapboxGLAndroidSDK:javadocrelease'.
> Javadoc generation failed. Generated Javadoc options file (useful for troubleshooting): '/__w/maplibre-gl-native/maplibre-gl-native/platform/android/MapboxGLAndroidSDK/build/tmp/javadocrelease/javadoc.options'

wipfli commented 1 year ago

Locally, also the make android-javadoc step works for me with:

docker run --rm -it ghcr.io/maplibre/android-ndk-r21b
git clone https://github.com/maplibre/maplibre-gl-native.git --recursive --depth 1
cd maplibre-gl-native/platform/android/
make android-javadoc

wipfli commented 1 year ago

Should I just rerun the release and wait for 1 hour until it reaches the javadoc step to try again?

wipfli commented 1 year ago

Rerunning... https://github.com/maplibre/maplibre-gl-native/actions/runs/3660654651/jobs/6192919945

wipfli commented 1 year ago

The good thing is that at least the library and documentation builds locally. So now we only need to get it to build in the runners. And then we need to push out the release artifacts to GitHub Releases and Maven.

Regarding Maven, the android-release.yml workflow uses a bunch of interesting secrets:

GPG_KEY_CONTENTS
OSSRH_USERNAME
OSSRH_PASSWORD
SIGNING_KEY_ID
SIGNING_PASSWORD
SONATYPE_STAGING_PROFILE_ID

6 things to just publish to one place, this is horrible. Now I understand why NPM gives you an auth token with which you can upload releases...

wipfli commented 1 year ago

https://github.com/maplibre/maplibre-gl-native/blob/522bf95f93ca660ab3eb615cbaa4a63ad6ccb31e/.github/workflows/android-release.yml#L180-L194

wipfli commented 1 year ago

This is related to the old idea of documenting what secrets we have: https://github.com/maplibre/maplibre-gl-native/issues/312

wipfli commented 1 year ago

Failed again in the make apackage step:

FAILURE: Build failed with an exception.

> Task :MapboxGLAndroidSDK:compileReleaseKotlin FAILED
* What went wrong:
Execution failed for task ':MapboxGLAndroidSDK:compileReleaseKotlin'.
> Could not resolve all artifacts for configuration ':MapboxGLAndroidSDK:releaseCompileClasspath'.
   > Could not download okhttp-3.12.3.jar (com.squareup.okhttp3:okhttp:3.12.3)
      > Could not get resource 'https://repo.maven.apache.org/maven2/com/squareup/okhttp3/okhttp/3.12.3/okhttp-3.12.3.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/squareup/okhttp3/okhttp/3.12.3/okhttp-3.12.3.jar'.
            > Read timed out

wipfli commented 1 year ago

Rerunning: https://github.com/maplibre/maplibre-gl-native/actions/runs/3660654651/jobs/6193312555

wipfli commented 1 year ago

Maven Read timed out again. It looks like we are not the only ones having this issue:

https://github.com/actions/runner-images/issues/1499

boldtrn commented 1 year ago

Github Actions do have a spotty network connection at times :( - we have similar issues for CI tests that require a connection, they randomly fail sometimes.

wipfli commented 1 year ago

And yet another run now hopefully without timeouts... https://github.com/maplibre/maplibre-gl-native/actions/runs/3664281356/jobs/6194598660

wipfli commented 1 year ago

Yeah this is disappointing. Some people in https://github.com/actions/runner-images/issues/1499 seem to just have switched to self-hosted runners...

wipfli commented 1 year ago

Timed out again...

boldtrn commented 1 year ago

A self hosted runner in a public repo seems to be discouraged, but maybe we have to go down that path and only run this on the main branch to avoid running malicious code?

wipfli commented 1 year ago

The container registry personal access token CR_PAT is actually not needed in the android-release.yml workflow. Because the container is public and the workflow only needs to read the container...

https://github.com/maplibre/maplibre-gl-native/blob/73973703426ebf11c3c6bea60857644c44d7b5e5/.github/workflows/android-release.yml#L12-L16

wipfli commented 1 year ago

I would really want to avoid self-hosted runners as far as we can, because:

Maintaining the runners is extra work which is often done manually. And everything we do not have in code will be lost from our collective knowledge on a one year timescale, as this issue around make a release nicely shows
Uses financial resources which I would rather want to allocate to compensate people
Makes it hard for contributors to have the identical setup in their forks for development

wipfli commented 1 year ago

I am now messing around with the workflow and timeout settings in my fork at https://github.com/wipfli/maplibre-gl-native/actions/runs/3665108379/jobs/6195967319

wipfli commented 1 year ago

If this ever gets to the firebase step I will let you know..

wipfli commented 1 year ago

It would be great if I could run the release-android.yml workflow locally. Does something like that exist?

wipfli commented 1 year ago

What I do at the moment is go though the individual steps and copy-paste them to my console...

wipfli commented 1 year ago

I guess that question is related to #322 and https://stackoverflow.com/a/59988803 gives a nice outline how one should approach CI environments.

wipfli commented 1 year ago

Many projects have used the fix with the maven options to get around the timeout. For example, Geoserver used the fix as well: https://github.com/geoserver/geoserver/commit/f461397c56d38e7455ca67a3e2be71506a2b6dbf

But now I am not even sure if we use maven to build... Because here it says gradlew:

https://github.com/maplibre/maplibre-gl-native/blob/82732b6e9d7770ef832be080c20414ce517b1486/platform/android/Makefile#L49

wipfli commented 1 year ago

What is the difference between graddlew and mvn anyway?

boldtrn commented 1 year ago

What is the difference between graddlew and mvn anyway?

Gradle is Androids default built system. We might have to try the Gradle settings instead, good point 👍 - essentially Gradle and Maven work very similar, you define your build, plugins, dependencies, etc. and use the tool to build.

wipfli commented 1 year ago

Thanks @boldtrn

wipfli commented 1 year ago

I keep reading that azure closes idle connections, and that gradle then crashed or so... https://github.com/gradle/gradle/pull/19196 seems to have implemented a 3 minutes timeout in gradle

wipfli commented 1 year ago

https://github.com/gradle/gradle/issues/14951

wipfli commented 1 year ago

Maybe we should just update our gradle version?

wipfli commented 1 year ago

Is this the place where the gradle version is defined? https://github.com/maplibre/maplibre-gl-native/blob/82732b6e9d7770ef832be080c20414ce517b1486/platform/android/build.gradle#L12

maplibre / maplibre-native

Make an Android release #585

587 makes the android-release.yml workflow run just on the normal github runners.