In GH mvn actions we validate the JVM bytecode builds against supported published Apache Spark builds
The cache key in https://github.com/NVIDIA/spark-rapids/actions/caches for the dependencies just uses the current date at the day granularity. This may miss the most recent spark-rapids-jni dependency worst case by ~24h that the current spark-rapids PR is trying to pick up for a new API
This will guarantee that we have the latest spark-rapids-jni in the active cache when the user re-runs the GH action after the nightly artifact is published.
Describe the bug
In GH mvn actions we validate the JVM bytecode builds against supported published Apache Spark builds
The cache key in https://github.com/NVIDIA/spark-rapids/actions/caches for the dependencies just uses the current date at the day granularity. This may miss the most recent spark-rapids-jni dependency worst case by ~24h that the current spark-rapids PR is trying to pick up for a new API
We could curl https://oss.sonatype.org/content/repositories/snapshots/com/nvidia/spark-rapids-jni/24.12.0-SNAPSHOT/ (maybe REST API) to determine the latest available timestamp and make it part of the cache key.
This will guarantee that we have the latest spark-rapids-jni in the active cache when the user re-runs the GH action after the nightly artifact is published.