search

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
547 stars 105 forks source link

PR build for Linux Java 11 with Spark 3.4 is not running #389

Closed kazuyukitanimura closed 1 week ago

kazuyukitanimura commented 3 weeks ago

Describe the bug

The github action PR build for Linux Java 11 with Spark 3.4 is not running

Steps to reproduce

Looks like the variable substitution is not working, i.e. PR Build / ${{ matrix.os }}/java ${{ matrix.java_version }}-spark-${{ matrix.spark-version }}/${{

Expected behavior

We should run tests for Java 11

Additional context

No response

kazuyukitanimura commented 3 weeks ago

Screenshot 2024-05-06 at 10 30 18 AM

advancedxy commented 1 week ago

Java 11 is excluded on purpose on pull request event as there are already too much combinations. Java8 and Java17 with Spark 3.4 on Linux are tested, which should be sufficient? They are all tested in a post-commit event though.

kazuyukitanimura commented 1 week ago

Thanks @advancedxy Do you know the original reason why this specific combination was excluded by any chance?

advancedxy commented 1 week ago

There's quota limit per repo for github runners(might not be a problem for apache project but for the forked ones) and we thought it would be sufficient to cover Java8(oldest) and Java17(quite the latest) with Spark 3.4 on Linux runners for pull requests. In that way, the runners required to run PR actions is reduced. The power consumption is reduced as well😁.

Is there any specific need to run Java11, such as it's the java version used in your production? If so, I think we can replace java 17/8 with java11.

kazuyukitanimura commented 1 week ago

Thanks. Looks like this is the history https://github.com/apache/datafusion-comet/pull/122 We can keep it as is for now. Closing