apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.89k stars 4.27k forks source link

[Distroless] Publish Python distroless variants #32914

Open damondouglas opened 1 month ago

damondouglas commented 1 month ago

This tasks focuses on publishing Python distroless variants. The following is a possible implementation.

The task :sdks:python:container:pushAll configured in sdks/python/container/build.gradle iterates through python versions, invoking :sdks:python:container:push<version>. Through various additional Gradle tasks, these ultimately converge to the :sdks:python:container:py<version>:docker tasks. Said :docker task uses a pushContainer flag to indicate whether to publish the resulting container image. A likely non-obtrusive way to publish distroless variants may be to refactor sdks/python/container/build.gradle from:

tasks.register("pushAll") {
  for(int ver=min_python_version; ver<=max_python_version; ++ver) {
      dependsOn ':sdks:python:container:push3' + ver
  }
}

to:


var min_distroless_version = <something>

tasks.register("pushAll") {
  for(int ver=min_python_version; ver<=max_python_version; ++ver) {
      dependsOn ':sdks:python:container:push3' + ver
  }

  // set the Gradle property for building distroless
  project.setProperty('docker-distroless', 'true')
  for(int ver=min_distroless_version; ver<=max_python_version; ++ver) {
      dependsOn ':sdks:python:container:push3' + ver
  }
}
damondouglas commented 5 days ago

Submitted https://issues.apache.org/jira/browse/INFRA-26298