[Feature Request]: Docker images with both runtime and launch environments

[amardeep]

What would you like to happen?

Container environments describe a way to create custom docker images for runtime configuration, but the launch step example still needs access to the source file.

Could you please document a setup where the docker image contains both runtime and launch configuration so that a docker run invocation is enough to launch the job with the dependencies too.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• [X] Component: Python SDK
• [ ] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam YAML
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [X] Component: Website
• [ ] Component: Infrastructure
• [ ] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [X] Component: Google Cloud Dataflow Runner

[liferoad]

Have you checked https://cloud.google.com/dataflow/docs/guides/using-custom-containers? or https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/dataflow/flex-templates/pipeline_with_dependencies

[amardeep]

I have. It works, but is not the most convenient as along side a docker image, it also requires maintaining a file in google cloud storage. So, it is not as simple as building a docker image and running it. It would have been nicer if beam supported a better way natively.