Update job_discogs_effnet to pin klay-beam (without pytorch) and use modern v1.0 style docker container by default (when running via Dataflow). Ensure that the job has:
a default docker container (no need to specify --sdk_container_image)
suitable launch+dev environment in <job_dir>/environment/dev.yml
pinned and compatible version of klay_beam and apache_beam that match the default docker container
valid example launch invocations in their README
On non-trivial change for this job is the new docker build process. We make a custom docker image, but do it "FROM" a klay_beam:bla-blah image. This results in a MUCH simpler local Dockerfile
Update job_discogs_effnet to pin
klay-beam(without pytorch) and use modern v1.0 style docker container by default (when running via Dataflow). Ensure that the job has:--sdk_container_image)<job_dir>/environment/dev.ymlklay_beamandapache_beamthat match the default docker containerOn non-trivial change for this job is the new docker build process. We make a custom docker image, but do it "FROM" a
klay_beam:bla-blahimage. This results in a MUCH simpler localDockerfile