using multi-stage build

[sevenEng]

for https://github.com/me-box/databox/issues/83

[mor1]

Dumb question-- why the separate Dockerfile-dev and Dockerfile?

[sevenEng]

Dockerfile-dev is actually a copy-past of previous Dockerfile not using multi-stage build, the name really doesn't mean anything

[mor1]

Ah-- delete it if it's dead?

[Toshbrown]

CI builds failing

[ERROR] Bad checksum for /home/opam/.opam/packages.dev/ppx_tools.5.0/f70ca1d07d565989a0041e312e8efc00ff2ad87e:

• 1e18ffaa90d51cba9e6d5eb7bd186637 [expected result]
• 5e1a86ff323fe772d69ca2d25abc2c9d [actual result] This may be fixed by running opam update.

[sevenEng]

moved all the dependencies to another container seveneng/databox-ocaml-base, maybe worth transferring it under me-box? @jptmoore I think maybe this could also save you some hustle when trying to dockerize some components.

[mor1]

Not sure breaking this out is a good idea. AIUI it's essentially caching the pre-install of a set of packages needed to support a couple of arbitrary packages (export-service and store-timeseries). Is there genuinely a set of OPAM package dependencies that we expect all Databox components to have? Also AIUI beware of doing things like apk update in a separate RUN command from subsequent RUN commands. The way image layer caching works, when you rebuild you're likely not to rerun the apk update line. This is unlikely to be what you want I think.

[sevenEng]

True, I don't think we expect all Databox components to depend on such set of packages. For now, maybe only export-service and store-timeseries could benefit from this. What I want it to be, is a union of all the dependencies that components developed in OCaml are in need of. To serve as a caching of pre-install deps, this largely cuts down the build time, both locally and in the docker hub too, in the case of export-service, it's the time of downloading and compiling ~80 deps and for store-timeseries, it's ~120 deps.

If moving this under me-box/ is not a good idea, I think I could keep this under my own namespace then. I find it smooth the image building process quite a lot. (that or my machine is just too old and slow :sweat: )

And thanks for the reminder of apk update line, I'll update it.

[mor1]

I wonder if it would be worth suggesting to the OCaml folks that they might produce a auto-built Docker image that contains as many packages installed as possible? Could be a large image though, and updating too frequently.

I guess the concern I have about creating such a thing is the case where dependencies are updated but the consumer doesn't notice they're not picking up the updates. I think I'm being overly concerned though. It does save a lot of time. Go ahead and move it under the me-box org. We'll see how it goes.

On 5 September 2017 at 22:10, Qi LI notifications@github.com wrote:

True, I don't think we expect all Databox components to depend on such set of packages. For now, maybe only export-service and store-timeseries could benefit from this. What I want it to be, is a union of all the dependencies that components developed in OCaml are in need of. To serve as a caching of pre-install deps, this largely cuts down the build time, both locally and in the docker hub too, in the case of export-service, it's the time of downloading and compiling ~80 deps and for store-timeseries, it's ~120 deps.

If moving this under me-box/ is not a good idea, I think I could keep this under my own namespace then. I find it smooth the image building process quite a lot. (that or my machine is just too old and slow 😓 )

And thanks for the remind of apk update line, I'll update it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/me-box/core-export-service/pull/27#issuecomment-327304546, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKP4P7tFmOKARnMB24wgYwaFigpUni5ks5sfbjOgaJpZM4PD9Wc .

-- Richard Mortier mort@cantab.net