Add `copy` to the yaml configuration

[riquito]

I want to deploy different services using the same image but with a different configuration file. Currently to achieve that I can

• build as many images that inherit from a common image plus a different COPY in each
• mount a single file volume

The first solution isn't feasible and the second it's overkill. I just want to copy a file, not keep it in sync

Proposed configuration:

myservice:
    image: foo/bar
    copy:
        - src:dest

[spetrashov]

+1

[ekapratama93]

+1

[mrkvost]

+1

[zzart]

+1

[mracos]

+1

[AlecJY]

+1

[petr-stupka]

+1

[meigallodixital]

+1

[sajjaadalipour]

+1

[RW83]

+1

[conker84]

+1

[Nevario]

+1

[marenaso]

+1

[ryshi]

+1 for reopening

[rhk4360]

+1

[jonsecchis]

+1 for reopening

Wouldn't a feature that operates on a volume after the container (or itself) reachs certain state work? It could be a flag like readonly, but copyonly instead! Then you would have the regular volumes operations pipeline/syntax with the added functionality of removing the link after a state of sync is reached inside the container...

[scottlindner]

+1 for reopening. Volume mounting is not always an option. Which is why I'm posting here today. I need to modify a large number of containers by hand to copy files in due to a unique scenario where volume mounting is forbidden.

[atheinzen]

+!

[virus-warnning]

+1

[ghost]

+1 for docker-compose copy feature

[afdevries]

+1

[ukbe]

+1

[stuaxo]

I'm starting to think this is pointless. Docker only add features around the particular use-case they built docker for - this is one of the features outside their use-case so they will probably never add it.

[delcon]

+1

[shin-]

@stuaxo Just because a lot of people are asking for footguns, doesn't mean we should automatically implement them.

In this case, files that are required by a service to run should either be baked into the build (declared in Dockerfile), made available through volumes, or (in the case of docker stack / Swarm mode) exist as config or secret objects. Copying files into potentially multiple destinations at runtime is slow, error-prone, and not failure-resilient.

[stuaxo]

Cheers, apologies if that came across as grumpy.
When you say "running a service" I guess you mean a long running thing like a web service, Docker has this covered well.

Dockers layering and caching are handy for other things ;

I put my old WordPress blog into docker, but only run-it sometimes for a few minutes so I can export data or check what a post looked like there.

This took a bit of work to use docker compose to separate out MySQL and Apache, but I wouldn't care if they where all smooged together.

Dockers caching is fun when experimenting with building desktop libraries like Cairo. Most of the things I was trying to do I've forgotten the details of, but are more on the building-something side of things rather than anything to do with running services.

[kingsman142]

+1 docker-compose copy needs to be added. It allows for much easier flow of configuration files and updates to containers.

[prologic]

Use docker cp and docker config

On Fri, Mar 2, 2018 at 21:19 James Hahn notifications@github.com wrote:

+1 docker-compose copy needs to be added. It allows for much easier flow of configuration files and updates to containers.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/docker/compose/issues/1664#issuecomment-370055493, or mute the thread https://github.com/notifications/unsubscribe-auth/ABOv-q5VO9n9HDvBP6YJ1BCePz5tGu-pks5tabdrgaJpZM4FTTPg .

--

James Mills / prologic

E: prologic@shortcircuit.net.au W: prologic.shortcircuit.net.au

[chinmaya-n]

+1 for docker-compose copy. The idea is to have copyonly for files as described here

[38leinaD]

+1

[smaley07]

+1

[WebMatrixware]

In this case, files that are required by a service to run should either be baked into the build (declared in Dockerfile), made available through volumes, or (in the case of docker stack / Swarm mode) exist as config or secret objects. Copying files into potentially multiple destinations at runtime is slow, error-prone, and not failure-resilient.

Except in many cases all you want is to update a single config file, in which case baking your own image of a standard official image is bad practice and builds in lots of technical debt for down the road, volumes are overkill X10 and you do not want or need the files to keep syncing not to mention the problems of having to deal with the whole folder and not just a single file in a default location, and you need to update a system file not just get a config setting.

I understand your concerns, but just telling everyone their need is not real will only drive people away from your project and the community that wants to use it and support your work. We want to work with you here to find a solution that works, but all you want to say is that we don't need what we think we need.

[darkguy2008]

Wow I can't believe this hasn't been done yet. I thought that this basic functionality would've been in docker-compose by now. Oh, the beauty of OSS projects...

Solved it with docker-compose --force-recreate --rebuild, it copies the new files if they've changed (or so it seems). Not ideal but it works.

[zeeke]

+1

[elemir]

+1

[avgoeid]

+1

[umedjan]

+1

[ascourtas]

+1 this is low key driving me nuts, it seems silly to have to mount a volume just for one config file.

[Payden-Pringle]

+1

We have a way to add files to a container using docker compose with volumes. I've noticed that the Compose documentation does have a configs setting now, so we can add single files as well.

https://docs.docker.com/compose/compose-file/#configs

However, what I believe people want is a way to merge files from the host into a directory in the container with preference being overwrite if exists, the way you'd expect a copy to work.

It's strange to me that the volumes configuration in the compose file has a nocopy option:

https://docs.docker.com/compose/compose-file/#volumes

volumes:
  - type: volume
  source: mydata
  target: /data
  volume:
    nocopy: true

Which is described like this: volume: configure additional volume options nocopy: flag to disable copying of data from a container when a volume is created

So it disables exactly what is desired?

My desired use case is web applications for development. I want the image to contain everything it needs to serve a website including a default website to serve, but have an option to merge/overwrite those files with the current working state of a website without other configuration (the key) beyond having the files in the right place on the host. Files that, when copied to the container, are ephemeral. Meaning not in a network share.

An example of a framework that would benefit from that is something like Laravel. Where I make an image that will be capable of serving the Laravel splash page with no other configuration (volumes, network shares, etc), but the compose file is written to copy files from the host to the container's Laravel root so that they can be overwritten.

Using a network share for this is extra upkeep for an ephemeral environment (development isn't always constant). Using a volume mount for it, with a framework like Laravel, requires building the app in that volume which means installing PHP and composer on the host or sharing the location on the host the app would be created at. Both creating more maintenance.

Edit:

After testing, it seems that when you mount a volume, the files that existed in the image at that location are copied over to the persistent volume with preference to the volume's original contents. Meaning it doesn't overwrite.

I actually think this is a solution for the desired outcome.

[su-narthur]

Hey, guys, just FYI, it's better to give a thumb-up reaction to the top message on this issue than add ing a "+1" comment, as issues can be sorted by how many reactions they've received.

[sentinel09]

+1

[flavienbwk]

+1 Please Docker team, consider your community.

[masaeedu]

@shin- The problem right now is that the volumes feature makes it impossible to plug in a single config file from an external volume into a container.

Suppose I want to plug an nginx config file into /etc/something/whatever.conf without annihilating everything else that's in that folder. You can't create a named volume for a single file (see moby/moby#30310), and you also can't create a directory volume containing a single file, then mount that particular file into the container (see moby/moby#32582).

The only option you're left with is to pollute fixed locations in your actual Docker host with the config files and then bind mount them in. This of course goes to pieces if you want to move around between Docker hosts, or work with a Docker host you don't have filesystem access to (e.g. online CI), or even just run multiple stacks in parallel.

Something has to give way here, either the volumes feature in Docker needs to change so that we can actually use it to accomplish injecting config files into our stack, or we need to work around it in compose using something like a copy config option.

[shin-]

FWIW I'm no longer a Compose maintainer, but it seems to me that that use case would be best served by injecting the configuration file at build time using a Dockerfile.

[masaeedu]

@shin- Unfortunately the configuration files are not known at build time, the images are prepared on a CI server long before we know where we're going to be deploying them and in what configuration.

[WebMatrixware]

I'm with @masaeedu, I understand the conceptual ideal of keeping the function of compose from creeping and therefore not wanting to add this feature, but in real life the loss of this feature GREATLY reduces the circumstances where compose can be used without

A) having to cobble together some kind of of hideous script work around to stand up the containers in question (the very thing compose was meant to help standardize and keep us from having to do),

B) having to add some other tool that handles all of our config (which while workable adds a huge amount of overhead for small to mid-sized projects that really just need to copy a specific .conf and .env to each container, something you do not want to do at image build time, that is equivalent to saying that rather than your software having a config file you should hard code all of your options into the C++, it leads to dozens of extra images to have to maintain), or

C) having to maintain a separate image for every possible config of a given piece of software that we will ever need / build a custom image every time we have to make a minor config change....which is both unworkable in the real world of production software, and also makes it impossible to have the kind of robust QA and testing pipeline that we need for production software.