Closed thaJeztah closed 2 years ago
The project name isn't important. I like the idea of a random project name. Even better – could we generate a hash of the path to fig.yml and use that?
I actually think the project name is pretty important. If you want to be able to do anything with the images in a CI pipeline, you need to know the project name to be able to retag them as something else.
Being able to override project name from an environment variables makes it easy to keep these unique, and predictable. I think no matter what happens, we need to support the override from environment variables.
I do like the idea of making the project name configurable from a file. I think a similar discussion happened in (#45). Having the project name in.fig/project-name
seems like it will lead to a lot of really small files. I think it would be easier to just put it into the fig.yml
itself (like it is suggested in #45, and one of the docker compose proposals).
What are the advantages of putting it in a separate directory and file?
The reason for putting it in a separate file has a number of reasons, I'll try to explains my thinking behind this;
--project-name=foobar
), which can be different from the automatic project-name..fig/project-name
holds the name of that instance of the project.Perhaps the file should be called instance-name
or similar.
I do like the idea of making the project name configurable from a file
Although possible to do this with this proposal (by manually creating the project-name
file), my primary use case was to have fig create the file in order to save the name that was used.
will lead to a lot of really small files.
True, the alternative is to skip the .fig
directory, however, if additional things are needed in future, extra file(s) can be added without cluttering the root of your project. Git does it similar and I think it's a clean approach.
I think it would be easier to just put it into the fig.yml
I think both options can be complementary; use the name from fig.yml
as the default if the name is not overridden by a flag or env-variable. Put the name that is actually used to start the project inside the .fig/project-name
Being able to override project name from an environment variables makes it easy to keep these unique, and predictable. I think no matter what happens, we need to support the override from environment variables.
Curious; how do you handle managing multiple projects? I.e. cd project-a && fig ps
then cd project-b fig <something>
?
Thanks for your feedback!
Curious; how do you handle managing multiple projects?
I usually run fig
from python-tox
, which lets me set the environment, so I'm never really setting it in my shell. I also tend to use tmux, so I have a different shells for every project.
I think both options can be complementary
Cool, I agree.
I don't know if I'm sold on .fig/instance-name
vs say .fig-instance.yml
that could contain any instance specific overrides, but I think this is more of a minor point.
Thinking of the project-name; If I understand your situation correctly, it's important to be able to have control over the images being produced, e.g. myapp:build12345
, or also names of the containers? Just to get a "feel" for your situation.
My thoughts behind "random" project-names was that, as long as Fig is able to locate the containers for a project, nice-looking names aren't important. If I, as a user, can access a services' container via its service-name (e.g. fig stop web
), the name of the actual container doesn't matter.
Even have been thinking that fig could keep track of containers by ID in a local storage inside the .fig
directory (which could be a performance gain on machines with a lot of running containers). But that's really something for a separate issue.
Any thoughts on "implicitly" creating the 'project-name' file or "explicitly" via fig init
? Perhaps if I have some time during the coming holidays I can experiment a bit (never coded anything in Python, so it will be really experimental :))
it's important to be able to have control over the images being produced, e.g. myapp:build12345, or also names of the containers?
I guess image is the really important one, but being able to ensure that container names don't conflict is also really important on shared hosts. I see this proposal actually aims to deal with that issue as well.
But I do think that predictable names for containers are still important. I can think of a couple tasks where it's important for external systems (not just fig) to be able to identify a container by name:
Both of these are pretty easy right now because I already know what the container name will be. If I have to lookup a name it's somewhat more involved.
Even have been thinking that fig could keep track of containers by ID in a local storage inside the .fig directory
It is true this could be a performance gain, but I worry this is the wrong approach. Adding any persistent state to fig introduces the possibility of lots of bugs (that come along with state). I would much rather see this being handled by a labeling system in dockerd. If fig is able to just label containers and then query for all containers with that label , it keeps all the state in a single place (dockerd). I know there was talk of this at some point, I'm not sure if it's on the roadmap.
Any thoughts on "implicitly" creating the 'project-name' file or "explicitly" via fig init?
I guess implicitly would be more backwards compatible. I would like to avoid a fig init
unless absolutely necessary. Instead of using the basedir, I could see it implicitly creating a project name of <basename>-<4 digit random id>
i can only say if fig starts generating random names now i will switch away from it because i run 1000+ containers on a single host and when i cant identify them by name any more its nothing for me.
@frank-dspeed thanks for adding your use-case. If I understand correctly, you start your containers with fig, but "manage" them directly via Docker after that, making use of the naming convention that Fig uses?
Perhaps this is of interest to you as well: https://github.com/docker/docker/pull/9882 - having meta-data would offer more ways to identify containers, not only their name.
ah yes i did such proposals a lot for example simply adding new filds and that but i ended in that case with simply adding a ENV and then i have that as costum fild i can parse it :dart:
I think managing the unambiguity of the project name shouldn't be fig's responsibility.
when i understood your proposal correctly, the generation of a random name would render the FIG_PROJECT_VARIABLE
and the --project-name
-option useless. which is handy for 'templating' services.
I think managing the unambiguity of the project name shouldn't be fig's responsibility.
I'm not so sure; Fig "silently" overwriting another projects' containers because if happened to be in a directory with the same name is sometimes nasty.
when i understood your proposal correctly, the generation of a random name would render the FIG_PROJECT_VARIABLE and the --project-name-option useless. which is handy for 'templating' services.
Following the discussion above, I think something like this would work
--project-name
is provided, use that (and write/update the .project-name
? Not sure)--project-name
is provided, but FIG_PROJECT_NAME
is, use FIG_PROJECT_NAME
(and write/update to .project-name
?).project-name
is set, use .project-name
.project-name
from someone's perspective who uses fig in scripts that take care of a name and pass it to fig, storing a name in the project (which is in my case rather a template) location makes no sense.
and still, i sensed it very intuitive that the directory name is used in the name of the resulting containers when i just ran fig up
.
and let say, you look at docker ps -a
, that won't be very helpful if it's filled up with random names.
would an option --random-name
help in your situation?
beside that i think the ability to prepend some [a-zA-Z0-9_].*
to reflect some namespacing would be useful for broader deployments.
What's about using the fig.yml to store such informations ?
schema-version: 1.1
project_name: foo
containers:
web:
build: .
command: python app.py
links:
- db
ports:
- "8000:8000"
db:
image: postgres
And to keep BC, in compose/project.py::from_config
@classmethod
def from_config(cls, name, config, client):
if 'schema-version' not in config:
config = {
'schema-version': '1.0',
'containers': config
}
dicts = []
for service_name, service in list(config.containers.items()):
if not isinstance(service, dict):
raise ConfigurationError('Service "%s" doesn\'t have any configuration options. All top level keys in your docker-compose.yml must map to a dictionary of configuration options.' % service_name)
service['name'] = service_name
dicts.append(service)
return cls.from_dicts(name, dicts, client)
@jderusse I'm very much for something like you suggested. Maybe you like to add your proposal to #45 as this is probably the better place to discuss it?
anyway to come to resolution on this ticket? It looks like the proposed #1233 gives you the best of both worlds by letting the user decide the intended action to take while not breaking any backwards compatibility.
Originally I wanted the name in the docker-compose.yml
, but after thinking about it more, I really like the idea of a separate file.
A separate file give you the option to keep it out of version control if you want (for cases where you want each user to be able to have a different project name).
With the new networking support there is feature request to be able to configure the network name as well (it defaults to the project name). So with a separate config file we could include things like default network name, default scale etc. All the meta-configuration that isn't part of the application definition, but is still relevant to running the composition.
I agree to what dnephin said. How about a file .docker-compose
? We could list default options for all command line arguments there.
@dnephin sgtm
@mikehaertl I used a .docker-compose
folder in my example, but a file could work as well, depending on how much information we want to store
@thaJeztah Oh, right, sorry. I missed that.
@mikehaertl just renamed them from .fig
to .docker-compose
;-)
Right :)
Actually I'd prefer a simple file. How about ini style? Global options on top, command specific options in a section:
project-name = blabla
[build]
no-cache
Some ideas for semi-related information we may want to store either in the same file, or same directory:
Now that we support multiple compose files, it might also be interesting to include the paths to the compose files to use as config.
client api version
possibly docker_host
?
That sounds like a good one as well (added)
I just ran into this problem.
I'm at a point where the basic structure of our docker configuration has been settled, and now I'm replicating it to other services. Since all of the projects have the same structure, they all use the same subdirectory (and hence, default project name). Which means I can't safely spool up 2 services on the same host.
I may have to change the structure so the project directory name ends up being the default, but I'm not 100% positive what that does about copying files at image build time.
Same here, I solved a similar issue with Makefile targets. But still, when you want to execute custom docker-compose
commands, you need to use -p customname
.
Right now I'm trying to figure out how to prevent docker-compose setups in /home/alice/myproject
and /home/bob/myproject
from stepping on each other's containers. Is there a way to deriving container names from the full file path instead of just the last directory name?
@chris-martin This is just a workaround...
alias docker-compose="docker-compose -p ${PWD}"
docker-compose strips /
out of project names, though?
Is it documented which characters are allowed in project names?
I did end up changing my directory structure and putting docker-compose.yml in the project root, which is a decent enough workaround
It grants some protection, if you tend to put all your git repositories into the same directory (eg, $HOME, or $HOME/projects), but it gets troublesome if you use another structure (say, one directory per organization, as I often do), or if your machine uses boot2docker, which tends to leak information across users on the same box unless you are careful to make sure docker-machine gives you distinct VMs.
I think #2294 or just being more hygienic with docker-machine would also fix my problems.
I'd also recommend to put it in the project root and have a different folder name.
In my case I'd like to set the initial project name because I have a shared machine on AWS and although I have the docker-compose.yml in the root a user may clone the repository into a directory with a different name. If that happens currently then they run into issues starting/stopping containers.
WIth the new v2
syntax this could be implemented like mentioned in https://github.com/docker/compose/issues/745#issuecomment-74028861
It's actually just a container_name_prefix
, isn't it?
@schmunk42
It's actually just a container_name_prefix, isn't it?
It's also used as volume_name_prefix
, image_name_prefix
and network_name_prefix
, so I guess that project_name
is a more generic term.
.docker-compose
sounds familiar to docker-compose.override.yml
in a lot of way. Plus, why introduce another configuration format (e.g. ini) when we already got YAML?
I'm therefore in favor of @jderusse's suggestion in #745 (comment) to add a project_name
key to the docker-compose.yml
, to enable the use case of @JosephEarl.
User-specific customization can already be implemented by using docker-compose.override.yml
and excluding it via .gitignore
.
If a third explicitly user-specific override file is desired, I'd suggest .docker-compose.local-override.yml
.
Regarding the structure of the YAML, I'd suggest creating a new top-level key called project
, which contains project_name
, and might contain other variables in the future. This avoids polluting the top-level namespace. To illustrate:
project:
project_name: "your-project"
network_prefix: "abc"
Note that, instead of using network_prefix
, it may be more understandable to directly customize the names of the networks themselves. As I understand it, there are two cases:
project_name
is enough.The proposal suggested above sounds good to me, I agree we don't want to introduce any extra config files
+1 on the concept of this issue
But also +1 on using a key in docker-compose.yml
instead of adding another file to our repos.
To sum up earlier comments: One proposal for the lookup sequence of the project name would be as follows; earlier items take precedence over later ones:
--project-name
if specifiedCOMPOSE_PROJECT_NAME
if specified.project_name
key from docker-compose.yml
(or wherever that setting is stored).basename
of the project directoryI'm unsure of the priority of 2 vs. 3:
--project-name
, COMPOSE_PROJECT_NAME
should definitely override project_name:
.COMPOSE_PROJECT_NAME
take precedence over project_name:
makes it possible to accidentally use the wrong project name due to the environment variable being set; this can be hard to debug. Maybe a warning message can be issues in this case?what's about introdusing a new property container_name_pattern
with the default value %(project_name)s_%(service_name)s_%(instance_number)s
to keep BC
this parameter lets us free to change it to hardcodedproject_%(service_name)s_%(instance_number)s
and definitively fix this issue
I have just arrived this issue after looking for this functionality for 10 minutes.
The first thing I did was searching for the right key in the Compose File Reference document, so +1 for project_name
key in docker-compose.yml
:+1: for @cr7pt0gr4ph7 strategy
+1 for @cr7pt0gr4ph7 strategy
Compose has syntax for environment substitution. Rather than using an environment variable for project name, maybe let people do it themselves if they care?
On the other hand, docker machine uses the end to decide which machine a compose build will be built on, so maybe the project name and the target should work alike. Hmm, this is a hard one.
https://github.com/docker/compose/issues/745#issuecomment-182296139 sounds right. The environment variable would take precedence over the value in the compose file, which acts as a default (assuming the project name belongs in the compose file).
Something to consider, what happens when you use multiple compose files that each define a project name?
I don't think it should be an error, that would force files to be either "primary" or "extras", which is not the case right now. Any file can be used in isolation or in combination with others.
Maybe a warning is reasonable, or maybe we just ignore it completely (since the value can be ignored by setting an option or environment variable as well).
Something to consider, what happens when you use multiple compose files that each define a project name?
If you do something like docker-compose -f compose1.yml -f compose2.yml up
and both files define the project name, the name used should be from compose2.yml
.
I think everyone is onto a solution I like already :- ) I.e. project_name
in docker-compose.yaml. I just want to mention another use case for having the project name in the docker-compose.yaml file:
In HTTP 500 Internal Server Error pages in my project, I tell the developer how to fix or troubleshoot the problem, for example I tell him/her that s/he can zcat database-dump.tgz | docker exec -i projectname_db_1 psql
if the server notices that no PostgreSQL database and user has been created — and then I want to choose projectname
myself, otherwise the help messages cannot be copy-pasted into the console.
Agree that there should be a warning when COMPOSE_PROJECT_NAME overrides the project name in the compose.yml - this behaviour makes sense based on the existing Docker commands, but is not particularly logical or obvious to a newcomer.
+1 for @cr7pt0gr4ph7 strategy
Tried searching for something before opening an issue myself as I couldn't find anything.
+1 for project_name
in docker.compose.yml file. With v2 I guess this makes more and more sense.
Agree that there should be a warning when COMPOSE_PROJECT_NAME overrides the project name in the compose.yml - this behaviour makes sense based on the existing Docker commands, but is not particularly logical or obvious to a newcomer.
It's pretty normal behaviour for environment variables to override config settings and for command line arguments to override config settings and environment variables.
Do we really a warning for that? It's not like anyone would accidentally create an environment variable named COMPOSE_PROJECT_NAME
.
Would it be OK to create a PR that just uses the project_name
from docker-compose.yml
to get this going? Or do we want the extra stuff like the using the project_name
value from the last yml
file referenced immediately?
How about this?
version: "2"
project:
default_name: "app"
I will change my implementation (https://github.com/docker/compose/pull/3118) for this format. But I'm not sure is there any need of project
section....
By default, Compose bases the project name on basename of the directory compose commands are run from. The project name can be overridden either by passing a
-p / --project-name
option for each command or setting theCOMPOSE_PROJECT_NAME
environment variable.Using the
--project-name
option for each command is error-prone and for- getting to pass the option when doing (for example)docker-compose up
, will result in another instance of the project being created under a different name or even containers of a different project being replaced.Using the
COMPOSE_PROJECT_NAME
environment variable is not useful when dealing with multiple projects and can cause similar problems.Proposal: make project-name persistent
To solve these problems, compose will save the name of a project to a file in the build-context when the project is first started / built.
If a project-file is found, compose automatically uses the project-name from that file and throw an exception if the user is trying to override the project-name using the
--project-name
option.File location
To allow further expansion of options, compose creates a hidden
.docker-compose
directory in the "root" directory of the build-context. Inside that directory, aproject-name
file is created, containing just the name of the project;Request for comment
There are a couple of things left to be discussed;
docker-compose init --project-name=foobar
)docker-compose destroy
)--file
) should the project-name also be applied to those? Should each compose-file get its own configuration?And, in a wider scope;
edit: renamed Fig to Compose