dnephin / dobi

A build automation tool for Docker applications
https://dnephin.github.io/dobi/
Apache License 2.0
310 stars 36 forks source link

[ERROR] failed to load config from "dobi.yaml": duplicate resource name <..> #109

Closed OnkelTem closed 6 years ago

OnkelTem commented 6 years ago

When I try to use one name for different records I get an error message like this:

[ERROR] failed to load config from "dobi.yaml": duplicate resource name <..>

So let's put I have an image called myimage and I just want to run it. This is a dobi.yaml which comes to my mind first:

image=myimage:
    image: myimage
    tags: ['mytag']
    context: images/myimage
    dockerfile: Dockerfile

job=myimage:
    use: myimage
    command: /entrypoint.sh

But this is not allowed, as I can't use "myimage" twice. So I have to rename them but since I don't have any new names in my mind (I don't have any new entities except of those I've already mentioned!) I have to conjure up some new names inevitably running into the naming problem:

image=myimage-image:
    image: myimage
    tags: ['mytag']
    context: images/myimage
    dockerfile: Dockerfile

job=myimage:
    use: myimage-image
    command: /entrypoint.sh

which looks pretty bogus, since: 1) I had to add -image suffix while I already had image= prefix. 2) In the second line I say one more time: image: myimage. 3) In the job specs I have to use myimage-image to reference image which doesn't even look like my original image which name was myimage.

As for (2) - it's inevitable I understand, but other two are the result of that uniqueness requirement. So why require uniqueness of keys after all? It doesn't make much sense for me. Just make type=name unique, not just name.

OnkelTem commented 6 years ago

Further thoughts on adding some syntactic sugar. They are quite expected.

First, let's not require image: field in the image= specs, default to the value which is already provided:

image=myimage:
    tags: ['mytag']
    context: images/myimage
    dockerfile: Dockerfile

Allow for tags specification right in the key line:

image=myimage:mytag:
    context: images/myimage
    dockerfile: Dockerfile

Let dockerfile default to Dockerfile:

image=myimage:mytag:
    context: images/myimage

and etc

dnephin commented 6 years ago

As for (2) - it's inevitable I understand, but other two are the result of that uniqueness requirement. So why require uniqueness of keys after all? It doesn't make much sense for me. Just make type=name unique, not just name.

The reason for this is because all resources are "runnable", not just jobs. If only type=name were unique, what would dobi myimage do? Does it build the image or does it run the job? It would be ambiguous. So I think it's necessary that the name part of the resource definition is unique.

For this reason I generally come up with a naming scheme for my resources. Ex: image=builder and job=build, image=linter, job=lint.

let's not require image: field in the image= specs, default to the value which is already provided

I agree, I like this idea. Maybe the image: field should also be renamed to name:.

Allow for tags specification right in the key line

I think this isn't possible for two reasons:

Let dockerfile default to Dockerfile

I agree. I think this is already done in version 0.10

OnkelTem commented 6 years ago

The reason for this is because all resources are "runnable", not just jobs.

Yep, and this is another problem. At first sight it saves typing, right? I.e not dobi job myjob, but simply dobi myjob and I kind of "save" 4 bytes (for myself). But on the other hand if you don't name a job with a verb or don't add a designating suffix like "-job", one wouldn't know just by looking at its name - is it a job or maybe an image or a mount or other resource, right?

And jobs is not really the worst case, think about possible naming conventions for mounts, images, aliases. I personally can't think about any except of artificially augmenting them with type specifier (suffix/prefix).

Now when it comes to the command line, it could be really tough to understand what is gonna be done, e.g.:

dobi something

The only thing we know for sure - dobi is gonna execute some default task of "something" resource. And that default task depends on the type of something of which we have no idea. Had it been written like:

dobi mount something

we would know that the resource is of type "mount" and dobi would create a host directory for the something mount.

dnephin commented 6 years ago

Yep, and this is another problem.

I don't see this is a problem at all. dobi also runs all dependencies. So dobi something is always going to run more than just the target task, it will run all the dependencies then finally it will run the target. Adding the resource type does not really tell you any more about what it might run. Also knowing it is a job doesn't really tell you much about what it's going to do either. Either way you have to read the config to know what it's going to do, or at least run dobi list to see the described tasks to get a quick idea of what it should do.

I don't think it's a problem that the type of the task is not obvious from the name. The task should be named in such a way that it matches the intent. Ex: I want to run tests, dobi test. I want to build all the images for release dobi release-images. These tasks could be any resource type with a bunch of dependencies, including an alias that is a list of other resources types.

dnephin commented 6 years ago

I opened #110 for making image.image optional

OnkelTem commented 6 years ago

Imagine that docker devs at some point would remove 'images' and 'volumes' command. By just reading provided ID it could borrow it's nature and decide what to do - couldn't it? This is how the mess with different resource types in Dobi looks like atm.

The task should be named in such a way that it matches the intent.

But resources are not tasks. It's Dobi who treats them as tasks assuming actually some default action to be performed against the provided resource name. But I already pointed out on this problem, so I won't repeat myself. What is important here is that this problem brings the bigger one: naming collisions (see above). With Dobi we need to think out the whole layer of unique names.

dnephin commented 6 years ago

Imagine that docker devs at some point would remove 'images' and 'volumes' command. By just reading provided ID it could borrow it's nature and decide what to do - couldn't it?

This is kind of how docker inspect works. The difference is that docker objects do not have globally unique ids or names. They are only checked for uniqueness with other objects of the same type. That was part of the design. dobi has a different design. It's a more specific use case.

This is how the mess with different resource types in Dobi looks like atm.

I don't see how it's a mess. Just pick appropriate names for resources.

What is important here is that this problem brings the bigger one: naming collisions

Yes, it does require you to pick unique names for each resource. If your proposal is to require the resource type along with the name of the resource then you could just make a convention in your projects to name resources that way: mount=mount-foo, image=image-foo, job=job-foo. The design of dobi doesn't need to change to accommodate that naming scheme.

dnephin commented 6 years ago

I'm going to close this since I'd like to keep the resource prefix and all resources in the same namespace. #110 was opened to remove duplication.