`deis push`: push to Deis from local Docker graph

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @renegare on December 6, 2014 14:25

In order to deploy a privately built image to deis cluster
As a developer
I would like to execute the following commands to push and deploy a local image

`deis push <local_image_name>:<tag>`
`deis deploy <local_image_name>:<tag>` # not to be confused with `deis pull`

Currently as of deis v1.0.2 all of this is possible, but through the [not so elegant] steps:

docker build -t <app_name>... # build your image locally
docker tag <app_name>:latest 127.0.0.1:5000/<app_name>:`git rev-parse --short HEAD` # tag your app locally
ssh -Nf -L 5000:127.0.0.1:5000 core@$REGISTRY_HOST # port-forward the deis-registry port (make sure also your docker daemon allows this insecure registry)
docker push 127.0.0.1:5000/<app_name>
killall ssh # not elegant
deis pull <core-host-docker-registry-service-ip>:5000/<app_name>:<tag> # actually deploy your image

This clearly not elegant (I have it running on a CI #ignant) and a few too many moving parts.

I'm hoping for a discussion around this, I am still oblivious to any related concerns around the possibility of this feature.

Copied from original issue: deis/deis#2680

Copied from original issue: deis/workflow#709

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @PierreKircher on December 6, 2014 14:30

wont work everywhere ..:> http://content.screencast.com/users/blackdolpinmedia/folders/Jing/media/e712a502-2c06-4fce-bca6-40d54557d8c7/00000013.png

to example .. is a simple vrrp ( floating ip aka lbl construct ) .. the nodes them self just got a privat lan.. so youd need to push the stuff throw the lbl + register the registry on the deis-router which is not the case right now .. nor does deis support auth for the internal reg .. (security consirn) .. use a external privat registry .. thats the best bet right now

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @renegare on December 6, 2014 14:49

@PierreKircher the hackyway I described above, is by creating a direct connection to the deis-registry (and has nothing to do with a load balancer or router). One can use fleetctl / deisctl to track down the IP of the deis-registry unit.

What I never made clear from the beginning is that deis does not currently support private docker registries (correct me if i am wrong). And that was the driving factor of this.

Not every one pushes through the Buildpacks, and Dockerfile deployment takes a long time to build.

Finally: no deis components have been modified in order to achieve this. More of hacked work flow than a hack in the code.

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @mboersma on December 6, 2014 15:38

@renegare we did actually have a PR at one point to add deis push to the mix: https://github.com/deis/deis/pull/1412. I can see how it would be useful. I've kept these commits in a branch in case we want to resurrect it. I think the main issue was how to deal with registry authentication, but there may be more to it than that.

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @renegare on December 9, 2014 12:36

@mboersma, am I right in assuming that if we had a deis push command, all requests will need to go through the lbl > router > controller > registry?

If so (which I think is entirely possible) performance concerns me and will invalidate the usefulness of this feature. E.g. pushing a heavy image blocks other app requests from getting through etc ...

How does git push flow internally work from a high level?

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @bacongobbler on December 9, 2014 16:6

http://docs.deis.io/en/latest/understanding_deis/concepts/#build-release-run

git push goes from lbl > router > builder, which builds the image, pushes it to the registry and notifies the controller that a new build has been created.

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @wenzowski on March 11, 2015 8:15

/bump to join this thread

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @carmstrong on June 7, 2015 19:55

This feature is on our open roadmap: http://docs.deis.io/en/latest/roadmap/roadmap/

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @renegare on June 8, 2015 13:46

Nice one guys. Been AWOL, but great to see this.

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @bacongobbler on June 26, 2015 22:24

So to think about this feature, there are a few things I can think of. If this is good, then I'll move to a more formal proposal with doc changes:

• break out the authentication API in the controller as its own microservice. It should be pluggable, having the ability to communicate to other auth servers (like LDAP). This'll most likely be pulling out the existing auth service built into the controller and re-implementing it as a standalone service
• update reference docs, API docs, architecture diagrams etc. to reference this new service
• lock down auth access through registry v2's pluggable auth system by communicating to this new auth microservice. This means we're blocked until we close #3814, since the v1 registry has no pluggable auth service
• provide a client workflow that wraps docker push with the auth server + a call to the controller to create a new build
• deprecate deis pull with this new workflow. This might be just a big fat warning in the client before we remove those endpoints in a future release.

Does that seem reasonable to everyone? :)

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @aledbf on June 26, 2015 23:32

@bacongobbler please do not remove deis pull. I use a remote registry to deploy to production, not something from a local docker graph

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @bacongobbler on June 26, 2015 23:38

I guess there's no true need to remove it. The motivation to remove deis pull is that it's a giant hack to pull in an image, with intimate logic around the v1 image layer format. It doesn't work with v1.6+ images at all and we cannot support it with the v2 registry unless we re-implemented the crazily slow "pull, push, pull" workflow with a docker engine, so we'd need to do that in tandem with #3814.

cc'ing @mboersma for visibility since he's working on the v2 registry

[Cryptophobia]

From @deis-admin on January 19, 2017 21:15

From @alexanderkiel on August 25, 2015 21:17

I really like the idea from @renegare of a simple deis deploy <image> and I like to extend it to remove the need of a deis push before it. The deis client could handle the retrieval of local or remote images and just deploy it through one single API. There is no need that the deis servers pull images from somewhere or that I have to push images into the deis registry before I can deploy them. All I want to do is: create an app, deploy an image and config it.