fastlane-queue / fastlane

Fastlane is a redis and docker based queueing service.
https://fastlane.readthedocs.io/en/latest/
MIT License
48 stars 14 forks source link

Run private images #9

Closed cassiobotaro closed 5 years ago

cassiobotaro commented 5 years ago

Is there a way to run private images? image

heynemann commented 5 years ago

Hi @cassiobotaro

Thanks for trying out fastlane! Hope it helps you as much as it's helping us!

Yes, fastlane allows any image from any repo, as long as docker can do a docker pull on it. Very likely your repository requires authentication for downloading the image. One simple fix is to change your docker repo to allow unauthenticated downloads, while requiring auth for upload (that's how we do it here).

If you can't do that, I have a test I'd like you to do. Please use this docker-compose.yml to get fastlane up:

version: '2'

services:
  mongo:
    image: mongo
    ports:
      - "10101:27017"
    volumes:
      - /tmp/fastlane/mongo:/data/db
  redis:
    image: redis
    ports:
      - "10100:6379"
    volumes:
      - /tmp/fastlane/redis:/data
  docker-host:
    image: docker:stable-dind
    privileged: true
  fastlane:
    image: heynemann/fastlane:1.3.4
    ports:
      - "10000:10000"
    environment:
      - DOCKER_HOSTS=[{"match":"","hosts":["<MY-MACHINE-IP>:2375"],"maxRunning":2}]
    depends_on:
      - mongo
      - redis
      - docker-host

And replace MY-MACHINE-IP for your ethernet card IP address (192. or 10. or something like that).

If you are running from mac os, you also need to expose your docker host API to the host machine. If you need to do it, you can check out how to do it here: https://github.com/docker/for-mac/issues/770

If you do change the port from 2375, please make sure you also change it accordingly in the docker-compose.yml.

After having the compose file properly set-up with your IP, run: docker-compose -f docker-compose.yml up -d

Fastlane should start pointing to your machine as the docker host. You can check everything is up and running by querying: 'http://localhost:10000/status/'

When you do this test, please post the results here!

Thanks again!

cassiobotaro commented 5 years ago

My attempt in steps was:

Expose docker api

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375

Copy and paste docker-compose file changing .

apparently worked

image

but when try to run a task:

image

heynemann commented 5 years ago

Can you try to do:

docker pull <url-to-image>

And see what happens?

cassiobotaro commented 5 years ago
cassiobotaro at gyr in ~ 
$ export DOCKER_HOST="tcp://127.0.0.1:2375"

cassiobotaro at gyr in ~ 
$ docker pull nuveo/base:python3           
python3: Pulling from nuveo/base
Digest: sha256:<xxxx>
Status: Image is up to date for nuveo/base:python3
heynemann commented 5 years ago

You must use the full name of the image. nuveo/base:python3 is not the full name (you missed the host, something like "docker.nuveo.com/nuveo/base:python3".

Does that help?

cassiobotaro commented 5 years ago

The image is hosted in docker hub, but is a private image. ~The docker host may not be a priority when updating the image?~ :thinking:

update: docker needs a client.login call before pull. This test works because I'am already logged in. I think the best solution is to create a public image.