ansible / ansible-container

DEPRECATED -- Ansible Container was a tool to build Docker images and orchestrate containers using only Ansible playbooks.
GNU Lesser General Public License v3.0
2.19k stars 394 forks source link

How to synchronize "/src/" parent with container #907

Open justinmayer opened 6 years ago

justinmayer commented 6 years ago

I've searched existing issues, including the tangentially-related #393, but I've yet to find a solution.

ISSUE TYPE
container.yml
version: "2"
settings:

  conductor:
    base: debian:stretch

  project_name: foo

services:
  foo:
    from: "python:3.6-stretch"
    roles:
      - containerize
    entrypoint: /usr/bin/dumb-init
    command: ls -la
    ports:
      - 8000:8000
    working_dir: /app
    user: deploy

registries: {}
OS / ENVIRONMENT
ansible-container --debug version
Ansible Container, version 0.9.2
Darwin, jBook.local, 16.7.0, Darwin Kernel Version 16.7.0: Mon Nov 13 21:56:25 PST 2017; root:xnu-3789.72.11~1/RELEASE_X86_64, x86_64
2.7.14 (default, Mar 10 2018, 00:01:04)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] /Users/justin/virtualenvs/kbs/bin/python2.7
{
  "ContainersPaused": 0,
  "Labels": [],
  "CgroupDriver": "cgroupfs",
  "ContainersRunning": 1,
  "ContainerdCommit": {
    "Expected": "89623f28b87a6004d4b785663257362d1658a729",
    "ID": "89623f28b87a6004d4b785663257362d1658a729"
  },
  "InitBinary": "docker-init",
  "NGoroutines": 47,
  "Swarm": {
    "ControlAvailable": false,
    "NodeID": "",
    "Error": "",
    "RemoteManagers": null,
    "LocalNodeState": "inactive",
    "NodeAddr": ""
  },
  "LoggingDriver": "json-file",
  "OSType": "linux",
  "HttpProxy": "docker.for.mac.http.internal:3128",
  "Runtimes": {
    "runc": {
      "path": "docker-runc"
    }
  },
  "DriverStatus": [
    [
      "Backing Filesystem",
      "extfs"
    ],
    [
      "Supports d_type",
      "true"
    ],
    [
      "Native Overlay Diff",
      "true"
    ]
  ],
  "OperatingSystem": "Docker for Mac",
  "Containers": 1,
  "HttpsProxy": "docker.for.mac.http.internal:3129",
  "BridgeNfIp6tables": true,
  "MemTotal": 2096103424,
  "SecurityOptions": [
    "name=seccomp,profile=default"
  ],
  "Driver": "overlay2",
  "IndexServerAddress": "https://index.docker.io/v1/",
  "ClusterStore": "",
  "InitCommit": {
    "Expected": "949e6fa",
    "ID": "949e6fa"
  },
  "GenericResources": null,
  "Isolation": "",
  "SystemStatus": null,
  "OomKillDisable": true,
  "ClusterAdvertise": "",
  "SystemTime": "2018-03-19T12:08:08.711597588Z",
  "Name": "linuxkit-025000000001",
  "CPUSet": true,
  "RegistryConfig": {
    "AllowNondistributableArtifactsCIDRs": [],
    "Mirrors": [],
    "IndexConfigs": {
      "docker.io": {
        "Official": true,
        "Name": "docker.io",
        "Secure": true,
        "Mirrors": []
      }
    },
    "AllowNondistributableArtifactsHostnames": [],
    "InsecureRegistryCIDRs": [
      "127.0.0.0/8"
    ]
  },
  "DefaultRuntime": "runc",
  "ContainersStopped": 0,
  "NCPU": 2,
  "NFd": 30,
  "Architecture": "x86_64",
  "KernelMemory": true,
  "CpuCfsQuota": true,
  "Debug": true,
  "ID": "IQQQ:RMYE:VRWR:I34Q:YXMG:Y6OZ:A3QX:SGED:CR5H:IVPY:EKVM:GPVJ",
  "IPv4Forwarding": true,
  "KernelVersion": "4.9.60-linuxkit-aufs",
  "BridgeNfIptables": true,
  "NoProxy": "",
  "LiveRestoreEnabled": false,
  "ServerVersion": "17.12.0-ce",
  "CpuCfsPeriod": true,
  "ExperimentalBuild": true,
  "MemoryLimit": true,
  "SwapLimit": true,
  "Plugins": {
    "Volume": [
      "local"
    ],
    "Network": [
      "bridge",
      "host",
      "ipvlan",
      "macvlan",
      "null",
      "overlay"
    ],
    "Authorization": null,
    "Log": [
      "awslogs",
      "fluentd",
      "gcplogs",
      "gelf",
      "journald",
      "json-file",
      "logentries",
      "splunk",
      "syslog"
    ]
  },
  "Images": 12,
  "DockerRootDir": "/var/lib/docker",
  "NEventsListener": 2,
  "CPUShares": true,
  "RuncCommit": {
    "Expected": "b2567b37d7b75eb4cf325b77297b140ea686ce8f",
    "ID": "b2567b37d7b75eb4cf325b77297b140ea686ce8f"
  }
}
{
  "KernelVersion": "4.9.60-linuxkit-aufs",
  "Components": [
    {
      "Version": "17.12.0-ce",
      "Name": "Engine",
      "Details": {
        "KernelVersion": "4.9.60-linuxkit-aufs",
        "Os": "linux",
        "BuildTime": "2017-12-27T20:12:29.000000000+00:00",
        "ApiVersion": "1.35",
        "MinAPIVersion": "1.12",
        "GitCommit": "c97c6d6",
        "Arch": "amd64",
        "Experimental": "true",
        "GoVersion": "go1.9.2"
      }
    }
  ],
  "Arch": "amd64",
  "BuildTime": "2017-12-27T20:12:29.000000000+00:00",
  "ApiVersion": "1.35",
  "Platform": {
    "Name": ""
  },
  "Version": "17.12.0-ce",
  "MinAPIVersion": "1.12",
  "GitCommit": "c97c6d6",
  "Os": "linux",
  "Experimental": true,
  "GoVersion": "go1.9.2"
}
SUMMARY

We store Ansible-related configuration at in a sub-directory of the project root — not in the project root itself. So ansible.cfg, container.yml, roles, etc. are all stored in project_root/deploy.

Given this set-up, I cannot figure out how to get the project code synchronized/copied into the container.

STEPS TO REPRODUCE

The following task, for example, only synchronizes the sub-directory (project_root/deploy) and not the desired project_root parent directory:

- name: Copy source into container
  synchronize:
    src: "/src/"
    dest: "/app"
  remote_user: deploy

No matter what local source path values I try for src: — including src: /Users/justin/Projects/foo, the full absolute path to the project root — a "directory not found" error is returned (even though the specified paths definitely exist):

TASK [containerize : Copy source into container] *******************************
fatal: [foo]: FAILED! => {"changed": false, "cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh=/usr/bin/ssh -S none -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null --blocking-io --rsh=/usr/local/bin/docker exec -i --out-format=<<CHANGED>>%i %n%L /Users/justin/Projects/foo/ f975bb5f53c9ac0f96b9e6a88e58d5eaa8aff7f22a28470193924c77b9a924c5:/app", "failed": true, "msg": "rsync: change_dir \"/Users/justin/Projects/foo\" failed: No such file or directory (2)\nrsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1196) [sender=3.1.2]\n", "rc": 23}
EXPECTED RESULTS

I expected to be able to find some way to synchronize/copy the parent folder of the current working directory into the container.

ACTUAL RESULTS

I cannot find any way to synchronize/copy the parent folder of the current working directory into the container.

treyd commented 6 years ago

What is the exact ansible-container command you are running, and from which directory are you running it? It sounds like you might need to be using the --src-mount-path option of build.

justinmayer commented 6 years ago

Exact command: ansible-container build

Running command from: project_root/deploy

@treyd: Does that help narrow down potential causes of this problem?

Voronenko commented 6 years ago

@justinmayer

I have similar setup, and I do resolve that in the following way:

on my makefile I have specified path to my "real code root"

# path to service
CODE_PATH=$(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))/../../

and on build I am passing it via --src-mount-path

build:  p-env/bin/ansible-container
        @echo ansible-container --vars-files vars_$(IMAGE_BASE).yml --debug --project-name $(ROLE_NAME) build --roles-path ./roles/ --src-mount-path $(CODE_PATH) -- -vvv
        @p-env/bin/ansible-container --debug --project-name $(ROLE_NAME) build --no-container-cache --roles-path ./roles/ --src-mount-path $(CODE_PATH) -- -vvv

Step of copiing sources to container from that location - it is synchronize module now.

  - name: Copy source into container
    synchronize:
      src: "/src/{{application_artifact_path}}"
      dest: "/app"
treyd commented 6 years ago

@justinmayer I think you need to use --src-mount-path (e.g. --src-mount-path project_root) and refer to the parent directory you want to have access to in your roles. The src directory is a bind-mount into the conductor, and so from within the conductor you can't get access to its parents.

Voronenko commented 6 years ago

@justinmayer is issue still actual ?

adpoe commented 5 years ago

I can confirm still seeing this.

No matter what I do, I'm unable to copy or synchronize files:

In essence, I cannot get either copy or synchronize to work with ansible-container. So I'm wondering how this is typically done.

I can provide more details if you have any specific questions, since I have setup to test/confirm with.

Voronenko commented 5 years ago

Mentioned part for sure works. Compare approaches below with the one you are using.

https://github.com/softasap/sa-container-bootstrap/tree/master/box-example/alpine-35 fallback to packer: https://github.com/softasap/sa-container-bootstrap/tree/master/box-example-packer/alpine-35

If source is out of context , you can use --src-mount-path switch

ansible-container --vars-files vars_$(IMAGE_BASE).yml --debug --project-name $(ROLE_NAME) build --roles-path ./roles/ --src-mount-path $(CODE_PATH) -- -vvv

Play part for example above might look like

  - name: Copy source into container
    synchronize:
      src: "/src/src"
      dest: "/app"

  - name: Copy migrations into container
    synchronize:
      src: "/src/migrations"
      dest: "/app"

  - name: Copy requirements.txt into container
    synchronize:
      src: "/src/requirements.txt"
      dest: "/app"

  - name: Copy start.sh into container
    synchronize:
      src: "/src/start.sh"
      dest: "/app"

  - name: Install Python dependencies
    pip:
      requirements: /app/requirements.txt
      executable: "{{container_pip}}"

  - name: RUNIT | Creating /etc/service/payment
    file: path="/etc/service/app" state="directory"

  - name: RUNIT | Template  /etc/service/payment/run
    template: src="{{role_dir}}/templates/application.runit.j2" dest="/etc/service/app/run" mode="u=rwx,g=rx,o=rx"