search

nginx-proxy / acme-companion

Automated ACME SSL certificate generation for nginx-proxy
MIT License
7.39k stars 821 forks source link

I'm stuck at writing dockerrun.aws.json for aws ecs #453

Open Feelthewind opened 5 years ago

Feelthewind commented 5 years ago

Hi. I followed along Wiki stuffs like base usage and persistent data parts. It says make three writable volumes, but i'm not sure about how to write this on dockerrun.aws.json file. All examples make use of anonymous or nameed volumes for these three volumes. I don't know how to implement this on dockerrun.aws.json file and i think i make host volumes for all the things.

{
  "AWSEBDockerrunVersion": 2,
  "volumes": [
    {
      "name": "vhost",
      "host": {
        "sourcePath": "/etc/nginx/vhost.d"
      }
    },
    {
      "name": "html",
      "host": {
        "sourcePath": "/usr/share/nginx/html"
      }
    },
    {
      "name": "certs",
      "host": {
        "sourcePath": "/var/app/current/certs"
      }
    },
    {
      "name": "sock",
      "host": {
        "sourcePath": "/var/app/current/sock"
      }
    }
  ],
  "containerDefinitions": [
    {
      "name": "client",
      "image": "west0321/multi-client",
      "hostname": "client",
      "essential": false,
      "memory": 64,
      "environment": [
        {
          "name": "VIRTUAL_HOST",
          "value": "pyochan.com,www.pyochan.com"
        },
        {
          "name": "LETSENCRYPT_HOST",
          "value": "pyochan.com,www.pyochan.com"
        }
      ]
    },
    {
      "name": "server",
      "image": "west0321/multi-server",
      "hostname": "api",
      "essential": false,
      "memory": 64,
      "environment": [
        {
          "name": "VIRTUAL_HOST",
          "value": "api.pyochan.com"
        },
        {
          "name": "LETSENCRYPT_HOST",
          "value": "api.pyochan.com"
        }
      ]
    },
    {
      "name": "worker",
      "image": "west0321/multi-worker",
      "hostname": "worker",
      "essential": false,
      "memory": 64
    },
    {
      "name": "nginx-proxy",
      "image": "jwilder/nginx-proxy",
      "hostname": "nginx-proxy",
      "essential": true,
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80
        },
        {
          "hostPort": 443,
          "containerPort": 443
        }
      ],
      "links": ["client", "server", "letsencrypt"],
      "memory": 64,
      "dockerLabels": {
        "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy": "true"
      },
      "mountPoints": [
        {
          "sourceVolume": "vhost",
          "containerPath": "/etc/nginx/vhost.d"
        },
        {
          "sourceVolume": "html",
          "containerPath": "/usr/share/nginx/html"
        },
        {
          "sourceVolume": "certs",
          "containerPath": "/etc/nginx/certs"
        },
        {
          "sourceVolume": "sock",
          "containerPath": "/tmp/docker.sock"
        }
      ]
    },
    {
      "name": "letsencrypt",
      "image": "jrcs/letsencrypt-nginx-proxy-companion",
      "hostname": "letsencrypt",
      "essential": false,
      "memory": 64,
      "mountPoints": [
        {
          "sourceVolume": "certs",
          "containerPath": "/etc/nginx/certs"
        },
        {
          "sourceVolume": "sock",
          "containerPath": "/var/run/docker.sock"
        }
      ],
      "volumesFrom": [
        {
          "sourceContainer": "nginx-proxy"
        }
      ]
    }
  ]
}

Could you tell me whether my dockerrun.aws.json file is correct or not. Would you be able to give an example for dockerrun.aws.json file for this library? Any help would be really appreciated.

https://github.com/Feelthewind/multi-docker

I got error message like below.

/var/log/ecs/ecs-agent.log.2018-10-12-04

2018-10-12T04:02:04Z [INFO] TCS Websocket connection closed for a valid reason 2018-10-12T04:02:04Z [INFO] Connected to TCS endpoint 2018-10-12T04:06:54Z [INFO] Handling http requestmethodGETfrom127.0.0.1:54736 2018-10-12T04:07:00Z [WARN] Could not resolve some containers: [[nginx-proxy(jwilder/nginx-proxy) (NONE->RUNNING) letsencrypt(jrcs/letsencrypt-nginx-proxy-companion) (NONE->RUNNING)]] for task awseb-MultiDocker-env-myufbmg9mm:10 arn:aws:ecs:ap-northeast-1:223630899131:task/a5f14f58-6d24-43f8-bce7-f11188e2ce71, TaskStatus: (STOPPED->STOPPED) Containers: [worker (NONE->RUNNING),client (NONE->RUNNING),server (NONE->RUNNING),nginx-proxy (NONE->RUNNING),letsencrypt (NONE->RUNNING),] 2018-10-12T04:07:00Z [ERROR] Task engine [arn:aws:ecs:ap-northeast-1:223630899131:task/a5f14f58-6d24-43f8-bce7-f11188e2ce71]: unable to progress task with circular dependencies

/var/log/eb-activity.log

[2018-10-12T04:55:27.034Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage0/EbExtensionPostBuild] : Completed activity. [2018-10-12T04:55:27.035Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage0/InfraCleanEbextension] : Starting activity... [2018-10-12T04:55:27.037Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage0/InfraCleanEbextension] : Completed activity. Result: Cleaned ebextensions subdirectories from /var/app/staging. [2018-10-12T04:55:27.037Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage0] : Completed activity. Result: Application update - Command CMD-AppDeploy stage 0 completed [2018-10-12T04:55:27.037Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage1] : Starting activity... [2018-10-12T04:55:27.037Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage1/AppDeployEnactHook] : Starting activity... [2018-10-12T04:55:27.038Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage1/AppDeployEnactHook/00stop-task.sh] : Starting activity... [2018-10-12T04:55:27.048Z] INFO [32096] - [Application update travis-3f59e23ce96309a4d80a84edb477232a01f914a2-1539320107@9/AppDeployStage1/AppDeployEnactHook/00stop-task.sh] : Completed activity. Result:

ampledata commented 5 years ago

@Feelthewind I'm in the same boat with the same errors and an almost identical config. Will post more as I continue to debug.

ampledata commented 5 years ago

Multi-docker Elastic Beanstalk.

My Dockerrun.aws.json

{
  "AWSEBDockerrunVersion": 2,
  "volumes": [
    {
      "name": "nginx-vhostd",
      "host": { "sourcePath": "/etc/nginx/vhost.d" }
    },
    {
      "name": "nginx-html",
      "host": { "sourcePath": "/usr/share/nginx/html" }
    },
    {
      "name": "nginx-certs",
      "host": { "sourcePath": "/home/ec2-user/certs" }
    },
    {
      "name": "docker-sock",
      "host": { "sourcePath": "/var/run/docker.sock" }
    }
  ],
  "containerDefinitions": [
    {
      "name": "nginx",
      "image": "jwilder/nginx-proxy",
      "essential": true,
      "memoryReservation": 128,
      "dockerLabels": {
        "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy": "true"
      },
      "mountPoints": [
        {
          "containerPath": "/etc/nginx/vhost.d",
          "sourceVolume": "nginx-vhostd"
        },
        {
          "containerPath": "/usr/share/nginx/html",
          "sourceVolume": "nginx-html"
        },
        {
          "containerPath": "/etc/nginx/certs",
          "sourceVolume": "nginx-certs",
          "readOnly": true
        },
        {
          "containerPath": "/tmp/docker.sock",
          "sourceVolume": "docker-sock",
          "readOnly": true
        }
      ],
      "portMappings": [
        { "containerPort": 80, "hostPort": 80 },
        { "containerPort": 443, "hostPort": 443 }
      ]
    },
    {
      "name": "nginx-letsencrypt",
      "image": "jrcs/letsencrypt-nginx-proxy-companion",
      "essential": true,
      "memoryReservation": 64,
      "links": [ "nginx" ],
      "mountPoints": [
        {
          "containerPath": "/etc/nginx/certs",
          "sourceVolume": "nginx-certs"
        },
        {
          "containerPath": "/var/run/docker.sock",
          "sourceVolume": "docker-sock",
          "readOnly": true
        }
      ]
    },
    {
      "name": "traccar",
      "image": "openjdk:8-jre-alpine",
      "memory": 256,
      "portMappings": [
        { "hostPort": 8082, "containerPort": 8082 },
        { "hostPort": 5055, "containerPort": 5055 }
      ],
      "environment": [
        { "name": "VIRTUAL_HOST", "value": "xxx.xxx.com" },
        { "name": "VIRTUAL_PORT", "value": "8082" },
        { "name": "LETSENCRYPT_HOST", "value": "xxx.xxx.com" },
        { "name": "LETSENCRYPT_EMAIL", "value": "xxx@xxx.com" }
      ]
    }
  ]
}

eb deploy error:

2018-10-26 04:36:34    INFO    Environment update is starting.
2018-10-26 04:36:49    INFO    Deploying new version to instance(s).
2018-10-26 04:37:17    ERROR   Failed to start ECS task: arn:aws:ecs:us-west-2:966821920696:task/b8ced724-6a08-420e-8e8c-d533865a8f50 is STOPPED.
2018-10-26 04:37:17    ERROR   ECS task stopped due to: Task failed to start. (nginx-letsencrypt:
nginx:
traccar: )
2018-10-26 04:37:25    ERROR   ECS task stopped due to: Task failed to start. (nginx:
traccar:
nginx-letsencrypt: )
2018-10-26 04:37:25    ERROR   Failed to start ECS task: arn:aws:ecs:us-west-2:966821920696:task/ee88ad37-d611-4f25-8d50-69eda81e6b0e is STOPPED.
2018-10-26 04:37:28    ERROR   Failed to start ECS task after retrying 2 times.
2018-10-26 04:37:30    ERROR   [Instance: i-0d2477023b6eea177] Command failed on instance. Return code: 1 Output: .' --severity ERROR
+ exit 1.
Hook /opt/elasticbeanstalk/hooks/appdeploy/enact/03start-task.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2018-10-26 04:37:30    INFO    Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2018-10-26 04:37:30    ERROR   Unsuccessful command execution on instance id(s) 'i-0d2477023b6eea177'. Aborting the operation.
2018-10-26 04:37:30    ERROR   Failed to deploy application.

ERROR: ServiceError - Failed to deploy application.
make: *** [deploy] Error 4

tail -100 /var/log/eb-activity.log:

[2018-10-26T04:37:03.600Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14/AppDeployStage1/AppDeployEnactHook/02update-credentials.sh] : Starting activity...
[2018-10-26T04:37:03.824Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14/AppDeployStage1/AppDeployEnactHook/02update-credentials.sh] : Completed activity.
[2018-10-26T04:37:03.824Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14/AppDeployStage1/AppDeployEnactHook/03start-task.sh] : Starting activity...
[2018-10-26T04:37:29.007Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14/AppDeployStage1/AppDeployEnactHook/03start-task.sh] : Activity execution failed, because: + . /opt/elasticbeanstalk/hooks/common.sh
  + RETRY=0
  + initctl start eb-ecs EB_EVENT_FILE=/tmp/eventsfile20181026-20389-13sqqwv
  initctl: Job failed to start
  + '[' -f /etc/elasticbeanstalk/.eb-ecs-start-no-retry ']'
  + warn 'Failed to start ECS task, retrying...'
  + echo 'Failed to start ECS task, retrying...'
  Failed to start ECS task, retrying...
  + eventHelper.py --msg 'Failed to start ECS task, retrying...' --severity WARN
  + RETRY=1
  + sleep 3
  + '[' 1 -gt 1 ']'
  + initctl start eb-ecs EB_EVENT_FILE=/tmp/eventsfile20181026-20389-13sqqwv
  initctl: Job failed to start
  + '[' -f /etc/elasticbeanstalk/.eb-ecs-start-no-retry ']'
  + warn 'Failed to start ECS task, retrying...'
  + echo 'Failed to start ECS task, retrying...'
  Failed to start ECS task, retrying...
  + eventHelper.py --msg 'Failed to start ECS task, retrying...' --severity WARN
  + RETRY=2
  + sleep 3
  + '[' 2 -gt 1 ']'
  + error_exit 'Failed to start ECS task after retrying 2 times.' 1
  + error 'Failed to start ECS task after retrying 2 times.'
  + echo 'Failed to start ECS task after retrying 2 times.'
  Failed to start ECS task after retrying 2 times.
  + eventHelper.py --msg 'Failed to start ECS task after retrying 2 times.' --severity ERROR
  + exit 1 (ElasticBeanstalk::ExternalInvocationError)
caused by: + . /opt/elasticbeanstalk/hooks/common.sh
  + RETRY=0
  + initctl start eb-ecs EB_EVENT_FILE=/tmp/eventsfile20181026-20389-13sqqwv
  initctl: Job failed to start
  + '[' -f /etc/elasticbeanstalk/.eb-ecs-start-no-retry ']'
  + warn 'Failed to start ECS task, retrying...'
  + echo 'Failed to start ECS task, retrying...'
  Failed to start ECS task, retrying...
  + eventHelper.py --msg 'Failed to start ECS task, retrying...' --severity WARN
  + RETRY=1
  + sleep 3
  + '[' 1 -gt 1 ']'
  + initctl start eb-ecs EB_EVENT_FILE=/tmp/eventsfile20181026-20389-13sqqwv
  initctl: Job failed to start
  + '[' -f /etc/elasticbeanstalk/.eb-ecs-start-no-retry ']'
  + warn 'Failed to start ECS task, retrying...'
  + echo 'Failed to start ECS task, retrying...'
  Failed to start ECS task, retrying...
  + eventHelper.py --msg 'Failed to start ECS task, retrying...' --severity WARN
  + RETRY=2
  + sleep 3
  + '[' 2 -gt 1 ']'
  + error_exit 'Failed to start ECS task after retrying 2 times.' 1
  + error 'Failed to start ECS task after retrying 2 times.'
  + echo 'Failed to start ECS task after retrying 2 times.'
  Failed to start ECS task after retrying 2 times.
  + eventHelper.py --msg 'Failed to start ECS task after retrying 2 times.' --severity ERROR
  + exit 1 (Executor::NonZeroExitStatus)

[2018-10-26T04:37:29.008Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14/AppDeployStage1/AppDeployEnactHook/03start-task.sh] : Activity failed.
[2018-10-26T04:37:29.008Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14/AppDeployStage1/AppDeployEnactHook] : Activity failed.
[2018-10-26T04:37:29.008Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14/AppDeployStage1] : Activity fa
iled.
[2018-10-26T04:37:29.008Z] INFO  [20389] - [Application update app-ecdd-181025_213432-stage-181025_213432@14] : Completed activity. Result:
  Application update - Command CMD-AppDeploy failed

docker logs:

WARNING: /etc/nginx/dhparam/dhparam.pem was not found. A pre-generated dhparam.pem will be used for now while a new one
is being generated in the background.  Once the new dhparam.pem is in place, nginx will be reloaded.
forego     | starting dockergen.1 on port 5000
forego     | starting nginx.1 on port 5100
dockergen.1 | 2018/10/26 04:37:24 Generated '/etc/nginx/conf.d/default.conf' from 3 containers
dockergen.1 | 2018/10/26 04:37:24 Running 'nginx -s reload'
dockergen.1 | 2018/10/26 04:37:24 Error running notify command: nginx -s reload, exit status 1
dockergen.1 | 2018/10/26 04:37:24 Watching docker events
fabioespinosa commented 5 years ago

If any of you managed to solve the issue, can you please write how?

buchdag commented 5 years ago

Could you try again ? #491 should make running this container on ECS way easier.

buchdag commented 5 years ago

Well, seems like docker-gen have an issue with the latest ECS AMI and is unable to correctly generate nginx configuration on this platform.

thecritic commented 5 years ago

Any progress on this?

buchdag commented 5 years ago

@thecritic no progress. As mentioned above this is blocked by a nginx-proxy / docker-gen bug specific to AWS ECS.

Hugozys commented 5 years ago

Is there any progress on this issue? I'm having the same issue while trying to deploy nginx letsencrypt-companion and my web app onto elastic beanstalk.

Hugozys commented 5 years ago

Here's my docker config and log info: dockerrun.aws.json:

{
  "AWSEBDockerrunVersion": "2",
  "volumes": [
    {
      "name": "vhost",
      "sourcePath": "/etc/nginx/vhost.d"
    },
    {
      "name": "html",
      "sourcePath": "/usr/share/nginx/html"
    },
    {
      "name": "certs",
      "sourcePath": "/var/app/current/certs"
    },
    {
      "name": "sock",
      "host":{
        "sourcePath": "/var/run/docker.sock"
      }
    }
  ],
  "containerDefinitions": [
    {
      "name": "nginx-proxy",
      "image": "jwilder/nginx-proxy",
      "essential": true,
      "memory": 128,
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80
        },
        {
          "hostPort": 443,
          "containerPort": 443
        }
      ],
      "links": [
        "web-app"
      ],
      "mountPoints": [
        {
          "sourceVolume": "vhost",
          "containerPath": "/etc/nginx/vhost.d"
        },
        {
          "sourceVolume": "certs",
          "containerPath": "/etc/nginx/certs",
          "readOnly": true
        },
        {
          "sourceVolume": "sock",
          "containerPath": "/tmp/docker.sock",
          "readOnly": true
        },
        {
          "sourceVolume": "awseb-logs-nginx-proxy",
          "containerPath": "/var/log/nginx"
        },
        {
          "sourceVolume": "html",
          "containerPath":"/usr/share/nginx/html"
        }
      ]
    },
    {
      "name": "nginx-proxy-le",
      "image": "jrcs/letsencrypt-nginx-proxy-companion",
      "essential": true,
      "memory": 128,
      "volumesFrom": [
        {
          "sourceContainer": "nginx-proxy"
        }
      ],
      "mountPoints": [
        {
          "sourceVolume": "certs",
          "containerPath": "/etc/nginx/certs"
        },
        {
          "sourceVolume": "sock",
          "containerPath": "/var/run/docker.sock",
          "readOnly": true
        }
      ]
    },
    {
      "name": "web-app",
      "image": "hugozzys/webapp:latest",
      "essential": true,
      "portMappings": [
        {
          "hostPort": 8080,
          "containerPort": 8080
        }
      ],
      "memory": 128,
      "environment": [
        {
          "name": "VIRTUAL_HOST",
          "value":"random.xxxx.com"
        },
        {
          "name": "LETSENCRYPT_HOST",
          "value":"random.xxxx.com"
        },
        {
          "name": "LETSENCRYPT_EMAIL",
          "value":"xxxx@example.com"
        },
        {
          "name": "VIRTUAL_PORT",
          "value": "8080" 
        }
      ]
    }
  ]
}

nginx-proxy log:

 WARNING: /etc/nginx/dhparam/dhparam.pem was not found. A pre-generated dhparam.pem will be used for now while a new one
is being generated in the background.  Once the new dhparam.pem is in place, nginx will be reloaded.
forego     | starting dockergen.1 on port 5000
forego     | starting nginx.1 on port 5100
dockergen.1 | 2019/09/14 22:04:15 Generated '/etc/nginx/conf.d/default.conf' from 3 containers
dockergen.1 | 2019/09/14 22:04:15 Running 'nginx -s reload'
dockergen.1 | 2019/09/14 22:04:15 Error running notify command: nginx -s reload, exit status 1
dockergen.1 | 2019/09/14 22:04:15 Watching docker events
dockergen.1 | 2019/09/14 22:04:15 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nginx: [emerg] no servers are inside upstream in /etc/nginx/conf.d/default.conf:58
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
dhparam generation complete, reloading nginx
dockergen.1 | 2019/09/14 22:04:20 Received event start for container 1498c878137d
dockergen.1 | 2019/09/14 22:04:20 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'

letsencrypt-companion log:

...........................++++
......++++
writing new private key to '/etc/nginx/certs/default.key.new'
-----
Info: a default key and certificate have been created at /etc/nginx/certs/default.key and /etc/nginx/certs/default.crt.
Info: Creating Diffie-Hellman group in the background.
A pre-generated Diffie-Hellman group will be used for now while the new one
is being created.
Generating DH parameters, 2048 bit long safe prime, generator 2
Reloading nginx proxy (ecs-awseb-BlogAdminStaging-env-1-wvae7y5qub-20-nginx-proxy-82bde6e6a69dc3c42400)...
2019/09/14 22:04:22 Generated '/etc/nginx/conf.d/default.conf' from 4 containers
2019/09/14 22:04:23 Generated '/app/letsencrypt_service_data' from 4 containers
2019/09/14 22:04:23 Running '/app/signal_le_service'
2019/09/14 22:04:23 Watching docker events
2019/09/14 22:04:23 Contents of /app/letsencrypt_service_data did not change. Skipping notification '/app/signal_le_service'
/etc/nginx/certs/random.xxxx.com /app
Reloading nginx proxy (ecs-awseb-BlogAdminStaging-env-1-wvae7y5qub-20-nginx-proxy-82bde6e6a69dc3c42400)...
2019/09/14 22:04:24 Generated '/etc/nginx/conf.d/default.conf' from 4 containers
Creating/renewal random.xxxx.com certificates... (random.xxxx.com)
2019-09-14 22:04:25,516:INFO:simp_le:1382: Generating new account key
2019-09-14 22:04:26,855:INFO:simp_le:1407: By using simp_le, you implicitly agree to the CA's terms of service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf
2019-09-14 22:04:27,293:INFO:simp_le:1479: Generating new certificate private key
2019-09-14 22:04:29,083:INFO:simp_le:360: Saving account_key.json
2019-09-14 22:04:29,085:INFO:simp_le:360: Saving key.pem
2019-09-14 22:04:29,086:INFO:simp_le:360: Saving chain.pem
2019-09-14 22:04:29,088:INFO:simp_le:360: Saving fullchain.pem
2019-09-14 22:04:29,089:INFO:simp_le:360: Saving cert.pem
/app
Warning: /etc/nginx/certs/accounts/acme-v01.api.letsencrypt.org/directory/default.json does not exist. Skipping ownership and permissions check.
Warning: /etc/nginx/certs/accounts/acme-v01.api.letsencrypt.org/directory does not exist. Skipping ownership and permissions check.
Warning: /etc/nginx/certs/accounts/acme-v01.api.letsencrypt.org does not exist. Skipping ownership and permissions check.
Warning: /etc/nginx/certs/accounts does not exist. Skipping ownership and permissions check.
Reloading nginx proxy (ecs-awseb-BlogAdminStaging-env-1-wvae7y5qub-20-nginx-proxy-82bde6e6a69dc3c42400)...
2019/09/14 22:04:29 Generated '/etc/nginx/conf.d/default.conf' from 4 containers
Sleep for 3600s
This is going to take a long time
Info: Diffie-Hellman group creation complete, reloading nginx.
Reloading nginx proxy (ecs-awseb-BlogAdminStaging-env-1-wvae7y5qub-20-nginx-proxy-82bde6e6a69dc3c42400)...
2019/09/14 22:04:50 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification ''
milenazuccarelli commented 4 years ago

Was anyone able to solve this? Experiencing this same issue, one year later

buchdag commented 4 years ago

@Hugozys @milenazuccarelli unfortunately there isn't anything we can do from this project perspective to solve this, as explained before the issues lies between docker-gen and the newest ECS AMI.

Spawnrad commented 4 years ago

@Feelthewind @Hugozys have you found a solution. I have exactly the same issue. what do you use now ?

thismatters commented 4 years ago

I have a little bit of a workaround in place, and it seems to work (at first glance). Essentially it involves getting the /etc/nginx/conf.d/default.conf which is autogenerated by nginx-proxy container and manually authoring the upstream section(s) to accurately reference whatever webapp(s) you are hosting. Prior to doing this is is advantageous to add "links" from nginx-proxy to your app(s) in Dockerrun.aws.json so that the docker network is set up between the two containers and an alias for the webapp is generated within the proxy. Then, I ensure that this default.conf is used by the nginx-proxy by setting /etc/nginx/conf.d/ as a volume on the proxy. Finally, I use an .ebextensions script to author the default.conf into place.

Dockerrun.aws.json

...
  "volumes": [
    {
      "name": "home-ec2-user-certs",
      "host": {
        "sourcePath": "/home/ec2-user/certs"
      }
    },
    {
      "name": "etc-nginx-vhost-d",
      "host": {
        "sourcePath": "/etc/nginx/vhost.d"
      }
    },
    {
      "name": "usr-share-nginx-html",
      "host": {
        "sourcePath": "/usr/share/nginx/html"
      }
    },
    {
      "name": "var-run-docker-sock",
      "host": {
        "sourcePath": "/var/run/docker.sock"
      }
    },
    {
      "name": "nginx-config-overwrite",
      "host": {
        "sourcePath": "/etc/nginx/conf.d/"
      }
    }
  ],
"containerDefinitions": [
    {
      "name": "nginx-proxy",
      "image": "jwilder/nginx-proxy",
      "essential": true,
      "memoryReservation": 128,
      "dockerLabels": {
        "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy": "true"
      },
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 80
        },
        {
          "containerPort": 443,
          "hostPort": 443
        }
      ],
      "links": [
        "prod-app"
      ],
      "mountPoints": [
        {
          "sourceVolume": "home-ec2-user-certs",
          "containerPath": "/etc/nginx/certs",
          "readOnly": true
        },
        {
          "sourceVolume": "etc-nginx-vhost-d",
          "containerPath": "/etc/nginx/vhost.d"
        },
        {
          "sourceVolume": "usr-share-nginx-html",
          "containerPath": "/usr/share/nginx/html"
        },
        {
          "sourceVolume": "var-run-docker-sock",
          "containerPath": "/tmp/docker.sock",
          "readOnly": true
        },
        {
          "sourceVolume": "nginx-config-overwrite",
          "containerPath": "/etc/nginx/conf.d"
        }
      ]
    },
...

/etc/nginx/conf.d/default.conf

...
upstream yyy.xxxxx.com {
  # I'm filling this part in manually!!
  server prod-app:8000;
  keepalive 256;
}
...

.ebextensions/05_nginx_config.config

files:
  "/etc/nginx/conf.d/default.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      map $http_x_forwarded_proto $proxy_x_forwarded_proto {
        default $http_x_forwarded_proto;
        ''      $scheme;
      }
      map $http_x_forwarded_port $proxy_x_forwarded_port {
        default $http_x_forwarded_port;
        ''      $server_port;
      }
      map $http_upgrade $proxy_connection {
        default upgrade;
        '' close;
       ...
thismatters commented 4 years ago

The workaround I mentioned before was not very reliable so I sought to correct the problem more permanently.

Use my fork of nginx-proxy in your deployment (until my pull request is accepted) to fix this:

Dockerrun.aws.json

...
"containerDefinitions": [
    {
      "name": "nginx-proxy",
      "image": "thismatters/nginx-proxy",    <--- that is the change!
      "essential": true,
      "memoryReservation": 128,
      "dockerLabels": {
        "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy": "true"
      },
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 80
        },
        {
          "containerPort": 443,
          "hostPort": 443
        }
      ],
      "links": [
        "prod-app"
      ],
      "environment": [{
        "name": "NGINX_PROXY_CONTAINER",
        "value": "true"
      }],
      "mountPoints": [
        {
          "sourceVolume": "home-ec2-user-certs",
          "containerPath": "/etc/nginx/certs",
          "readOnly": true
        },
        {
          "sourceVolume": "etc-nginx-vhost-d",
          "containerPath": "/etc/nginx/vhost.d"
        },
        {
          "sourceVolume": "usr-share-nginx-html",
          "containerPath": "/usr/share/nginx/html"
        },
        {
          "sourceVolume": "var-run-docker-sock",
          "containerPath": "/tmp/docker.sock",
          "readOnly": true
        }
      ]
    },
    {
      "name": "nginx-proxy-letsencrypt",
      "image": "jrcs/letsencrypt-nginx-proxy-companion",
      "essential": true,
      "memoryReservation": 128,
      "volumesFrom": [
        {
          "sourceContainer": "nginx-proxy"
        }
      ],
      "mountPoints": [
        {
          "sourceVolume": "home-ec2-user-certs",
          "containerPath": "/etc/nginx/certs"
        },
        {
          "sourceVolume": "var-run-docker-sock",
          "containerPath": "/var/run/docker.sock",
          "readOnly": true
        }
      ]
    },
...
buchdag commented 3 years ago

While working on GitHub Actions I think I found (and patched) the issue in docker-gen that prevent the whole stack from working on Amazon ECS.

I'm trying to reach @jwilder to discuss docker-gen maintenance.

In the meantime, if anyone is interested in testing if this patch actually fixes nginx-proxy / docker-gen on Amazon ECS, I can provide Example Dockerfile that will build patched images.