problem running reproxy in AWS ECS on Fargate

[avowkind]

reproxy maps var/run/docker.sock into its container using a volume so that it can query the settings of the other containers. I can use docker context ecs and docker compose to bring up a docker composer file as a cluster on AWS ECS on Fargate.
I get the following error in the reproxy log 2021/06/17 13:55:42.903 [ERROR] failed to fetch running containers: can't list containers: failed connection to docker socket: Get "http://localhost/v1.22/containers/json": dial unix /var/run/docker.sock: connect: no such file or directory And this is because the line :

 volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

is not allowed. We cannot bind host path /var/run/docker.sock as a mount point.

it is discussed here: https://stackoverflow.com/questions/53759932/aws-fargate-volumes

This may be well beyond the scope of reproxy.

[umputun]

I'm not sure how else reproxy can retrieve the info about other containers without access to docker's socket. In order to discover what containers will be handled it needs the knowledge which is gathered by using docker API from within reproxy.

I read the SO link but the proposed workaround doesn't seem to be applicable for this case.

How do you even know from one container running on the fargate list/ips/names of other containers? I mean if container A needs to access container B directly, how this can be done? Are they on the same docker network and directly available by names? I'm not familiar enough with this magic and have no clue how all of this done on Fargate.

[avowkind]

I agree it would certainly require a different approach which is beyond the scope of reproxy.

The containers in the cluster are all on the same subnet and ECS maps the final IPs of the containers onto domain names that match the container names. Hence using nginx I can have a config like

location /mms/ {
  proxy_pass http://cvs-mms:14000/;
}

Where the domain matches the container name in the docker compose file.

services:

Computer Vision Multi Model Server for MXNET

cvs-mms:

One might build something like reproxy using the AWS describeTasks API.

The other approach would be to allow the reproxy to be full configured from its own labels or env vars. With a list of the other containers and their domain names/ports

  reproxy.pass.1.route: "(/mongoa/)"
  reproxy.pass.1.dest: “http://cvs-mongo/$$1”
  reproxy.pass.2.route: "^/(.*)"
  reproxy.pass.2.dest: "http://csv-frontend/$$1”

This bypasses the IP discovery via the docker device and just relies on the dns.

On 17/06/2021, at 14:27, Umputun @.***> wrote:

I'm not sure how else reproxy can retrieve the info about other containers without access to docker's socket. In order to discover what containers will be handled it needs the knowledge which is gathered by using docker API from within reproxy.

I read the SO link but the proposed workaround doesn't seem to be applicable for this case.

How do you even know from one container running on the fargate list/ips/names of other containers? I mean if container A needs to access container B directly, how this can be done? Are they on the same docker network and directly available by names? I'm not familiar enough with this magic and have no clue how all of this done on Fargate.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/umputun/reproxy/issues/92#issuecomment-862866851, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMFYFK6GBTG35XATAV6NXDTTFMQBANCNFSM462UZ3MA.

[umputun]

I think, you can do smth similar to the provided nginx config in reproxy already with either static or file provider or, if you have consul involved, with the consul-catalog provider.

For the first suggestion - this probably can be a separate provider, but from first sight, it seems to be a little bit too exotic to be included in the core code. From another hand, if someone can develop such a provider and it doesn't involve a lot of code and a lot of dependencies I will consider it. Alternatively, it can be implemented as a part of an external plugin. The current plugin protocol doesn't support custom providers yet but this is something I'm planning to add.

For the second suggestion, unless I missed something fundamental, I can't see how this any different from the current static provider. In case you not familiar with this one it is as simple as this:

reproxy:
    image: umputun/reproxy:latest
    restart: always
    ports:
      - "80:8080"
      - "443:8443"
    environment:
      - STATIC_ENABLED=true
      - STATIC_RULES=
          /mongoa/(.*),http://cvs-mongo/$$1,;
         ^/(.*),http://csv-frontend/$$1,;

if you don't like putting rules this way, you can check the file provider which allows similar rules in yaml you can mount to reproxy container if fargate allows such mapping

[avowkind]

Thanks for being so engaged. I did miss the static provider and it may do the job. I’ll give it a go.

On 19/06/2021, at 17:04, Umputun @.***> wrote:

I think, you can do smth similar to the provided nginx config in reproxy already with either static or file provider or, if you have consul involved, with the consul-catalog provider.

For the first suggestion - this probably can be a separate provider, but from first sight, it seems to be a little bit too exotic to be included in the core code. From another hand, if someone can develop such a provider and it doesn't involve a lot of code and a lot of dependencies I will consider it. Alternatively, it can be implemented as a part of an external plugin. The current plugin protocol doesn't support custom providers yet but this is something I'm planning to add.

For the second suggestion, unless I missed something fundamental, I can't see how this any different from the current static provider https://github.com/umputun/reproxy#static-provider. In case you not familiar with this one it is as simple as this:

reproxy: image: umputun/reproxy:latest restart: always ports:

• "80:8080"
• "443:8443" environment:
• STATIC_ENABLED=true
• STATIC_RULES= /mongoa/(.),http://cvs-mongo/$$1,; ^/(.),http://csv-frontend/$$1,; if you don't like putting rules this way, you can check the file provider https://github.com/umputun/reproxy/tree/master/examples/file which allows similar rules in yaml you can mount to reproxy container if fargate allows such mapping

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/umputun/reproxy/issues/92#issuecomment-864356456, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMFYFIW243QNCN4HAILSFDTTQQMBANCNFSM462UZ3MA.