Can't ping docker service with --network=none

[thomasyu888]

For the submissions, we are asking participants to host docker services. The example is:

docker run -p 8080:8080 nlpsandbox/date-annotator-example:latest

By doing this, we are able to view this at localhost:8080/api/v1/ui. Unfortunately, due to security concerns, we have to do

docker run -p 8080:8080 --network=none nlpsandbox/date-annotator-example:latest

By doing this, we are unable to set ping the server any more. I also tried:

docker run -p 8080:8080 --network=host nlpsandbox/date-annotator-example:latest

But this also doesn't work. I will have to investigate more, but this is a huge blocker.

[thomasyu888]

My guess is that we will have to look into docker networking, and block all outbound traffic but accept inbound traffic from a specific ip. Not quite sure how to achieve this yet, but I am investigating.

[tschaffter]

We probably won't address this issue using docker configuration alone, rather we need to configure the hosts to filter traffic. For the sake of the exercise and come up with a scalable solution, let's consider network with three hosts (EC2):

• Controller Host: runs the Synapse Workflow Orchestrator
• Data Host: runs the 2014 i2b2 data node
• App Host: runs the app (web service) that we need to evaluate

Specifications

• Controller Host
  • pull submission from Synapse
  • pull the Docker images of the Orchestrator stack (docker-compose) from DockerHub or Synapse
  • push evaluation results to Synapse
  • sends requests to the Data Host
  • sends requests to the App Host
• Data Host
  • pull the Docker images of the data node stack (docker-compose) from DockerHub or Synapse
  • respond to requests from the Controller Host
• App Host
  • pull Docker images from Synapse (private and public images) and from DockerHub (public images)
  • respond to requests from the Controller Host
• Other
  • Every other traffic is blocked, possibly with temporary exception to update host packages (apt-get update && apt-get upgrade)

We should seek the expertise of @ahayden to make sure that we configure this network right. I'll setup a call for the three of us next week.

@ahayden meanwhile if the above objective is clear, please let us know if you can give us any pointers.

[thomasyu888]

With the current implementation, the issue currently is that the controller host has to run on the same instance as the app host.

We would need to set up docker agents like we did for the DM challenge, where the instance that runs the app has limited connectivity.

[thomasyu888]

I found a none elegant way to do this:

docker run -p 8080:8080 --network=none nlpsandbox/date-annotator-example:latest

docker exec 706f59e557ed curl -X POST "http://0.0.0.0:8080/api/v1/dates" -H  "accept: application/json" -H  "Content-Type: application/json" -d "[{\"createdAt\":\"2020-11-03T23:51:01.359Z\",\"createdBy\":{\"email\":\"john.smith@example.com\",\"firstName\":\"John\",\"lastName\":\"Smith\",\"username\":\"John78\"},\"id\":0,\"updatedAt\":\"2020-11-03T23:51:01.359Z\",\"updatedBy\":{\"email\":\"john.smith@example.com\",\"firstName\":\"John\",\"lastName\":\"Smith\",\"username\":\"John78\"},\"text\":\"On 09-03-1999, Ms Chloe Price met with Dr Joe.\",\"type\":\"pathology\"}]"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   516  100   143  100   373  13000  33909 --:--:-- --:--:-- --:--:-- 46909
[{"format":"MM-DD-YYYY","length":10,"noteId":0,"start":3,"text":"09-03-1999"},{"format":"YYYY","length":4,"noteId":0,"start":9,"text":"1999"}]

docker exec 706f59e557ed curl https://www.google.com/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: www.google.com

The caveat is that I have to call the request post from inside the container

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.