criblio / appscope

Gain observability into any Linux command or application with no code modification
https://appscope.dev
Apache License 2.0
267 stars 33 forks source link

BUG: scope start does not appear to locate the host namespace with docker-in-docker #1197

Closed iapaddler closed 1 year ago

iapaddler commented 1 year ago

stept to repro:

docker pull docker:dind
docker run -it -p 9420:9420 --hostname dind --name dind --privileged -d --rm docker:dind
docker exec -it dind /bin/sh
   docker pull cribl/cribl:next-nightly
   docker pull ubuntu:18.04
   docker run --privileged -d -e CRIBL_EDGE=1 -p 9420:9420 -v /var/run/appscope:/var/run/appscope -v /var/run/docker.sock:/var/run/docker.sock -v /:/hostfs:ro --restart unless-stopped --name cribl-edge cribl/cribl:next-nightly
   docker run -d -v /var/run/appscope/:/var/run/appscope/ --privileged --rm ubuntu:18.04 tail -f /dev/null
michalbiesek commented 1 year ago

Here is the overview of the model presented in the example:

stateDiagram-v2
    Host: Host 
    DindContainer: Docker in docker container
    EdgeContainer: Edge container
    OtherContainer: Ubuntu container
    state Host {
    DindContainer
    }
    state DindContainer {
    EdgeContainer
    OtherContainer
    }

Notes:

  1. In Docker in docker container layer we do not have cron.d running Partially handled by 4e0e249
  2. To escape to the parent namespace:
    • Edge container -> Docker in docker container
    • Docker in docker container -> Host

We need a file descriptor that allows us to switch the mount namespace to the parent. Handled by proper command line described in https://github.com/criblio/appscope/issues/1197#issuecomment-1315411724

  1. Ubuntu container and Edge container are not visible from Host namespace (nested endpoint) - we can check the process itself but we need the container PID to run setup command

The way of identification of container PID is changed in #1195

  1. We need to add the possibility to attach from host into process which runs in nested container Fixed by 90e0fd7
michalbiesek commented 1 year ago

Following command allows us to run scope start in the context of the host:

docker pull docker:dind
docker run -it -p 9420:9420 --hostname dind --name dind --privileged -d --rm -v /:/hostfs:ro  docker:dind
docker exec -it dind /bin/sh
   docker pull cribl/cribl:next-nightly
   docker pull ubuntu:18.04
   docker run --privileged -d -e CRIBL_EDGE=1 -p 9420:9420 -v /var/run/appscope:/var/run/appscope -v /var/run/docker.sock:/var/run/docker.sock -v /hostfs:/hostfs:ro --restart unless-stopped --name cribl-edge cribl/cribl:next-nightly
   docker run -d -v /var/run/appscope/:/var/run/appscope/ --privileged --rm ubuntu:18.04 tail -f /dev/null

We need to propagate procfs from host to Edge via, two: mount points