Running container (docker) image

[aviramha]

Currently, users can use Docker + mirrord by either ways:

1. Create a "dev" Dockerfile that contains the app image and changes the entrypoint/cmd to run the original command using mirrord (while mounting/copying k8s credentials so mirrord would work)
2. Work in a dev container and just run mirrord from there (via VSCode/IntelliJ or CLI)

It might make sense to add a more sugary way of doing it - for example mirrord runc <image_name>. This would actually run the container image using Docker or any Docker compatible API while doing all the necessary work to make it work out of the box.

[t4lz]

Wouldn't mirrord exec -f .miorrord/mirrord.json docker -- run my-image work?

[aviramha]

Wouldn't mirrord exec -f .miorrord/mirrord.json docker -- run my-image work?

No. in macOS docker doesn't even run it on your machine, it runs it in a VM so you wouldn't even have right binaries nor accessible for the container.

In general, the docker CLI doesn't execute the process - it communicates with the daemon that does the execution.

[t4lz]

Got it, thanks. With unix sockets it could communicate with the docker engine on the cluster and run it there, right?. Maybe that's useful too?

[aviramha]

but that would occur on the remote pod docker engine if existing which isn't what they'd imagine to happen..

[sharkymcdongles]

I tried to run this via this example however the cluster DNS doesn't resolve:

mirrord exec -f .miorrord/mirrord.json docker -- run my-image

[aviramha]

I tried to run this via this example however the cluster DNS doesn't resolve:

mirrord exec -f .miorrord/mirrord.json docker -- run my-image

Hi! This isn't supported at the moment as discussed in this issue. What you can do is run mirrord inside the container itself.

[uneet77]

Hi @aviramha , I am trying to implement step no. 1 as you have suggested and its little bit messy to implement. Can you suggest imporvements please?

1. Install mirrord CLI in the image
2. Install Docker in Docker Image as CLI needs Docker or containerd runtime to operate.
3. Mount kubectl config using -v "~/.kube/config:/kube/config". I want to know is there a better way to do this?

[aviramha]

Hi @uneet77 - why do you think you need step 2? This seems redundant

[uneet77]

I get it, step 2 is not needed. Thanks Do you think mounting the kube config is the only requirement for the mirrord CLI to work? Or do we need to pass any additional k8s information to image?

[aviramha]

I get it, step 2 is not needed. Thanks Do you think mounting the kube config is the only requirement for the mirrord CLI to work? Or do we need to pass any additional k8s information to image?

The k8s config might point to binaries (aws, az, gcloud) to get auth from, so those might be needed but need to see the kubeconfig info to understand better.

In general mounting the kube config shuold be enough - you should get to a point where kubectl get pods works (then you can get rid of the kubectl binary)

[danielloader]

To extend this conversation, I've got mirrord working happily inside a docker container, but I'd like to expose the application that's tunnelled to the rest of the local network namespace. I appreciate this is probably not a considered usecase but thought I'd ask!

Inspired by https://mirrord.dev/docs/using-mirrord/web-browsing/

docker-compose.yml for a test reproduction environment (tested it against EKS, hence AWS specificity in the example):

version: '3.8'

services:
  proxy:
    build:
      dockerfile_inline: |
        FROM debian:12-slim
        RUN <<EOF
          apt-get -y update
          apt-get install --no-install-recommends -y microsocks curl awscli ca-certificates lsof
          rm -rf /var/lib/apt/lists/*
        EOF
        RUN curl -fsSL https://raw.githubusercontent.com/metalbear-co/mirrord/main/scripts/install.sh | bash
    entrypoint: ["/usr/local/bin/mirrord"]
    command:
      - exec 
      - "--"
      - microsocks
      - -b
      - "0.0.0.0"
    healthcheck:
      test: curl --fail --socks5 127.0.0.1:1080 http://checkip.amazonaws.com || exit 1
      interval: 10s
      timeout: 2s
      retries: 3
      start_period: 5s
    ports:
      - 1080:1080
    restart: on-failure
    volumes:
      - $HOME/.kube/config:/root/.kube/config:ro
      - $HOME/.aws/credentials:/root/.aws/credentials:ro
      - $HOME/.aws/config:/root/.aws/config:ro
  test:
    image: quay.io/curl/curl:latest
    depends_on:
      - proxy
    command:
      - --progress-bar
      - --retry-connrefused
      - --retry 
      - "1000"
      - --retry-delay 
      - "10"
      - --retry-max-time 
      - "40"
      - --socks5-hostname
      - "proxy:1080"
      - http://checkip.amazonaws.com
    restart: "no"

Result:

• Healthcheck curl inside the container works.
• Curl from another container in compose doesn't work.
• Curl from host OS via the exposed port into the proxy container doesn't work.

My assumption is that the bind address being set on microsocks isn't being matched by mirrord wrapping it.

lsof seems to confirm this.

With mirrord running and wrapping microsocks:

docker compose exec proxy lsof -i TCP
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
microsock   1 root    3u  IPv4 852481      0t0  TCP localhost:40756->localhost:43671 (ESTABLISHED)
microsock   1 root    4u  IPv4 852481      0t0  TCP localhost:40756->localhost:43671 (ESTABLISHED)
microsock   1 root    5u  IPv4 852491      0t0  TCP localhost:1080 (LISTEN)
mirrord    87 root    9u  IPv4 849887      0t0  TCP localhost:43671 (LISTEN)
mirrord    87 root   10u  IPv4 854296      0t0  TCP 7bdbd7eca98d:41226->ec2-18-175-10-198.eu-west-2.compute.amazonaws.com:443 (ESTABLISHED)
mirrord    87 root   11u  IPv4 856213      0t0  TCP 7bdbd7eca98d:41240->ec2-18-175-10-198.eu-west-2.compute.amazonaws.com:443 (ESTABLISHED)
mirrord    87 root   12u  IPv4 855073      0t0  TCP localhost:43671->localhost:40756 (ESTABLISHED)

Without (microsocks as the entrypoint instead of mirrord):

docker compose exec proxy lsof -i TCP
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
microsock   1 root    3u  IPv4 869928      0t0  TCP *:1080 (LISTEN)

So my question - can you configure mirrord to expose the process on 0.0.0.0 so I can just wrap a socks5 proxy using mirrord in docker for easier use in our local docker-compose stacks?

Would be useful for both proxying the host browser via the exposed 1080 port, and access from other containers in the compose network stack.

Edit: A further extended example if you wanted http as well as socks5 proxies:

version: '3.8'

services:
  socks5-proxy:
    build:
      dockerfile_inline: |
        FROM debian:12-slim
        RUN <<EOF
          apt-get -y update
          apt-get install --no-install-recommends -y microsocks curl awscli ca-certificates lsof
          rm -rf /var/lib/apt/lists/*
        EOF
        RUN curl -fsSL https://raw.githubusercontent.com/metalbear-co/mirrord/main/scripts/install.sh | bash
    entrypoint: ["/usr/local/bin/mirrord"]
    command:
      - exec 
      - "--"
      - microsocks
      - -b 
      - "0.0.0.0"
    healthcheck:
      test: curl --fail --socks5 127.0.0.1:1080 http://checkip.amazonaws.com || exit 1
      interval: 10s
      timeout: 2s
      retries: 3
      start_period: 5s
    ports:
      - 1080:1080
    restart: on-failure
    volumes:
      - $HOME/.kube/config:/root/.kube/config:ro
      - $HOME/.aws/credentials:/root/.aws/credentials:ro
      - $HOME/.aws/config:/root/.aws/config:ro
  http-proxy:
    build:
      dockerfile_inline: |
        FROM debian:12-slim
        RUN <<EOF
          apt-get -y update
          apt-get install --no-install-recommends -y tinyproxy curl awscli ca-certificates lsof
          rm -rf /var/lib/apt/lists/*
        EOF
        RUN curl -fsSL https://raw.githubusercontent.com/metalbear-co/mirrord/main/scripts/install.sh | bash
        COPY <<EOF /etc/tinyproxy/tinyproxy.conf
        Port 8888
        Listen 0.0.0.0
        Timeout 600
        Allow 127.0.0.1
        Allow 0.0.0.0
        EOF
    entrypoint: ["/usr/local/bin/mirrord"]
    command:
      - exec 
      - "--"
      - tinyproxy
      - -d 
    healthcheck:
      test: curl --fail --proxy http://127.0.0.1:8888 http://checkip.amazonaws.com || exit 1
      interval: 10s
      timeout: 2s
      retries: 3
      start_period: 5s
    ports:
      - 8888:8888
    restart: on-failure
    volumes:
      - $HOME/.kube/config:/root/.kube/config:ro
      - $HOME/.aws/credentials:/root/.aws/credentials:ro
      - $HOME/.aws/config:/root/.aws/config:ro
  test-socks5-proxy:
    image: quay.io/curl/curl:latest
    depends_on:
      - socks5-proxy
    command:
      - --progress-bar
      - --retry-connrefused
      - --retry 
      - "10"
      - --retry-delay 
      - "10"
      - --retry-max-time 
      - "40"
      - --socks5-hostname
      - "socks5-proxy:1080"
      - http://checkip.amazonaws.com
    restart: "no"
  test-http-proxy:
    image: quay.io/curl/curl:latest
    depends_on:
      - http-proxy
    environment:
      - http_proxy=http://http-proxy:8888
      - https_proxy=http://http-proxy:8888
    command:
      - --progress-bar
      - --retry-connrefused
      - --retry 
      - "10"
      - --retry-delay 
      - "10"
      - --retry-max-time 
      - "40"
      - http://checkip.amazonaws.com
    restart: "no"

[aviramha]

hi @danielloader , appreciate the elaborate explanation. This actually seems like an issue on its own, and I'd love for us to prioritize it and fix it. Can you create an issue reporting that we modify the local listen address?