dreamcat4 / docker-images

Dreamcat4's Docker Images (Trusted Builds)
MIT License
160 stars 66 forks source link

How do you use --direct-phys? #55

Closed FallingSnow closed 2 years ago

FallingSnow commented 2 years ago

I'm trying to pass an SR-IOV interface to a test container but adding --direct-phys seems to be parsed incorrectly.

myapp:
  image: alpine
  command: sleep inf
  environment:
    - pipework_cmd=--direct-phys eth5 @CONTAINER_NAME@ udhcpc
  net: none

pipework:
  image: dreamcat4/pipework:latest
  restart: always
  volumes:
    - /var/run/docker.sock:/docker.sock
  privileged: true
  pid: host # THIS REQUIRES COMPOSE v1.3.0 & DOCKER v1.6.0
  net: host
  environment:
    #- debug=true
    - run_mode=batch,daemon
    - host_routes=true
    - route_add_delay=1

Output from docker-compose up: (Note the pipework_1 | /entrypoint.sh: line 190: set: --: invalid option line)

Recreating test_pipework_1 ... done
Starting test_myapp_1      ... done
Attaching to test_pipework_1, test_myapp_1
pipework_1  | ln: failed to create symbolic link '/sys/fs/cgroup/openrc/name=openrc': Operation not permitted
pipework_1  | /entrypoint.sh: line 190: set: --: invalid option
pipework_1  | set: usage: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
pipework_1  | Device "eth1" does not exist.
pipework_1  | event_line=2022-05-08T20:33:35.700228242Z container start f86c1f1a102f483f802ffa0db3f9bee29be14fc91b5385380789ac955fad216d (com.docker.compose.config-hash=4c7d33be91c4b5e95432c49c4faa425f9378546b38a366f1704e594a9ccbd773, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=test, com.docker.compose.project.config_files=docker-compose.yml, com.docker.compose.project.working_dir=/opt/docker/test, com.docker.compose.service=pipework, com.docker.compose.version=1.29.2, image=dreamcat4/pipework:latest, name=test_pipework_1)
pipework_1  | event_line=2022-05-08T20:33:35.787127797Z container start f6dfcc6f7ed3eee8631dcb7cbd382962445120d901ac8ac16e485834122a472c (com.docker.compose.config-hash=81a7c0e72c42a72d5316b36c8961531b614684662726684d402794969a89de26, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=test, com.docker.compose.project.config_files=docker-compose.yml, com.docker.compose.project.working_dir=/opt/docker/test, com.docker.compose.service=myapp, com.docker.compose.version=1.29.2, image=alpine, name=test_myapp_1)
pipework_1  | /entrypoint.sh: line 190: set: --: invalid option
pipework_1  | set: usage: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
pipework_1  | Device "eth1" does not exist.
dreamcat4 commented 2 years ago

never used that feature myself

thing is this tool is now deprecated. you can find here a network driver instead for some mellanox card or other. (dont know if it can also work for your specific hardware, you will need to check)

https://github.com/Mellanox/docker-sriov-plugin

FallingSnow commented 2 years ago

I've actually already tried using rdma/sriov-plugin. The issue I was running into was I couldn't find a way to get it to use DHCP. It provides a way to statically assign subnet (--subnet=) and gateway/ip but I don't think it supports doing it through DHCP.

FallingSnow commented 2 years ago

Do you know of any other way to pass a network device to a container?

dreamcat4 commented 2 years ago

It provides a way to statically assign subnet (--subnet=) and gateway/ip but I don't think it supports doing it through DHCP.

ok then maybe the better way forward is to then raise the matter as a new issue over on that other repo, where it can be fixes? (but i see no such issue there yet)

because like i said: this tool is now deprecated. it has been for very long time now. it is just never coming back at this point. if you need support then use the newer tool which is currently being supported :)

FallingSnow commented 2 years ago

I understand.

Seems as though that tool is unmaintained at this point as everything moved to k8s.

Looks like I'll need to find another solution. Thank you for your time!

dreamcat4 commented 2 years ago

wait a minute!

because if you look on the network graph view of the mellanox repo

https://github.com/Mellanox/docker-sriov-plugin/network

then you can see there has been this 1 guy who has kept on making many more commits, up until january 2022

so maybe if you try his fork of it, or ask that guy for assistance. might be a further help to your predicament. he does not have issues enabled on his fork however

https://github.com/FoxDenHome/docker-sriov-plugin

FallingSnow commented 2 years ago

Oh wow, didn't see that. I'll look into his fork and see if it makes a difference. Thank you!