search

egil / IoT-Edge-Device-in-Docker

A docker image that enables running the aziot-edge in a docker container.
https://hub.docker.com/r/egilhansen/iothub-edge-device
MIT License
0 stars 0 forks source link

Your not actually mounting the docker.sock #1

Open mmoles-Growlink opened 1 week ago

mmoles-Growlink commented 1 week ago

docker run -d --restart unless-stopped --privileged -it -v /var/run/docker.sock:/var/run/docker.sock -v /sys/fs/cgroup:/sys/fs/cgroup:rw -e connectionString='<IOT_EDGE_DEVICE_CONNECTION_STRING>' --hostname=edgedevice1 --name iot-edge-device egilhansen/iothub-edge-device:1.5.0-amd64 --dns 8.8.8.8 --log-driver "json-file" --log-opt "max-file=10" --log-opt "max-size=200k"

-v /var/run/docker.sock:/var/run/docker.sock This part is correct and this should mount the docker.sock, but in the docker-entrypoint.sh
exec /lib/systemd/systemd --log-level=info

When this fires off, it will reset /var/run/ and anything you had put in there will be removed. Docker then runs and creates a new docker.sock file so everything works.

The host and edge container dont see each others containers. Right now, when this edge container runs, it uses its nested docker instance causing it to pull images and create containers inside its own instance basically creating a massive layer where all your edge modules live. When you delete this edge container , you lose all the modules and images that it pulled.

If we can actually pass the docker.sock , then instead of creating the module containers under itself, it can create them as siblings instead of children. Images are shared with the host and both instances of docker see the same thing. If you update this container, all the other module containers will still persist.

I have tried moving the docker.sock into a different location and targeting that, but i get the issue from edge deamon 

root@edgedevice1:/# iotedge check --verbose

Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
√ aziot-identity-service package is up-to-date - OK
√ host time is close to reference time - OK
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
× read all preloaded certificates from the Certificates Service - Error
    could not load cert with ID "aziot-edged-trust-bundle"
    Caused by:
        parameter "id" has an invalid value
        caused by: not found
        caused by: could not load cert with ID "aziot-edged-trust-bundle"
                   Caused by:
                       parameter "id" has an invalid value
                       caused by: not found
√ read all preloaded key pairs from the Keys Service - OK
√ check all EST server URLs utilize HTTPS - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK

Connectivity checks (aziot-identity-service)
--------------------------------------------
√ host can connect to and perform TLS handshake with iothub AMQP port - OK
√ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with iothub MQTT port - OK

Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
√ configuration up-to-date with config.toml - OK
√ container engine is installed and functional - OK
× configuration has correct URIs for daemon mgmt endpoint - Error
    SocketError - SocketErrorCode (ConnectionRefused) : Connection refused /var/run/iotedge/mgmt.sock
    One or more errors occurred. (Got bad response: )
        caused by: SocketError - SocketErrorCode (ConnectionRefused) : Connection refused /var/run/iotedge/mgmt.sock
                   One or more errors occurred. (Got bad response: )
        caused by: docker returned exit status: 1, stderr = SocketError - SocketErrorCode (ConnectionRefused) : Connection refused /var/run/iotedge/mgmt.sock
                   One or more errors occurred. (Got bad response: )
√ aziot-edge package is up-to-date - OK
√ container time is close to host time - OK
‼ DNS server - Warning
    Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
    Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
    You can ignore this warning if you are setting DNS server per module in the Edge deployment.
        caused by: Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
                   Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
                   You can ignore this warning if you are setting DNS server per module in the Edge deployment.
‼ production readiness: logs policy - Warning
    Container engine is not configured to rotate module logs which may cause it run out of disk space.
    Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
    You can ignore this warning if you are setting log policy per module in the Edge deployment.
        caused by: Container engine is not configured to rotate module logs which may cause it run out of disk space.
                   Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
                   You can ignore this warning if you are setting log policy per module in the Edge deployment.
× production readiness: Edge Agent's storage directory is persisted on the host filesystem - Error
    Could not check current state of edgeAgent container
        caused by: Could not check current state of edgeAgent container
        caused by: docker returned exit status: 1, stderr = Error: No such object: edgeAgent
× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error
    Could not check current state of edgeHub container
        caused by: Could not check current state of edgeHub container
        caused by: docker returned exit status: 1, stderr = Error: No such object: edgeHub
√ Agent image is valid and can be pulled from upstream - OK
√ proxy settings are consistent in aziot-edged, aziot-identityd, moby daemon and config.toml - OK

Connectivity checks
-------------------
√ container on the default network can connect to upstream AMQP port - OK
√ container on the default network can connect to upstream HTTPS / WebSockets port - OK
√ container on the default network can connect to upstream MQTT port - OK
    skipping because of not required in this configuration
× container on the IoT Edge module network can connect to upstream AMQP port - Error
    Container on the azure-iot-edge network could not connect to my-iothub.azure-devices.net:5671
        caused by: Container on the azure-iot-edge network could not connect to my-iothub.azure-devices.net:5671
        caused by: docker returned exit status: 125, stderr = docker: Error response from daemon: network azure-iot-edge not found.
× container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error
    Container on the azure-iot-edge network could not connect to my-iothub.azure-devices.net:443
        caused by: Container on the azure-iot-edge network could not connect to my-iothub.azure-devices.net:443
        caused by: docker returned exit status: 125, stderr = docker: Error response from daemon: network azure-iot-edge not found.
√ container on the IoT Edge module network can connect to upstream MQTT port - OK
    skipping because of not required in this configuration
27 check(s) succeeded.
2 check(s) raised warnings.
6 check(s) raised errors.
2 check(s) were skipped due to errors from other checks.

root@edgedevice1:/# journalctl -u aziot-edged.service -f
Jun 27 16:47:05 edgedevice1 systemd[1]: Started Azure IoT Edge daemon.
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [INFO] - Starting Azure IoT Edge Daemon
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [INFO] - Version - 1.5.0
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [INFO] - Obtaining Edge device provisioning data...
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [INFO] - Device is myDevice on my-iothub.azure-devices.net
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [INFO] - Initializing module runtime...
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [INFO] - Using runtime network id azure-iot-edge
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [WARN] - container runtime error
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: Caused by:
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]:     0: error trying to connect: Permission denied (os error 13)
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]:     1: Permission denied (os error 13)
Jun 27 16:47:05 edgedevice1 aziot-edged[1825]: 2024-06-27T16:47:05Z [ERR!] - Failed to initialize module runtime: runtime operation error: initialize module runtime
Jun 27 16:47:05 edgedevice1 systemd[1]: aziot-edged.service: Main process exited, code=exited, status=1/FAILURE
Jun 27 16:47:05 edgedevice1 systemd[1]: aziot-edged.service: Failed with result 'exit-code'.
^C
egil commented 1 week ago

Admittedly I'm not an docker expert, so I have no idea how to improve things. If you come up with a solution I'll happily accept a pull request.

mmoles-Growlink commented 1 week ago

I'm lost as well. I believe this is a security issue, something like AppArmor maybe. I really don't know what I am doing with that kind of stuff. With docker I have learned that you can't always trust error messages to mean what they say.

In any case, your DinD is cool because even if it works in this nested fashion , it proves that version can run on that system.