search

Azure / iot-edge-v1

Azure IoT Edge
http://azure.github.io/iot-edge/
Other
525 stars 258 forks source link

V2 - HSM Init failure: 44 #642

Closed HenrikBach1 closed 6 years ago

HenrikBach1 commented 6 years ago

Hi,

I'm roughly following your guide https://docs.microsoft.com/en-us/azure/iot-edge/how-to-install-iot-edge-linux-arm, with exceptions on installation of the Docker and installation of the Debian packages and their dependencies, to install Azure IoT Edge runtime on a linux ARM32v7/armhf compatible device with Docker provided via Yocto/OpenEmbedded projects.

The docker service is functioning as expected on the device:

root@ddau3:~# docker run -it --rm ubuntu /bin/bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
d81ad7e8dedc: Pull complete
cc32849e56f0: Pull complete
ab1a7c883e79: Pull complete
7f11fb189b43: Pull complete
2fa5deb5232b: Pull complete
Digest: sha256:30e04ddada6eb09c12330c7df72cad1573916c7100168c34076808169ff6d805
Status: Downloaded newer image for ubuntu:latest
root@b7cd5a155e8b:/# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04 LTS"
root@b7cd5a155e8b:/# exit
root@ddau3:~#

I’ve transferred a simple, but functioning and working, copy of a config.yaml (from a Raspberry Pi 3/Raspbian Stretch 9 Lite, June release) to the folder of the device: /etc/iotedge/ and starts the Azure IoT Edge runtime:

root@ddau3:~# /etc/init.d/iotedge start
Starting iotedged : iotedge<6>2018-07-17T08:42:24Z [INFO] - Starting Azure IoT Edge Security Daemon
<6>2018-07-17T08:42:24Z [INFO] - Version - 1.0.0 (52ef77db24126bf473265fc09c53d35290a2dd6b)
<6>2018-07-17T08:42:24Z [INFO] - Using default configuration
<6>2018-07-17T08:42:24Z [INFO] - Using runtime network id azure-iot-edge
<6>2018-07-17T08:42:24Z [INFO] - Initializing the module runtime...
<6>2018-07-17T08:42:24Z [INFO] - Finished initializing the module runtime.
<6>2018-07-17T08:42:24Z [INFO] - Configuring /tmp/iotedge as the home directory.
<6>2018-07-17T08:42:24Z [INFO] - Configuring certificates...
<6>2018-07-17T08:42:24Z [INFO] - Transparent gateway certificates not found, operating in quick start mode...
<6>2018-07-17T08:42:24Z [INFO] - Finished configuring certificates.
<6>2018-07-17T08:42:24Z [INFO] - Initializing hsm...
<3>2018-07-17T08:42:24Z [ERR!] - An hsm error occurred.
<3>2018-07-17T08:42:24Z [ERR!] -        caused by: HSM failure
<3>2018-07-17T08:42:24Z [ERR!] -        caused by: HSM Init failure: 44
.
root@ddau3:~# docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 2
Server Version: 17.06.0-dev
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3addd840653146c90a254301d6c3a663c7fd6429
runc version: 9d6821d1b53908e249487741eccd567249ca1d99-dirty (expected: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4)
init version: N/A (expected: )
Kernel Version: 4.9.0-xilinx
Operating System: <unknown>
OSType: linux
Architecture: armv7l
CPUs: 2
Total Memory: 1005MiB
Name: ddau3
ID: LMFR:GOMG:5X3M:HK3K:KTF5:3I5R:6NEG:JH7C:ANVP:LK6K:WWKG:H4RR
Docker Root Dir: /mnt/data/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 localhost:5000
 127.0.0.0/8
Registry Mirrors:
 http://localhost:5000/
Live Restore Enabled: false

WARNING: No swap limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
root@ddau3:~#

Both devices are NOT configured as IoT Edge Gateway(s), so in this respect no resource contraints should give above cause.

Are someone (@dariuszparys, @varunpuranik ) able to tell me, what has happened in the Security Daemon/Manager or what is needs? At the moment, the device doesn’t have any TPM or HSM implemented.

Help is appreciated.

Thank you.

/Henrik

varunpuranik commented 6 years ago

Hi Henrik,

Did you perform all the steps in Install the IoT Edge Security Daemon as recommended?

It seems like the libiothsm-std has not been installed on the device correctly. Can you confirm if that has been installed correctly?

Thanks, Varun

darobs commented 6 years ago

Hello @HenrikBach1

Is the environment variable IOTEDGE_HOMEDIR set, or set to a path where the user running iotedged does not have write permissions? That "/tmp/iotedge" looks unusual. The default is "/var/lib/iotedge".

FYI: The error is the line number of the failure... That would be here, but that doesn't tell me much.

HenrikBach1 commented 6 years ago

Hi folks,

Thank you for your suggestions.

@varunpuranik No, due to we haven't any deb package management on the device, I, manually, extracted all needed packages and their dependencies by these steps:

  1. Downloaded the required package to "/tmp",
  2. Created a subfolder for each package, in the "/tmp" folder, and extracted its contents into the subfolder by: ar x *.deb, and
  3. Extracted the contents of the data.*.* file relative to "/",

for the packages:

@darobs Creating the needed folder and setting the environment variable to that folder did the trick. After that, the initialization of the HSM went well as seen below.

However, doing a iotedge list:

root@ddau3:~# /etc/init.d/iotedge status
iotedge is running.
root@ddau3:~# iotedge --version
iotedge 1.0.0 (52ef77db24126bf473265fc09c53d35290a2dd6b)
root@ddau3:~# iotedge list
An error in the management http client occurred.
        caused by: Http error
        caused by: Invalid uri unix:///var/run/iotedge/mgmt.sock
root@ddau3:~# <6>2018-07-21T02:27:02Z [INFO] - Finished initializing hsm.
<6>2018-07-21T02:27:02Z [INFO] - Detecting if configuration file has changed...
<6>2018-07-21T02:27:02Z [INFO] - Change to configuration file detected.
<6>2018-07-21T02:27:02Z [INFO] - Removing all modules...
<6>2018-07-21T02:27:02Z [INFO] - Finished removing modules.
<6>2018-07-21T02:27:02Z [INFO] - Provisioning edge device...
<6>2018-07-21T02:27:02Z [INFO] - Manually provisioning device "some" in hub "something.some.com"
<6>2018-07-21T02:27:02Z [INFO] - Finished provisioning edge device.
<6>2018-07-21T02:27:02Z [INFO] - Starting management API...
<3>2018-07-21T02:27:02Z [ERR!] - An http error occurred.
<3>2018-07-21T02:27:02Z [ERR!] -        caused by: IO error
<3>2018-07-21T02:27:02Z [ERR!] -        caused by: No such file or directory (os error 2)
.

fails.

What do I need, to go any further, and should I create a new issue for above new state/case?

Thank you.

/Henrik

HenrikBach1 commented 6 years ago

Hi,

I notice in the config.yaml configuration file, that you're referencing to, that the iotedge service is provisioned by SystemD. However, we are not (yet) ready to support this, and thus we only support SysV.

So, what do I need to rewrite/-configure in the configuration file and to which example values?

I.e.:

...
###############################################################################
# Listen settings
###############################################################################
#
# Configures the listen addresses for the daemon.
#     management_uri - used by the Edge Agent and 'iotedge' CLI to start,
#                      stop, and manage modules
#     workload_uri   - used by modules to retrieve tokens and certificates
#
# The following uri schemes are supported:
#     http - listen over TCP
#     unix - listen over Unix domain socket
#     fd   - listen using systemd socket activation
#
# These values can be different from the connect URIs. For instance, when
# using the fd:// scheme for systemd:
#     listen address is fd://iotedge.workload,
#     connect address is unix:///var/run/iotedge/workload.sock
#
###############################################################################

listen:
  management_uri: "fd://iotedge.mgmt.socket"
  workload_uri: "fd://iotedge.socket"
...

And/Or, may it be something with the value of the connection string, due to the message Manually provisioning device "some" in hub "something.some.com", but it is given in the configuration file as:

...
###############################################################################
# Provisioning mode and settings
###############################################################################
#
# Configures the identity provisioning mode of the daemon.
#
# Supported modes:
#     manual - using an iothub connection string
#     dps    - using dps for provisioning
#
###############################################################################

provisioning:
  source: "manual"
  device_connection_string: "HostName=hebh-IoT-Hub-Test.azure-devices.net;DeviceId=runes_device;SharedAccessKey=*"
...

Thank you.

/Henrik

darobs commented 6 years ago

Hi Henrik.

Yes, our default configuration (in the package) specifies systemd socket units.

You are correct, you can avoid using these by assigning the listen uris to be the same as the connect uris. There is a caveat here - the sockets may get created by the application with a default file permission that won't allow other processes permission - especially if you use the default location of /var/run/iotedge. We are working on a solution for that, as we need it for every distribution that doesn't support systemd socket units.

darobs commented 6 years ago

Hello again Henrik,

Learning that you're not running system reminded me of something else:

The "something.some.com" is also coming from the default config settings. When system starts the iotedged, it runs:

/usr/bin/iotedged -c /etc/iotedge/config.yaml

If you don't specify the "-c" option, you get that default, which isn't at all useful to you.

HenrikBach1 commented 6 years ago

Hi @darobs ,

Thank you for your help and suggestions.

  1. The IoT Edge Daemon fails with a new error message as seen below:
root@ddau3:~# /usr/bin/iotedged -c /etc/iotedge/config.yaml
<6>2018-07-23T04:35:08Z [INFO] - Starting Azure IoT Edge Security Daemon
<6>2018-07-23T04:35:08Z [INFO] - Version - 1.0.0 (52ef77db24126bf473265fc09c53d35290a2dd6b)
<6>2018-07-23T04:35:08Z [INFO] - Using config file: /etc/iotedge/config.yaml
<6>2018-07-23T04:35:08Z [INFO] - Using runtime network id azure-iot-edge
<6>2018-07-23T04:35:08Z [INFO] - Initializing the module runtime...
<6>2018-07-23T04:35:08Z [INFO] - Finished initializing the module runtime.
<6>2018-07-23T04:35:08Z [INFO] - Configuring /var/lib/iotedge as the home directory.
<6>2018-07-23T04:35:08Z [INFO] - Configuring certificates...
<6>2018-07-23T04:35:08Z [INFO] - Transparent gateway certificates not found, operating in quick start mode...
<6>2018-07-23T04:35:08Z [INFO] - Finished configuring certificates.
<6>2018-07-23T04:35:08Z [INFO] - Initializing hsm...
<6>2018-07-23T04:35:08Z [INFO] - Finished initializing hsm.
<6>2018-07-23T04:35:08Z [INFO] - Detecting if configuration file has changed...
<6>2018-07-23T04:35:08Z [INFO] - Change to configuration file detected.
<6>2018-07-23T04:35:08Z [INFO] - Removing all modules...
<6>2018-07-23T04:35:08Z [INFO] - Finished removing modules.
<6>2018-07-23T04:35:08Z [INFO] - Provisioning edge device...
<6>2018-07-23T04:35:08Z [INFO] - Manually provisioning device "runes_device" in hub "hebh-IoT-Hub-Test.azure-devices.net"
<6>2018-07-23T04:35:08Z [INFO] - Finished provisioning edge device.
<6>2018-07-23T04:35:08Z [INFO] - Starting management API...
<3>2018-07-23T04:35:08Z [ERR!] - An http error occurred.
<3>2018-07-23T04:35:08Z [ERR!] -        caused by: Systemd error
<3>2018-07-23T04:35:08Z [ERR!] -        caused by: Missing required environment variable - LISTEN_FDNAMES.
  1. Is there an environment variable, that I can use to point at my configuration file instead, since it isn't looking for the configuration values stored in /etc/iotedge/config.yaml (as suggested in the above documentation), but some where else, which I can't find, nor figure out (searching for default.yaml taken from iotedge\edgelet\iotedged\src\settings.rs)?

Thank you.

/Henrik

darobs commented 6 years ago

The iotedged loads settings in the following manner:

  1. It starts by loading the default setting from the contents in this file.
  2. If a config file is given on the command line ("-c"), these settings are merged.
  3. We read the environment for variables which have a prefix of "iotedge" and merge those into the settings.

When we set this up in Systemd, we tell the service to start with "-c /etc/iotedge/config.yaml" which is also created as part of the packaging. We then expect the user to edit this file for the correct settings. I would expect a SysV Init script would do something similar.

The problem with the config.yaml in the package is that it's using those system socket units to specify the listening ports. That "Missing required environment variable - LISTEN_FDNAMES." error is due to the iotedged trying to use those system socket units - systemd passed the file descriptors of the socket units in the LISTEN_FDNAMES environment variable.

Without systemd, you should just set the "listen" sockets to the same as the "connect" socket in your config.yaml. For example

connect: management_uri: "unix:///var/run/iotedge/mgmt.sock" workload_uri: "unix:///var/run/iotedge/workload.sock"

listen: management_uri: "unix:///var/run/iotedge/mgmt.sock" workload_uri: "unix:///var/run/iotedge/workload.sock"

Thank you for you patience with this. I know getting Yocto set up properly is a significant effort, and if you aren't using systemd, this is making it harder. Yocto support is a feature in the backlog - I expect we can make this work much better in the near future.

HenrikBach1 commented 6 years ago

@darobs ,

Thank you, too, for your patience answering my questions. You guys are doing a tremendous job for this exiting new technology.

However, after having changed the listen sockets in the configuration file to:

listen:
  management_uri: "unix:///var/run/iotedge/mgmt.sock"
  workload_uri: "unix:///var/run/iotedge/workload.sock"
#  management_uri: "fd://iotedge.mgmt.socket"
#  workload_uri: "fd://iotedge.socket"

and (re-)starting the service, I get this error message (again):

root@ddau3:~# /usr/bin/iotedged -c /etc/iotedge/config.yaml
<6>2018-07-24T02:14:35Z [INFO] - Starting Azure IoT Edge Security Daemon
<6>2018-07-24T02:14:35Z [INFO] - Version - 1.0.0 (52ef77db24126bf473265fc09c53d35290a2dd6b)
<6>2018-07-24T02:14:35Z [INFO] - Using config file: /etc/iotedge/config.yaml
<6>2018-07-24T02:14:35Z [INFO] - Using runtime network id azure-iot-edge
<6>2018-07-24T02:14:35Z [INFO] - Initializing the module runtime...
<6>2018-07-24T02:14:35Z [INFO] - Finished initializing the module runtime.
<6>2018-07-24T02:14:35Z [INFO] - Configuring /var/lib/iotedge as the home directory.
<6>2018-07-24T02:14:35Z [INFO] - Configuring certificates...
<6>2018-07-24T02:14:35Z [INFO] - Transparent gateway certificates not found, operating in quick start mode...
<6>2018-07-24T02:14:35Z [INFO] - Finished configuring certificates.
<6>2018-07-24T02:14:35Z [INFO] - Initializing hsm...
<6>2018-07-24T02:14:35Z [INFO] - Finished initializing hsm.
<6>2018-07-24T02:14:35Z [INFO] - Detecting if configuration file has changed...
<6>2018-07-24T02:14:35Z [INFO] - Change to configuration file detected.
<6>2018-07-24T02:14:35Z [INFO] - Removing all modules...
<6>2018-07-24T02:14:35Z [INFO] - Finished removing modules.
<6>2018-07-24T02:14:35Z [INFO] - Provisioning edge device...
<6>2018-07-24T02:14:35Z [INFO] - Manually provisioning device "runes_device" in hub "hebh-IoT-Hub-Test.azure-devices.net"
<6>2018-07-24T02:14:35Z [INFO] - Finished provisioning edge device.
<6>2018-07-24T02:14:35Z [INFO] - Starting management API...
<3>2018-07-24T02:14:35Z [ERR!] - An http error occurred.
<3>2018-07-24T02:14:35Z [ERR!] -        caused by: IO error
<3>2018-07-24T02:14:35Z [ERR!] -        caused by: No such file or directory (os error 2)

  1. Do you have any clue to, what the service needs from me, now?

  2. With respect to your declared environment variables (given in "./edgelet/iotedged/src/lib.rs", and "./edgelet/iotedge/src/main.rs"), they cover almost every aspect, except, an environment variable, which could replace the hard-coded path "-c /etc/iotedge/config.yaml" to something like "-c $IOTEDGE_CONF". The variable could either be set in scripts, or the iotedged service/daemon/manager, itself, with a default path, if its value is not already given by an argument or defined in an environment variable.

Thank you.

/Henrik

darobs commented 6 years ago

I have an ask for your second point: Would you please provide that is feedback in User Voice?

Out of curiosity, does /var/run/iotedge/ exist? That's probably part of the problem.

Also, be aware of the caveat I mentioned before - the sockets may get created by the application with a default file permission that won't allow other processes permission. The (unsatisfactory) workaround is to create a directory that is writable by the "iotedge" user and when the socket files are created, do this:

chmod 660 mgmt.sock
chmod 666 workload.sock

We are working on a solution for that right now, as we need it for every distribution that doesn't support systemd socket units.

HenrikBach1 commented 6 years ago

@darobs ,

My feedback is now entered into User Voice: https://feedback.azure.com/forums/907045-azure-iot-edge/suggestions/34944301-environment-variable-iotedge-conf-needed-instead.

No, the /var/run/iotedge/ didn't exist. So now, it is created. Before doing anything further, I tried to start the service:

root@ddau3:~# mkdir -p /var/run/iotedge/
root@ddau3:~# export IOTEDGE_HOMEDIR=/var/lib/iotedge
root@ddau3:~# /etc/init.d/iotedge start
Starting iotedged : iotedge<6>2018-07-25T02:29:23Z [INFO] - Starting Azure IoT Edge Security Daemon
<6>2018-07-25T02:29:23Z [INFO] - Version - 1.0.0 (52ef77db24126bf473265fc09c53d35290a2dd6b)
<6>2018-07-25T02:29:23Z [INFO] - Using default configuration
<6>2018-07-25T02:29:23Z [INFO] - Using runtime network id azure-iot-edge
<6>2018-07-25T02:29:23Z [INFO] - Initializing the module runtime...
<6>2018-07-25T02:29:23Z [INFO] - Finished initializing the module runtime.
<6>2018-07-25T02:29:23Z [INFO] - Configuring /var/lib/iotedge as the home directory.
<6>2018-07-25T02:29:23Z [INFO] - Configuring certificates...
<6>2018-07-25T02:29:23Z [INFO] - Transparent gateway certificates not found, operating in quick start mode...
<6>2018-07-25T02:29:23Z [INFO] - Finished configuring certificates.
<6>2018-07-25T02:29:23Z [INFO] - Initializing hsm...
<6>2018-07-25T02:29:23Z [INFO] - Finished initializing hsm.
<6>2018-07-25T02:29:23Z [INFO] - Detecting if configuration file has changed...
<6>2018-07-25T02:29:23Z [INFO] - Change to configuration file detected.
<6>2018-07-25T02:29:23Z [INFO] - Removing all modules...
<6>2018-07-25T02:29:23Z [INFO] - Finished removing modules.
<6>2018-07-25T02:29:23Z [INFO] - Provisioning edge device...
<6>2018-07-25T02:29:23Z [INFO] - Manually provisioning device "some" in hub "something.some.com"
<6>2018-07-25T02:29:23Z [INFO] - Finished provisioning edge device.
<6>2018-07-25T02:29:23Z [INFO] - Starting management API...
<6>2018-07-25T02:29:23Z [INFO] - Listening on unix:///var/run/iotedge/mgmt.sock with 1 thread for management API.
<6>2018-07-25T02:29:23Z [INFO] - Starting workload API...
<6>2018-07-25T02:29:23Z [INFO] - Listening on unix:///var/run/iotedge/workload.sock with 1 thread for workload API.
<6>2018-07-25T02:29:23Z [INFO] - Starting watchdog with 60 second frequency...
<6>2018-07-25T02:29:23Z [INFO] - Checking edge runtime status
<6>2018-07-25T02:29:23Z [INFO] - Creating and starting edge runtime module edgeAgent
<3>2018-07-25T02:29:23Z [ERR!] - Io(Custom { kind: Other, error: StringError("failed to lookup address information: Name or service not known") })
<4>2018-07-25T02:29:23Z [WARN] - Error in watchdog when checking for edge runtime status:
<4>2018-07-25T02:29:23Z [WARN] - Identity error
<4>2018-07-25T02:29:23Z [WARN] -        caused by: IoT Hub service error
<4>2018-07-25T02:29:23Z [WARN] -        caused by: Core error
<4>2018-07-25T02:29:23Z [WARN] -        caused by: Hyper error
<4>2018-07-25T02:29:23Z [WARN] -        caused by: failed to lookup address information: Name or service not known
<6>2018-07-25T02:30:23Z [INFO] - Checking edge runtime status
<6>2018-07-25T02:30:23Z [INFO] - Creating and starting edge runtime module edgeAgent
<3>2018-07-25T02:30:23Z [ERR!] - Io(Custom { kind: Other, error: StringError("failed to lookup address information: Name or service not known") })

Then, I followed your suggestions, before starting the service:

root@ddau3:~# ls -halt /var/run/iotedge/
total 0
drwxr-xr-x  2 root root  80 Jul 25 02:29 .
srwxr-xr-x  1 root root   0 Jul 25 02:29 workload.sock
srwxr-xr-x  1 root root   0 Jul 25 02:29 mgmt.sock
drwxr-xr-x 12 root root 600 Jul 25 02:27 ..
root@ddau3:~# cd /var/run/iotedge/
root@ddau3:/var/run/iotedge# chmod 660 mgmt.sock
root@ddau3:/var/run/iotedge# chmod 666 workload.sock
root@ddau3:/var/run/iotedge# ls -halt /var/run/iotedge/
total 0
drwxr-xr-x  2 root root  80 Jul 25 02:29 .
srw-rw-rw-  1 root root   0 Jul 25 02:29 workload.sock
srw-rw----  1 root root   0 Jul 25 02:29 mgmt.sock
drwxr-xr-x 12 root root 600 Jul 25 02:27 ..
root@ddau3:/var/run/iotedge# /etc/init.d/iotedge start
Starting iotedged : iotedge<6>2018-07-25T02:46:38Z [INFO] - Starting Azure IoT Edge Security Daemon
<6>2018-07-25T02:46:38Z [INFO] - Version - 1.0.0 (52ef77db24126bf473265fc09c53d35290a2dd6b)
<6>2018-07-25T02:46:38Z [INFO] - Using default configuration
<6>2018-07-25T02:46:38Z [INFO] - Using runtime network id azure-iot-edge
<6>2018-07-25T02:46:38Z [INFO] - Initializing the module runtime...
<6>2018-07-25T02:46:38Z [INFO] - Finished initializing the module runtime.
<6>2018-07-25T02:46:38Z [INFO] - Configuring /var/lib/iotedge as the home directory.
<6>2018-07-25T02:46:38Z [INFO] - Configuring certificates...
<6>2018-07-25T02:46:38Z [INFO] - Transparent gateway certificates not found, operating in quick start mode...
<6>2018-07-25T02:46:38Z [INFO] - Finished configuring certificates.
<6>2018-07-25T02:46:38Z [INFO] - Initializing hsm...
<6>2018-07-25T02:46:38Z [INFO] - Finished initializing hsm.
<6>2018-07-25T02:46:38Z [INFO] - Detecting if configuration file has changed...
<6>2018-07-25T02:46:38Z [INFO] - No change to configuration file detected.
<6>2018-07-25T02:46:38Z [INFO] - Provisioning edge device...
<6>2018-07-25T02:46:38Z [INFO] - Manually provisioning device "some" in hub "something.some.com"
<6>2018-07-25T02:46:38Z [INFO] - Finished provisioning edge device.
<6>2018-07-25T02:46:38Z [INFO] - Starting management API...
<6>2018-07-25T02:46:38Z [INFO] - Listening on unix:///var/run/iotedge/mgmt.sock with 1 thread for management API.
<6>2018-07-25T02:46:38Z [INFO] - Starting workload API...
<6>2018-07-25T02:46:38Z [INFO] - Listening on unix:///var/run/iotedge/workload.sock with 1 thread for workload API.
<6>2018-07-25T02:46:38Z [INFO] - Starting watchdog with 60 second frequency...
<6>2018-07-25T02:46:38Z [INFO] - Checking edge runtime status
<6>2018-07-25T02:46:38Z [INFO] - Creating and starting edge runtime module edgeAgent
<3>2018-07-25T02:46:39Z [ERR!] - Io(Custom { kind: Other, error: StringError("failed to lookup address information: Name or service not known") })
<4>2018-07-25T02:46:39Z [WARN] - Error in watchdog when checking for edge runtime status:
<4>2018-07-25T02:46:39Z [WARN] - Identity error
<4>2018-07-25T02:46:39Z [WARN] -        caused by: IoT Hub service error
<4>2018-07-25T02:46:39Z [WARN] -        caused by: Core error
<4>2018-07-25T02:46:39Z [WARN] -        caused by: Hyper error
<4>2018-07-25T02:46:39Z [WARN] -        caused by: failed to lookup address information: Name or service not known

As noticed above, the service is run by the root account.

I am pretty sure, that you have provided installation and configuration scripts (looked for ".sh") with your .deb packages, but I can't find any in the libiothsm-std.deb, nor iotedge.deb packages.

At the moment, we don't have any package manager on our device image.

Then, which scripts should I extract and run manually, or look into to extract configuration steps from, given your sources here in GitHub?

Thank you.

/Henrik

HenrikBach1 commented 6 years ago

Hi @darobs ,

I notice, that following your suggestions for setting the appropriate permissions in the /var/run/iotedge folder before calling the iotedge service, something is setting them back in the service:

root@ddau3:~# date
Sat Aug 11 09:37:05 UTC 2018
root@ddau3:~# ls -halt /var/run/iotedge/
total 0
drw-rw-rw-  2 root root  80 Aug 11 09:31 .
srwxr-xr-x  1 root root   0 Aug 11 09:31 workload.sock
srwxr-xr-x  1 root root   0 Aug 11 09:31 mgmt.sock
drwxr-xr-x 12 root root 600 Jul 25 02:27 ..
root@ddau3:~# chmod 660 /var/run/iotedge/mgmt.sock
root@ddau3:~# chmod 666 /var/run/iotedge/workload.sock
root@ddau3:~# ls -halt /var/run/iotedge/
total 0
drw-rw-rw-  2 root root  80 Aug 11 09:31 .
srw-rw-rw-  1 root root   0 Aug 11 09:31 workload.sock
srw-rw----  1 root root   0 Aug 11 09:31 mgmt.sock
drwxr-xr-x 12 root root 600 Jul 25 02:27 ..
root@ddau3:~# date
Sat Aug 11 09:37:41 UTC 2018
root@ddau3:~# IOTEDGE_HOMEDIR=/var/lib/iotedge && chmod 660 /var/run/iotedge/mgmt.sock && chmod 666 /var/run/iotedge/workload.sock /var/run/iotedge && /etc/init.d/iotedge start -c /etc/iotedge/config.yaml
Starting iotedged : iotedge<6>2018-08-11T09:38:06Z [INFO] - Starting Azure IoT Edge Security Daemon
<6>2018-08-11T09:38:06Z [INFO] - Version - 1.0.0 (52ef77db24126bf473265fc09c53d35290a2dd6b)
<6>2018-08-11T09:38:06Z [INFO] - Using default configuration
<6>2018-08-11T09:38:06Z [INFO] - Using runtime network id azure-iot-edge
<6>2018-08-11T09:38:06Z [INFO] - Initializing the module runtime...
<6>2018-08-11T09:38:06Z [INFO] - Finished initializing the module runtime.
<6>2018-08-11T09:38:06Z [INFO] - Configuring /var/lib/iotedge as the home directory.
<6>2018-08-11T09:38:06Z [INFO] - Configuring certificates...
<6>2018-08-11T09:38:06Z [INFO] - Transparent gateway certificates not found, operating in quick start mode...
<6>2018-08-11T09:38:06Z [INFO] - Finished configuring certificates.
<6>2018-08-11T09:38:06Z [INFO] - Initializing hsm...
<6>2018-08-11T09:38:06Z [INFO] - Finished initializing hsm.
<6>2018-08-11T09:38:06Z [INFO] - Detecting if configuration file has changed...
<6>2018-08-11T09:38:06Z [INFO] - No change to configuration file detected.
<6>2018-08-11T09:38:06Z [INFO] - Provisioning edge device...
<6>2018-08-11T09:38:06Z [INFO] - Manually provisioning device "some" in hub "something.some.com"
<6>2018-08-11T09:38:06Z [INFO] - Finished provisioning edge device.
<6>2018-08-11T09:38:06Z [INFO] - Starting management API...
<6>2018-08-11T09:38:06Z [INFO] - Listening on unix:///var/run/iotedge/mgmt.sock with 1 thread for management API.
<6>2018-08-11T09:38:06Z [INFO] - Starting workload API...
<6>2018-08-11T09:38:06Z [INFO] - Listening on unix:///var/run/iotedge/workload.sock with 1 thread for workload API.
<6>2018-08-11T09:38:06Z [INFO] - Starting watchdog with 60 second frequency...
<6>2018-08-11T09:38:06Z [INFO] - Checking edge runtime status
<6>2018-08-11T09:38:06Z [INFO] - Creating and starting edge runtime module edgeAgent
<3>2018-08-11T09:38:06Z [ERR!] - Io(Custom { kind: Other, error: StringError("failed to lookup address information: Name or service not known") })
<4>2018-08-11T09:38:06Z [WARN] - Error in watchdog when checking for edge runtime status:
<4>2018-08-11T09:38:06Z [WARN] - Identity error
<4>2018-08-11T09:38:06Z [WARN] -        caused by: IoT Hub service error
<4>2018-08-11T09:38:06Z [WARN] -        caused by: Core error
<4>2018-08-11T09:38:06Z [WARN] -        caused by: Hyper error
<4>2018-08-11T09:38:06Z [WARN] -        caused by: failed to lookup address information: Name or service not known

root@ddau3:~# date
Sat Aug 11 09:38:19 UTC 2018
root@ddau3:~# ls -halt /var/run/iotedge/
total 0
drw-rw-rw-  2 root root  80 Aug 11 09:38 .
srwxr-xr-x  1 root root   0 Aug 11 09:38 mgmt.sock
srwxr-xr-x  1 root root   0 Aug 11 09:38 workload.sock
drwxr-xr-x 12 root root 600 Jul 25 02:27 ..
root@ddau3:~# date
Sat Aug 11 09:38:25 UTC 2018
root@ddau3:~#

/Henrik

myagley commented 6 years ago

There is currently an issue when unlinking the socket and retaining the permissions. This has been addressed in https://github.com/Azure/iotedge/pull/113 and will be part of the next release. The release should be out this week.

HenrikBach1 commented 6 years ago

Thank you, @myagley, for the update. Looking forward to the next release.

HenrikBach1 commented 6 years ago 
root@ddau3:~# /usr/bin/iotedge --version
iotedge 1.0.1 (6e5e86dcf0c9a3732fc72a64d9ec9b0fcb2d6fad)

rocks. :-)

Issue is closed.

/Henrik