search

aws-samples / arduino-aws-greengrass-iot

This library uses AWS C-SDK to implement an Arduino class AWSGreenGrassIoT to make it easy to securely connect sensors/actuator to AWS IoT core, directly or by mean of an AWS greengrass device (i.e. Raspberry PI) using X509 certificates.
Apache License 2.0
44 stars 13 forks source link

IP address that cannot be connected is selected #6

Open moritalous opened 4 years ago

moritalous commented 4 years ago

I have Greengrass installed on my Raspberry Pi. Docker is installed on the Raspberry Pi. The results of ifconfig are:

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:80:a8:17:03  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether b8:27:eb:59:ce:be  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 6954186  bytes 775818076 (739.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6954186  bytes 775818076 (739.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.12  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 2001:ce8:157:7ae:666e:b43f:d3f:4ab3  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::5541:24c9:57a9:8889  prefixlen 64  scopeid 0x20<link>
        ether b8:27:eb:0c:9b:eb  txqueuelen 1000  (Ethernet)
        RX packets 4405047  bytes 1295076920 (1.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4931290  bytes 1709660416 (1.5 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The connection from Arduino to Greengrass fails because the IP address of Docker0 is selected.

In latest aws_greengrass_discovery.c (https://github.com/aws/amazon-freertos/blob/master/libraries/freertos_plus/aws/greengrass/src/aws_greengrass_discovery.c) in FreeRTOS repository, check connection with GGD_SecureConnect_Connect, The IP address that was able to connect is selected.

bvitali commented 4 years ago

Can you check which addresses you see, in IoT Core console->Greengrass->Cores->Connectivity? Did you deploy the GGroup with manual or automatic addres discovery? The greengrass discovery, doesn’t occur locally, so your environment should not play any role: the device queries IoT Core directly and extract IP(s) of the greengrass core from there.

On 30. May 2020, at 14:55, moritalous notifications@github.com wrote:

I have Greengrass installed on my Raspberry Pi. Docker is installed on the Raspberry Pi. The results of ifconfig are:

docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255 ether 02:42:80:a8:17:03 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether b8:27:eb:59:ce:be txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 6954186 bytes 775818076 (739.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 6954186 bytes 775818076 (739.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.0.12 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 2001:ce8:157:7ae:666e:b43f:d3f:4ab3 prefixlen 64 scopeid 0x0 inet6 fe80::5541:24c9:57a9:8889 prefixlen 64 scopeid 0x20 ether b8:27:eb:0c:9b:eb txqueuelen 1000 (Ethernet) RX packets 4405047 bytes 1295076920 (1.2 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 4931290 bytes 1709660416 (1.5 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 The connection from Arduino to Greengrass fails because the IP address of Docker0 is selected.

In latest aws_greengrass_discovery.c (https://github.com/aws/amazon-freertos/blob/master/libraries/freertos_plus/aws/greengrass/src/aws_greengrass_discovery.c https://github.com/aws/amazon-freertos/blob/master/libraries/freertos_plus/aws/greengrass/src/aws_greengrass_discovery.c) in FreeRTOS repository, check connection with GGD_SecureConnect_Connect, The IP address that was able to connect is selected.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/aws-samples/arduino-aws-greengrass-iot/issues/6, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMR7VE67PXIET5BHHW5T23RUD62XANCNFSM4NOV4RKA.

moritalous commented 4 years ago

Thank you for your reply.

Can you check which addresses you see, in IoT Core console->Greengrass->Cores->Connectivity?

My Greengrass Core Connectivity Setting is below

Core endpoints
127.0.0.1
Port 8883
172.17.0.1      <- docker0
Port 8883
192.168.0.12      <- wlan0
Port 8883
2001:ce8:157:7ae:666e:b43f:d3f:4ab3
Port 8883
::1
Port 8883
fe80::5541:24c9:57a9:8889
Port 8883

My ESP32 Log is hare.

Connected to wifi
Response from greengrass discovery
Discovered Greengrass IP address 172.17.0.1, port: 8883
E (24662) aws_iot: failed! mbedtls_net_connect returned -0x44
Failed to connect to Greengrass
Connection to AWS IoT Greengrass failed

I uninstalled Docker and removed the docker0 interface.

sudo apt remove docker-ce docker-ce-cli docker-compose -y
sudo ip link delete docker0
sudo systemctl restart greengrass

My Connectivity Setting has changed as below

Core endpoints
127.0.0.1
Port 8883
192.168.0.12
Port 8883
2001:ce8:157:7ae:666e:b43f:d3f:4ab3
Port 8883
::1
Port 8883
fe80::5541:24c9:57a9:8889
Port 8883

My ESP32 Log is hare. 

Connected to wifi
Response from greengrass discovery
Discovered Greengrass IP address 192.168.0.12, port: 8883
Connected to AWS IoT Greengrass

Did you deploy the GGroup with manual or automatic addres discovery?

I choose Automatically detect and override connection information on GREENGRASS GROUP Setting.

Thank you.

bvitali commented 4 years ago

Thanks for your info. I will need to find the time to check why this is happening and implement the proper fix. Sorry about the inconvenience: container support in greengrass was added after I wrote the code and evidently is having some unexpected effect on the GG core discovery. Regards Bruno

On 31. May 2020, at 03:10, moritalous notifications@github.com wrote:

Thank you for your reply.

Can you check which addresses you see, in IoT Core console->Greengrass->Cores->Connectivity?

My Greengrass Core Connectivity Setting is below

Core endpoints 127.0.0.1 Port 8883 172.17.0.1 <- docker0 Port 8883 192.168.0.12 <- wlan0 Port 8883 2001:ce8:157:7ae:666e:b43f:d3f:4ab3 Port 8883 ::1 Port 8883 fe80::5541:24c9:57a9:8889 Port 8883 My ESP32 Log is hare.

Connected to wifi Response from greengrass discovery Discovered Greengrass IP address 172.17.0.1, port: 8883 E (24662) aws_iot: failed! mbedtls_net_connect returned -0x44 Failed to connect to Greengrass Connection to AWS IoT Greengrass failed I uninstalled Docker and removed the docker0 interface.

sudo apt remove docker-ce docker-ce-cli docker-compose -y sudo ip link delete docker0 sudo systemctl restart greengrass My Connectivity Setting has changed as below

Core endpoints 127.0.0.1 Port 8883 192.168.0.12 Port 8883 2001:ce8:157:7ae:666e:b43f:d3f:4ab3 Port 8883 ::1 Port 8883 fe80::5541:24c9:57a9:8889 Port 8883 My ESP32 Log is hare.

Connected to wifi Response from greengrass discovery Discovered Greengrass IP address 192.168.0.12, port: 8883 Connected to AWS IoT Greengrass Did you deploy the GGroup with manual or automatic addres discovery?

I choose Automatically detect and override connection information on GREENGRASS GROUP Setting.

Thank you.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aws-samples/arduino-aws-greengrass-iot/issues/6#issuecomment-636405856, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMR7VFOCQXOMV4DC2BCJDTRUGVANANCNFSM4NOV4RKA.

rodmaz commented 3 years ago

Any update on this? It seems that when we run GG on Docker, lambda GGIPDetector in fact retrieves the internal container IP and not the host.

2021-05-08T02:30:27.992Z][INFO]-Got new IP set: [127.0.0.1 172.18.0.2]
[2021-05-08T02:30:27.992Z][INFO]-Send message: {"ConnectivityInfo":[{"Id":"AUTOIP_127.0.0.1_0","HostAddress":"127.0.0.1","PortNumber":8883,"Metadata":""},{"Id":"AUTOIP_172.18.0.2_1","HostAddress":"172.18
.0.2","PortNumber":8883,"Metadata":""}]}
[2021-05-08T02:30:27.993Z][INFO]-function is initialized, starting to process work items ...    {"functionName": "arn:aws:lambda:::function:GGIPDetector:1"}
[2021-05-08T02:30:27.993Z][INFO]-Sending POST to https://greengrass-ats.iot.us-east-1.amazonaws.com:8443/greengrass/connectivityInfo/thing/cc435b449491ec0c766e59eea30986d1
[2021-05-08T02:30:28.797Z][INFO]-The server says: {"message":null,"traceId":"bea83aa3-c97e-bb5f-f4ec-762408d1421a"}
rodmaz commented 3 years ago

Some updates for anyone checking this. We decided to run the container in Docker host networking mode (network_mode: host). This solves the problem of retrieving the actual connectivity IP addresses. However we hit another wall. The GGIPDetector collects all interfaces and IPs. In some real-world situations, we may have 5-10 interfaces and the AWS Greengrass connectivity endpoint was rejecting the requests.

So instead of using AWS provided GGIPDetector lambda to monitor connectivity, we simply wrote our own connectivity monitoring lambda.