search

aws-greengrass / aws-greengrass-nucleus

The Greengrass nucleus component provides functionality for device side orchestration of deployments and lifecycle management for execution of Greengrass components and applications. This includes features such as starting, stopping, and monitoring execution of components and apps, interprocess communication server for communication between components, component installation and configuration management.
Apache License 2.0
109 stars 45 forks source link

Unable to get Tunneling working on GGv2 on Ubuntu 20.04 #877

Closed kmoralescr closed 3 years ago

kmoralescr commented 3 years ago

Describe the bug Hello AWS team, I might be missing something with the public component, at first sight everything is OK, however I'm seeing may errors in the aws.greengrass.SecureTunneling.log and I constantly seeing Proxy server rejected web socket upgrade request: (HTTP/1.1 403 Forbidden) "Invalid access-token"

To Reproduce I did have ggv2 in ubuntu 16.04LTS, due to potential security requirements, I also tried in Ubuntu 20.04 with the same results. Steps to reproduce the behavior. If possible, provide a minimal amount of code that causes the bug. image

image With my tunneling open, now I do try to activate my localproxy image

I did checked the content of the token, all seems to match. I also compared the destination token and matched with the MQTT message.

I also force my laptop to accept outbound connection iptables -A OUTPUT -p tcp --dport 443 -j ACCEPT

Expected behavior I was expecting an smooth tunneling, just like the component installation.

Actual behavior alway I got the Proxy server rejected web socket upgrade request: (HTTP/1.1 403 Forbidden) "Invalid access-token" Checking ggv2 logs i see may errors.

Environment

Additional context

I wondering if there is a missing priviledge somewhere. I suspected on the localproxy I had, so I decided to compile it, took a while, however the same error. I'm including the logs aws.greengrass.SecureTunneling.log Appreciate your support. thanks Keiner

2021-02-28T02:41:51.338Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [INFO ] 2021-02-27 20:41:51.338 [Thread-1] SecureTunnelingTask - Successfully subscribed to topic: $aws/things/keiner_laptop/tunnels/notify. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceNa
me=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.631Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [INFO ] 2021-02-27 20:44:05.630 [Thread-1] SubscribeResponseHandler - Received new tunnel notification message.. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunn
eling, currentState=RUNNING}
2021-02-28T02:44:05.644Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.643 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process: 2021-02-27T20:44:05.640Z [WARN]  {FileUtils.cpp}: Permissions to given file/dir path '/tmp/' is not set to recommended va
lue... {Permissions: {desired: 745, actual: 777}}. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.646Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.646 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process: 2021-02-27T20:44:05.640Z [WARN]  {FileUtils.cpp}: Permissions to given file/dir path '/tmp/device-client-settings.json131
31856035475339241614480110832' is not set to recommended value... {Permissions: {desired: 644, actual: 664}}. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.648Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.647 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process: 2021-02-27T20:44:05.640Z [INFO]  {Config.cpp}: Successfully fetched JSON config file: {. {scriptName=services.aws.greengr
ass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.649Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.647 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "endpoint": "replace_with_endpoint_value",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.649Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.647 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "cert": "replace_with_certificate_file_location",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.649Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.647 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "key": "replace_with_private_key_file_location",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.649Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.647 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "root-ca": "replace_with_root_ca_file_location",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.649Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.648 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "thing-name": "replace_with_thing_name",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.650Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.648 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "logging": {. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.650Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.648 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "level": "ERROR",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.650Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.648 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "type": "STDOUT",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.650Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.649 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "file": "/var/log/aws-iot-device-client/aws-iot-device-client.log". {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.650Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.649 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   },. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.651Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.649 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "jobs": {. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.651Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.649 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "enabled": false,. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.651Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.649 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "handler-directory": "replace_with_path_to_handler_dir". {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.652Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.649 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   },. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.652Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "tunneling": {. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.652Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "enabled": true. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.652Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   },. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.654Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "device-defender":    {. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.654Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "enabled":  false,. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "interval": 300. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   },. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.650 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   "fleet-provisioning": {. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.651 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "enabled": false,. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.652 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "template-name": "replace_with_template_name",. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.652 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:     "csr-file": "replace_with_csr-file-path". {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.652 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process:   }. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.652 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process: }. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}
2021-02-28T02:44:05.655Z [INFO] (Copier) aws.greengrass.SecureTunneling: stdout. [ERROR] 2021-02-27 20:44:05.652 [pool-3-thread-1] SubscribeResponseHandler - Secure Tunneling Process: 2021-02-27T20:44:05.640Z [DEBUG] {Config.cpp}: Did not find a runtime configuration file, assuming Fleet Provisioning has not run for this device. {scriptName=services.aws.greengrass.SecureTunneling.lifecycle.run.script, serviceName=aws.greengrass.SecureTunneling, currentState=RUNNING}

E.g. what is the impact of the bug?

MikeDombo commented 3 years ago

I have forwarded your issue to the appropriate team internally, however I'd probably also recommend that you file a support ticket or use the AWS Forum.

This GitHub repository is only for the main Greengrass core software (nucleus) and does not include the Secure Tunneling software.

fufranci commented 3 years ago

Please see the instruction of how to use localproxy here: https://github.com/aws-samples/aws-iot-securetunneling-localproxy

When running localproxy and use -t to specify the token, it is expecting the token itself is following after -t, not the filename containing the token.

Also, from your log, I don't see the secure tunneling component logged any error. So I am assuming you are having problem with the local proxy only.

kmoralescr commented 3 years ago

thanks @fufranci , you got it!, now it's working as expected. For community reference, I'll let an example documented, I did checked the documentation however it's not that intuitive and I naturally was very tempted to use the file address instead (in all AWS IoT examples, pointing to a file is OK, e.g: pointing to the certificates). Please do consider to add some additional wording in the error log as well

Thanks to your help, I have successfully run those commands: A) using the token directly in terminal image B) using the env variable AWSIOT_TUNNEL_ACCESS_TOKEN image

Thanks again