search

aws-samples / aws-iot-securetunneling-localproxy

AWS Iot Secure Tunneling local proxy reference C++ implementation
https://docs.aws.amazon.com/iot/latest/developerguide/what-is-secure-tunneling.html
Apache License 2.0
73 stars 70 forks source link

Port forwarding problem #75

Closed steffenstolze closed 2 years ago

steffenstolze commented 2 years ago

Describe the bug

We want to swap our own SSH jump host solution with AWS IoT Secure Tunneling and we're facing some minor issues. We want to forward SSH (Port 22) as well as a web server, running in a Docker container, exposed on Port 3011.

Only SSH can be forwarded.

To Reproduce

We've tested this on a brand new EC2 with Amazon Linux. Running a Docker container, exposing a Vue website to localhost on port 3011.

Expected behavior

I want to open the website on source side at http://localhost:3011/

Actual behavior

Browser can't open the page.

Logs

[2022-03-23T20:22:15.683484]{28953}[debug]   Detect port mapping configuration provided through CLI in destination mode:
[2022-03-23T20:22:15.683506]{28953}[debug]   ----------------------------------------------------------
[2022-03-23T20:22:15.683526]{28953}[debug]   UI = 3011
[2022-03-23T20:22:15.683543]{28953}[debug]   SSH = 22
[2022-03-23T20:22:15.683559]{28953}[debug]   ----------------------------------------------------------
[2022-03-23T20:22:15.683589]{28953}[debug]   /home/dependencies/aws-iot-securetunneling-localproxy/build/bin/config does not exist!
[2022-03-23T20:22:15.683623]{28953}[info]    Starting proxy in destination mode
[2022-03-23T20:22:15.691444]{28953}[info]    Attempting to establish web socket connection with endpoint wss://data.tunneling.iot.eu-central-1.amazonaws.com:443
[2022-03-23T20:22:15.694260]{28953}[debug]   Resolved proxy server IP: 3.123.222.142
[2022-03-23T20:22:15.695046]{28953}[debug]   Connected successfully with proxy server
[2022-03-23T20:22:15.698704]{28953}[debug]   Successfully completed SSL handshake with proxy server
[2022-03-23T20:22:15.717992]{28953}[info]    Web socket session ID: 02d0c1fffe4bb9ac-00001452-000022a1-77b72bc14379bb94-5e3ab51a
[2022-03-23T20:22:15.718035]{28953}[debug]   Web socket subprotocol selected: aws.iot.securetunneling-2.0
[2022-03-23T20:22:15.718056]{28953}[info]    Successfully established websocket connection with proxy server: wss://data.tunneling.iot.eu-central-1.amazonaws.com:443
[2022-03-23T20:22:15.718082]{28953}[debug]   Seting up web socket pings for every 5000 milliseconds
[2022-03-23T20:22:15.718132]{28953}[debug]   Scheduled next read:
[2022-03-23T20:22:15.718274]{28953}[debug]   No serviceId_to_tcp_client mapping for service_id: 
[2022-03-23T20:22:15.718308]{28953}[debug]   Extracting service Ids from control message 5
[2022-03-23T20:22:15.718342]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:22:15.718377]{28953}[debug]   Starting web socket read loop while web socket is already reading. Ignoring...
[2022-03-23T20:22:15.718404]{28953}[debug]   Starting web socket read loop while web socket is already reading. Ignoring...
[2022-03-23T20:22:36.122625]{28953}[debug]   Received service id :UI ,stream id: 1
[2022-03-23T20:22:36.122689]{28953}[info]    Attempting to establish tcp socket connection to: 3011
[2022-03-23T20:22:36.122722]{28953}[debug]   Web socket read loop stopped
[2022-03-23T20:22:36.122795]{28953}[debug]   Resolved destination host to IP: 127.0.0.1 , connecting ...
[2022-03-23T20:22:36.122906]{28953}[info]    Connected to 127.0.0.1, port: 3011
[2022-03-23T20:22:36.122933]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:22:36.176619]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:22:36.177531]{28953}[debug]   Prepare to send data message: service id: UI stream id: 1
[2022-03-23T20:22:36.177632]{28953}[debug]   Write buffer has enough space, continue tcp read loop for UI
[2022-03-23T20:22:36.206616]{28953}[debug]   Starting web socket read loop continue reading...
....
[2022-03-23T20:22:36.240773]{28953}[debug]   Prepare to send data message: service id: UI stream id: 1
[2022-03-23T20:22:36.240811]{28953}[debug]   Write buffer has enough space, continue tcp read loop for UI
[2022-03-23T20:22:36.353605]{28953}[debug]   Handling explicit reset by closing TCP for service id: UI
[2022-03-23T20:22:36.353678]{28953}[info]    Disconnected from: 127.0.0.1:3011
[2022-03-23T20:22:36.353749]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:22:36.353786]{28953}[debug]   Starting web socket read loop while web socket is already reading. Ignoring...
[2022-03-23T20:22:36.638305]{28953}[debug]   Received service id :UI ,stream id: 2
[2022-03-23T20:22:36.638376]{28953}[info]    Attempting to establish tcp socket connection to: 3011
[2022-03-23T20:22:36.638405]{28953}[debug]   Web socket read loop stopped
[2022-03-23T20:22:36.638470]{28953}[debug]   Resolved destination host to IP: 127.0.0.1 , connecting ...
[2022-03-23T20:22:36.638613]{28953}[info]    Connected to 127.0.0.1, port: 3011
[2022-03-23T20:22:36.638643]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:22:36.652470]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:22:36.653433]{28953}[debug]   Prepare to send data message: service id: UI stream id: 2
[2022-03-23T20:22:36.653468]{28953}[debug]   Write buffer has enough space, continue tcp read loop for UI
[2022-03-23T20:22:41.653989]{28953}[debug]   Handling tcp socket error for service id: UI . error message:End of file
[2022-03-23T20:22:41.654054]{28953}[info]    Disconnected from: 127.0.0.1:3011
[2022-03-23T20:22:57.486868]{28953}[debug]   Starting web socket read loop while web socket is already reading. Ignoring...
[2022-03-23T20:23:37.464563]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:23:37.467497]{28953}[debug]   Received service id :UI ,stream id: 3
[2022-03-23T20:23:37.467541]{28953}[info]    Attempting to establish tcp socket connection to: 3011
[2022-03-23T20:23:37.467572]{28953}[debug]   Web socket read loop stopped
[2022-03-23T20:23:37.467605]{28953}[debug]   Resolved destination host to IP: 127.0.0.1 , connecting ...
[2022-03-23T20:23:37.467712]{28953}[info]    Connected to 127.0.0.1, port: 3011
[2022-03-23T20:23:37.467734]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:23:37.467792]{28953}[debug]   Starting web socket read loop continue reading...
[2022-03-23T20:23:37.469016]{28953}[debug]   Prepare to send data message: service id: UI stream id: 3
[2022-03-23T20:23:37.469092]{28953}[debug]   Write buffer has enough space, continue tcp read loop for UI
[2022-03-23T20:23:42.473145]{28953}[debug]   Handling tcp socket error for service id: UI . error message:End of file
[2022-03-23T20:23:42.473201]{28953}[info]    Disconnected from: 127.0.0.1:3011
[2022-03-23T20:23:42.473332]{28953}[debug]   Starting web socket read loop while web socket is already reading. Ignoring...

If we try to curl localhost:3011 from source:

curl localhost:3011 -vvv
*   Trying ::1:3011...
* Connected to localhost (::1) port 3011 (#0)
> GET / HTTP/1.1
> Host: localhost:3011
> User-Agent: curl/7.77.0
> Accept: */*
> 
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server

Additional context

We can open an SSH session through the tunnel using ssh ec2-user@localhost -p 2222 -I aws-linux-key.pem

What we also can do, is to local forward port 3011 using SSH via ssh -L 3011:localhost:3011 -N ec2-user@localhost -p 2222 -I aws-linux-key.pem through the localproxy tunnel.

If we curl now, it works:

curl 127.0.0.1:3011 -vvv
*   Trying 127.0.0.1:3011...
* Connected to 127.0.1.1 (127.0.0.1) port 3011 (#0)
> GET / HTTP/1.1
> Host: 127.0.0.1:3011
> User-Agent: curl/7.77.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
....
<!DOCTYPE html>...</html>

Environment (please complete the following information):

shangabl commented 2 years ago

Hello steffenstolze,

I was able to get this to work.

Destination:

./localproxy -d SSH=22,UI=3011 -v 6 -r us-east-1  -t <token>
[2022-05-18T23:09:40.034750]{29153}[warning] Found access token supplied via CLI arg. Consider using environment variable AWSIOT_TUNNEL_ACCESS_TOKEN instead
[2022-05-18T23:09:40.034854]{29153}[debug]   Detect port mapping configuration provided through CLI in destination mode:
[2022-05-18T23:09:40.034878]{29153}[debug]   ----------------------------------------------------------
[2022-05-18T23:09:40.034888]{29153}[debug]   UI = 3011
[2022-05-18T23:09:40.034900]{29153}[debug]   SSH = 22
[2022-05-18T23:09:40.034912]{29153}[debug]   ----------------------------------------------------------
[2022-05-18T23:09:40.034937]{29153}[debug]   /home/ec2-user/config does not exist!
[2022-05-18T23:09:40.034980]{29153}[info]    Starting proxy in destination mode
[2022-05-18T23:09:40.035000]{29153}[trace]   Setting up web socket...
[2022-05-18T23:09:40.036705]{29153}[trace]   Calling control_callback with type: websocket_stream_single_ssl_type
[2022-05-18T23:09:40.036763]{29153}[info]    Attempting to establish web socket connection with endpoint wss://data.tunneling.iot.us-east-1.amazonaws.com:443
[2022-05-18T23:09:40.036783]{29153}[trace]   Resolving proxy server host: data.tunneling.iot.us-east-1.amazonaws.com
[2022-05-18T23:09:40.043791]{29153}[debug]   Resolved proxy server IP: 54.204.110.197
[2022-05-18T23:09:40.043861]{29153}[trace]   Calling lowest_layer with type: websocket_stream_single_ssl_type
[2022-05-18T23:09:40.044400]{29153}[debug]   Connected successfully with proxy server
[2022-05-18T23:09:40.044428]{29153}[trace]   Calling lowest_layer with type: websocket_stream_single_ssl_type
[2022-05-18T23:09:40.044451]{29153}[trace]   Calling lowest_layer with type: websocket_stream_single_ssl_type
[2022-05-18T23:09:40.044464]{29153}[trace]   Performing SSL handshake with proxy server
[2022-05-18T23:09:40.044479]{29153}[debug]   SSL host verification is off
[2022-05-18T23:09:40.044491]{29153}[trace]   Calling next_layer().async_handshake with type: websocket_stream_single_ssl_type
[2022-05-18T23:09:40.047295]{29153}[debug]   Successfully completed SSL handshake with proxy server
[2022-05-18T23:09:40.047326]{29153}[trace]   Performing websocket handshake with proxy server
[2022-05-18T23:09:40.047351]{29153}[trace]   Calling async_handshake with type: websocket_stream_single_ssl_type
[2022-05-18T23:09:40.047420]{29153}[trace]   Web socket ugprade request(*not entirely final):
...

Source:

./localproxy -s SSH=2222,UI=3011 -v 6 -r us-east-1 -t <token>
[2022-05-18T23:05:14.034184]{905}[warning] Found access token supplied via CLI arg. Consider using environment variable AWSIOT_TUNNEL_ACCESS_TOKEN instead
[2022-05-18T23:05:14.034263]{905}[debug]   Detect port mapping configuration provided through CLI in source mode:
[2022-05-18T23:05:14.034276]{905}[debug]   ----------------------------------------------------------
[2022-05-18T23:05:14.034287]{905}[debug]   UI = 3011
[2022-05-18T23:05:14.034294]{905}[debug]   SSH = 2222
[2022-05-18T23:05:14.034302]{905}[debug]   ----------------------------------------------------------
[2022-05-18T23:05:14.034319]{905}[debug]   /tmp/localproxy/cmake-build-debug/bin/config does not exist!
[2022-05-18T23:05:14.034343]{905}[info]    Starting proxy in source mode
[2022-05-18T23:05:14.034354]{905}[trace]   Setting up web socket...
[2022-05-18T23:05:14.035725]{905}[trace]   Calling control_callback with type: websocket_stream_single_ssl_type
[2022-05-18T23:05:14.035771]{905}[info]    Attempting to establish web socket connection with endpoint wss://data.tunneling.iot.us-east-1.amazonaws.com:443
[2022-05-18T23:05:14.035783]{905}[trace]   Resolving proxy server host: data.tunneling.iot.us-east-1.amazonaws.com
[2022-05-18T23:05:14.043686]{905}[debug]   Resolved proxy server IP: 52.204.64.70
[2022-05-18T23:05:14.043702]{905}[trace]   Calling lowest_layer with type: websocket_stream_single_ssl_type
[2022-05-18T23:05:14.110584]{905}[debug]   Connected successfully with proxy server
[2022-05-18T23:05:14.110598]{905}[trace]   Calling lowest_layer with type: websocket_stream_single_ssl_type
[2022-05-18T23:05:14.110609]{905}[trace]   Calling lowest_layer with type: websocket_stream_single_ssl_type
[2022-05-18T23:05:14.110619]{905}[trace]   Performing SSL handshake with proxy server
[2022-05-18T23:05:14.110627]{905}[debug]   SSL host verification is off
[2022-05-18T23:05:14.110634]{905}[trace]   Calling next_layer().async_handshake with type: websocket_stream_single_ssl_type
[2022-05-18T23:05:14.246010]{905}[debug]   Successfully completed SSL handshake with proxy server
[2022-05-18T23:05:14.246038]{905}[trace]   Performing websocket handshake with proxy server
[2022-05-18T23:05:14.246053]{905}[trace]   Calling async_handshake with type: websocket_stream_single_ssl_type
[2022-05-18T23:05:14.246102]{905}[trace]   Web socket ugprade request(*not entirely final):
...
[2022-05-18T23:10:14.469017]{905}[trace]   Calling async_write with type: websocket_stream_single_ssl_type
[2022-05-18T23:10:14.469033]{905}[debug]   Write buffer has enough space, continue tcp read loop for UI
[2022-05-18T23:10:14.469042]{905}[trace]   Begin tcp socket read loop for service id : UI
[2022-05-18T23:10:14.469051]{905}[trace]   Sent 90 bytes over websocket for service id: UI
[2022-05-18T23:10:14.469060]{905}[trace]   Web socket write buffer drain for service id: UI
[2022-05-18T23:10:14.469068]{905}[trace]   web_socket_outgoing_message_queue is empty, no more messages to send.
[2022-05-18T23:10:14.647746]{905}[debug]   Starting web socket read loop continue reading...
[2022-05-18T23:10:14.647772]{905}[trace]   Calling async_read_some with type: websocket_stream_single_ssl_type
[2022-05-18T23:10:14.647788]{905}[trace]   write done service id UI
[2022-05-18T23:10:14.647795]{905}[trace]   Wrote 853 bytes to tcp socket
[2022-05-18T23:10:14.647803]{905}[trace]   TCP write buffer drain complete
[2022-05-18T23:10:14.647811]{905}[trace]   Done writing for: UI
[2022-05-18T23:10:14.647857]{905}[trace]   Reading from tcp socket for service id UI
[2022-05-18T23:10:14.647866]{905}[error]   Handling tcp socket error for service id: UI . error message:End of file
[2022-05-18T23:10:14.647876]{905}[info]    Disconnected from: 127.0.0.1:59052
[2022-05-18T23:10:14.647915]{905}[trace]   Web socket write buffer drain for service id: UI
[2022-05-18T23:10:14.647926]{905}[trace]   Reset stream for service id: UI
[2022-05-18T23:10:14.647939]{905}[trace]   Sending messages over web socket for service id: UI
[2022-05-18T23:10:14.647948]{905}[trace]   Current queue size: 0
[2022-05-18T23:10:14.647956]{905}[trace]   Put data 10 bytes into the web_socket_outgoing_message_queue for service id: UI
[2022-05-18T23:10:14.647964]{905}[trace]   Calling async_write with type: websocket_stream_single_ssl_type
[2022-05-18T23:10:14.647991]{905}[trace]   Sent 10 bytes over websocket for service id: UI
[2022-05-18T23:10:14.648002]{905}[trace]   Setting up tcp socket for service id: UI
[2022-05-18T23:10:14.648021]{905}[debug]   Resolving bind address host: localhost
[2022-05-18T23:10:14.648035]{905}[debug]   Port to connect 3011
[2022-05-18T23:10:14.648050]{905}[trace]   web_socket_outgoing_message_queue is empty, no more messages to send.
[2022-05-18T23:10:14.648081]{905}[debug]   Resolved bind IP: 127.0.0.1
[2022-05-18T23:10:14.648103]{905}[info]    Listening for new connection on port 3011
[2022-05-18T23:10:15.754274]{905}[debug]   socket port 3011
[2022-05-18T23:10:15.754310]{905}[debug]   endpoint mapping:
[2022-05-18T23:10:15.754321]{905}[debug]   UI = 3011
[2022-05-18T23:10:15.754328]{905}[debug]   SSH = 2222

curl localhost:3011
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Do you have the source logs? I can try to see if I there is anything there that explains the issue.

Thanks!