richardfan1126 / nitro-enclave-python-demo

Using Python to implement basic features on AWS Nitro Enclaves
Apache License 2.0
32 stars 9 forks source link

Trouble to get communication via vsock-proxy to work #4

Closed kmac8361 closed 3 years ago

kmac8361 commented 3 years ago

Hi Richard, great demo code for http proxy and attestation...

I got all your code to work successfully. But setting up to a different external server instead of KMS I cannot get to work.

I've tried all sorts of combinations for IP address and URL but cannot get communication to work to a server on other side of vsock-proxy. Any help or suggestion much appreciated.

My server uses port 9443 and is running at public IP address 3.80.106.151 (AWS instance). I completely opened up security group and verified I can run outside Nitro enclave with no issue.

Here are contents of my run script:

Assign an IP address to local loopback

ifconfig lo 127.0.0.1

# Add a hosts record, pointing API endpoint to local loopback
# Note: I tried putting full URL path but no difference
# echo "127.0.0.1   https://3.80.106.151:9443/api/v1/hecompute/bfv" >> /etc/hosts
echo "127.0.0.1   3.80.106.151" >> /etc/hosts

# Run traffic forwarder in background and start the server
nohup python3 /palisade-release/traffic-forwarder.py 9443 3 8000 &
# Note: I tried using local port 443 but no difference.  I believe it should be 9443 same as remote server side port
# nohup python3 /palisade-release/traffic-forwarder.py 443 3 8000 &
python3 /palisade-release/server.py

My server will call a bash script which performs a 'curl' command to remote server like below: $ curl -s -k -X POST -H "Content-Type: application/json" -d "@/tmp/runcompute.data" https://3.80.106.151:9443/api/v1/hecompute/bfv

And I run vsock proxy in another window (I use 9443 port which server is listening): $ vsock-proxy --config /etc/vsock_proxy/vsock-proxy.yaml 8000 3.80.106.151 9443 NOTE: At first I got 'Not allowed to use IP address'. But I added to YAML and no longer get that error allowlist:

I check the return status of curl command and it always returns '7' which is cannot connect to server host. I also attach to server docker container in another window and can see no communication is happening. The same bash script running curl command does work fine outside of Nitro enclave.

And I can see my client running script just fine.... just no response $ python3 client.py 54

  Run Compute Operation: bfv add - Input Vector 1: {1 2 3 4 5}
  Run Compute Operation: bfv add - Input Vector 2: {6 7 8 9 10}

  Run Compute Operation: bfv add - Keys Generated!!

  Run Compute Operation: bfv add - Inputs Encrypted!!

  Run Compute Operation: bfv add - Command: [curl -s -k -X POST -H "Content-Type: application/json" -d 
  "@/tmp/runcompute.data" https://3.80.106.151:9443/api/v1/hecompute/bfv]

  Run Compute Operation: bfv add - Wait for Response

  Curl command result....
  7

  ERROR: Run Compute Operation: bfv add - Failure

Thanks in advance for any help Kurt

kmac8361 commented 3 years ago

I realized my /etc/hosts entry should probably have DNS name of AWS server instance I was trying to communicate to. I updated to below but it still made no difference. Still get curl return code 7 and cannot establish communication to server.

echo "127.0.0.1 ec2-3-80-106-151.compute-1.amazonaws.com" >> /etc/hosts

kmac8361 commented 3 years ago

Resolved issue. I didn't realize I needed to update client.py file as well. Code worked after updating client.