search

atsign-foundation / noports

Connect to any device with no external listening ports open
https://noports.com
BSD 3-Clause "New" or "Revised" License
267 stars 15 forks source link

feat: C daemon #1084

Closed XavierChanth closed 3 months ago

XavierChanth commented 4 months ago

- What I did

- How I did it

- How to verify it

- Description for the changelog feat: C daemon

gkc commented 4 months ago

Getting some timeout failures when only running the C daemons in the tests - e.g. https://github.com/atsign-foundation/noports/actions/runs/9317363043/job/25647499506

@XavierChanth Absence of timestamps in the C SDK's logs makes it difficult to establish precise sequence of events ... can timestamps be added please?

gkc commented 4 months ago

@XavierChanth fixed the problem with the C daemon auth upon startup, was a bug in start_daemons.sh which wasn't clearing the 'extraFlags'

gkc commented 4 months ago

@XavierChanth this run shows what looks like some buggy behaviour in the C sdk in the socket handling ... see lines 498, 509, 1024, 1035 in the gh action output

I'm attaching all of the client and daemon log files from the test run so you can look in more detail d16cfd88.tgz

XavierChanth commented 4 months ago

@XavierChanth this run shows what looks like some buggy behaviour in the C sdk in the socket handling ... see lines 498, 509, 1024, 1035 in the gh action output

I'm attaching all of the client and daemon log files from the test run so you can look in more detail d16cfd88.tgz

Thanks, will take a look

gkc commented 4 months ago

@XavierChanth have you figured out why the e2e tests are still failing with the c daemon?

XavierChanth commented 4 months ago

@XavierChanth have you figured out why the e2e tests are still failing with the c daemon?

No, I'm not able to repro in my fedora environment... I was focused on network recovery today, but I can look into this issue with the tests on Monday.

My client was working really well in my environment, so long as it doesn't lose a network connection...

gkc commented 3 months ago

@XavierChanth have you figured out why the e2e tests are still failing with the c daemon?

No, I'm not able to repro in my fedora environment... I was focused on network recovery today, but I can look into this issue with the tests on Monday.

My client was working really well in my environment, so long as it doesn't lose a network connection...

Having stared at the logs for a bit, I can see that the C daemon on Ubuntu is consistently missing the last character of the base64-encoded shared_key e.g.

[DEBG] 2024-06-09 09:26:25.930165 | connection |    SENT: "lookup:shared_key@apricoteventual"
[DEBG] 2024-06-09 09:26:25.960517 | connection |    RECV: "data:W+I8EffHNaYjaaRagd9LF0YHZev8wZ0JoAB3IgPe1Zy/L18BYYugObYKukhSt9vvhDqH8dwDpilF14NNoGMI0P6G3o0RLJ7VdhBlTjwKw7v2wAJQDYCJ2khNgFXZG3HOzaOonnEzYjMLoWjBNFblSNu/0Qkodd+qZLpLuoBIHTG8c+PUvXzsWIqfv0FV8xvQehUWZ/5zn85U6AsTisWTADYPeN0zx3FV2oKkK7eJxHagJiDH/OP11OvVq9638iiwfZy3TIjPNtBjtH8Ljl8zOxgqat9gha9Z16DAFyLBKui0xeJCh6g1co7nkP2FUC7yboKrIicRDMH2biS8Cf12oQ="
[ERRO] 2024-06-09 09:26:25.971536 | atclient_monitor | Invalid shared encryption key length was decoded.

Using exactly the same atSigns running locally on my mac, it consistently does NOT miss the last character:

[DEBG] 2024-06-09 09:18:50.478604 | connection |    SENT: "lookup:shared_key@apricoteventual"
[DEBG] 2024-06-09 09:18:50.583750 | connection |    RECV: "data:W+I8EffHNaYjaaRagd9LF0YHZev8wZ0JoAB3IgPe1Zy/L18BYYugObYKukhSt9vvhDqH8dwDpilF14NNoGMI0P6G3o0RLJ7VdhBlTjwKw7v2wAJQDYCJ2khNgFXZG3HOzaOonnEzYjMLoWjBNFblSNu/0Qkodd+qZLpLuoBIHTG8c+PUvXzsWIqfv0FV8xvQehUWZ/5zn85U6AsTisWTADYPeN0zx3FV2oKkK7eJxHagJiDH/OP11OvVq9638iiwfZy3TIjPNtBjtH8Ljl8zOxgqat9gha9Z16DAFyLBKui0xeJCh6g1co7nkP2FUC7yboKrIicRDMH2biS8Cf12oQ=="
XavierChanth commented 3 months ago

@cpswan FYI, we are getting an off by 1 error when using the cc compiler.

The scary thing is that the off by 1 is the output of a base64 decoding... I already said this earlier when I first encountered this issue, but... what.

XavierChanth commented 3 months ago

Jeremy is doing a manual test, once that is done please merge @JeremyTubongbanua

XavierChanth commented 3 months ago

Once this is merged I will tag c0.1.0

JeremyTubongbanua commented 3 months ago

Confirmed working manual test

image

Using v5.3.0 client (@soccer0) and C Daemon device (@soccer99)

JeremyTubongbanua commented 3 months ago

Some issues I ran into

1. Building C Daemon SSHNPD

Ubuntu installed cmake 3.22 which was not sufficient for our builds, so I had to find a way to install cmake 3.29.

To install cmake 3.29, I used pip by:

sudo apt-get install -y python3-pip
pip install cmake==3.29
pip install -v | grep cmake

# I find `/jeremyvps/.local/lib/python3.10/site-packages/cmake/data/bin` as the bin folder for the cmake binary that pip installed for me

nano ~/.bashrc

# Add line `export PATH="$PATH:/jeremyvps/.local/lib/python3.10/site-packages/cmake/data/bin` to my ~/.bashrc

# Update bash
source ~/.bashrc

cmake --version # Should be 3.29

2. v5.3.0 client-side sshnp key exchange errors

Would run into this error:

v5.3.0 client

./sshnp -f @soccer0 -t @soccer99 -h @rv_eu -i ~/.ssh/id_ed25519 -d vps
2024-06-12 13:38:34.291944 : Sending daemon feature check request
2024-06-12 13:38:34.292038 : Resolving remote username for user session
2024-06-12 13:38:34.714188 : Resolving remote username for tunnel session
2024-06-12 13:38:34.714918 : Fetching host and port from srvd
2024-06-12 13:38:35.814524 : Received host and port from srvd
2024-06-12 13:38:35.814611 : Waiting for daemon feature check response
2024-06-12 13:38:35.814624 : Received daemon feature check response
2024-06-12 13:38:35.816947 : Required daemon features are supported
2024-06-12 13:38:35.949207 : Sending session request to the device daemon
2024-06-12 13:38:36.047653 : Waiting for response from the device daemon
2024-06-12 13:38:36.503629 : Received response from the device daemon
2024-06-12 13:38:36.508431 : Creating connection to socket rendezvous
2024-06-12 13:38:36.633964 : Starting user session
kex_exchange_identification: Connection closed by remote host
Connection closed by 127.0.0.1 port 54167

After a couple of retries, to different rendezvous servers (am, eu, ap), it would work eventually.

Example of it working:

./sshnp -f @soccer0 -t @soccer99 -h @rv_am -i ~/.ssh/id_ed25519 -d vps
2024-06-12 13:43:13.994642 : Sending daemon feature check request
2024-06-12 13:43:13.994709 : Resolving remote username for user session
2024-06-12 13:43:14.404192 : Resolving remote username for tunnel session
2024-06-12 13:43:14.404510 : Fetching host and port from srvd
2024-06-12 13:43:15.780340 : Received host and port from srvd
2024-06-12 13:43:15.780446 : Waiting for daemon feature check response
2024-06-12 13:43:15.780458 : Received daemon feature check response
2024-06-12 13:43:15.784427 : Required daemon features are supported
2024-06-12 13:43:15.855798 : Sending session request to the device daemon
2024-06-12 13:43:15.962725 : Waiting for response from the device daemon
2024-06-12 13:43:16.575239 : Received response from the device daemon
2024-06-12 13:43:16.580149 : Creating connection to socket rendezvous
2024-06-12 13:43:16.706060 : Starting user session
Warning: Permanently added '[localhost]:54308' (ED25519) to the list of known hosts.