facebook / wdt

Warp speed Data Transfer (WDT) is an embeddedable library (and command line tool) aiming to transfer data between 2 systems as fast as possible over multiple TCP paths.
https://www.facebook.com/WdtOpenSource
Other
2.86k stars 391 forks source link

wdt with 3 Mac computers with macOS Sierra 10.12.5 #160

Closed constantinevassil closed 7 years ago

constantinevassil commented 7 years ago

Here is my configuration: 3 Mac computers with macOS Sierra 10.12.5 Using mac AirPort for router. 1) iMac - wired LAN to AirPort- IP:10.0.1.9 2) MacBookPro - WiFi to AirPort- IP:10.0.1.10 3) Mac-mini - WiFi to AirPort- IP:10.0.1.7

Executing commands from 1): ssh-copy-id MyUserName@10.0.1.10 - OK ssh-copy-id MyUserName@10.0.1.7 - OK So there is no need for ssh password

This is OK:copy from MacBook wdt --encryption_type=none -directory /Users/MyName/Desktop/Test | ssh MyUserName@10.0.1.10 wdt --encryption_type=none -directory /Users/MyName/Documents/books -

I can see the files in Desktop/Test

This is NOT OK:copy from mac-mini wdt --encryption_type=none -directory /Users/MyName/Desktop/Test | ssh MyUserName@10.0.1.7 wdt --encryption_type=none -directory /Users/MyName/Documents/books -

I cannot see the files in Desktop/Test

Following: https://github.com/facebook/wdt/blob/master/build/BUILD.md CTEST_OUTPUT_ON_FAILURE=1 make test on 1) iMac 2) MacBookPro all tests passed, on 3) mac-mini CTEST_OUTPUT_ON_FAILURE=1 make test is returning the following: (stopped because it took very long) make_test.txt

wdt --encryption_type=none -directory /Users/MyName/Desktop/Test | ssh MyName@10.0.1.7 wdt --encryption_type=none -directory /Users/MyName/Documents/books - log: wdt.txt

ldemailly commented 7 years ago

your sender never seem to connect

can you run

wdt --encryption_type=none -directory /Users/MyName/Desktop/Test 2> /dev/null | ssh MyName@10.0.1.7 wdt --encryption_type=none -directory /Users/MyName/Documents/books -

(so we can see the sender side) and paste the output

constantinevassil commented 7 years ago

I got the following: bash: wdt: command not found

ldemailly commented 7 years ago

so please put the wdt binary in the path on your third computer

constantinevassil commented 7 years ago

from 10.0.1.9 if I do: ssh MyName@10.0.1.7 and then wdt wdt is executed, so it can be found

what do you mean by "put the wdt binary in the path on your third computer"?

if I do this: ssh MyName@10.0.1.7 then export I get ... declare -x PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin" ...

ls /usr/local/bin I see wdt so the 10.0.1.7 has wdt in the path

BUT CTEST_OUTPUT_ON_FAILURE=1 make test on 10.0.1.7 [ FAILED ] 3 tests, listed below: [ FAILED ] FdTest.FdTestBasic [ FAILED ] FdTest.FdTestBasicResumption [ FAILED ] DupSend.DuplicateSend

I still did sudo make install so I have it in the PATH

ldemailly commented 7 years ago

try putting the full path:

replace wdt by /usr/local/bin/wdt

constantinevassil commented 7 years ago

Thank you! This seems to be working. Let me test it more. What about failed tests? And why I should explicitly enter the path?

constantinevassil commented 7 years ago

I tested now on a server in data center. Got the following error: E0613 23:59:21.342402 38752256 ClientSocket.cpp:55] wdt> Failed getaddrinfo www.myserver.com , 59092 : 8 : nodename nor servname provided, or not known

tried: wdt --ipv4=true --encryption_type=none -directory /Users/MyName/Desktop/Test | ssh MyUserName@10.0.1.7 /usr/local/bin/wdt --ipv4=true --encryption_type=none -directory /Users/MyName/Documents/books - with same result

ldemailly commented 7 years ago

try using -hostname

see comments and last line of sample in:

https://github.com/facebook/wdt/wiki/Getting-Started-with-the-WDT-command-line#everything-in-1-command

constantinevassil commented 7 years ago

I see wdt -hostname fe80::202:c9ff:fe51:dd50 -directory destdir | ssh srchost wdt -directory srcdir - I got the ipv6 hostname from the ethernet configuration screen but it is still not working.

ldemailly commented 7 years ago

it doesn't have to be ipv6 - this is just an example - it's whichever ip your receiver is reachable under from your source server

constantinevassil commented 7 years ago

I tested with ipv4 also: wdt -hostname aaa.bbb.ccc.ddd -directory ~/Desktop/MyDir | ssh myusername@aa1.bb1.cc1.dd1 /usr/local/bin/wdt -directory ~/Desktop/MyDir -

here are the logs: server.txt

ldemailly commented 7 years ago

2 issues:

W0614 16:37:21.541368 264495104 SenderThread.cpp:93] wdt>   Thread[7, port: 60887]  Connection took 2 attempt(s) and 6.07 seconds. port 60887

it's not normal it took 6 second to connect - what's your ping time between your 2 machines ?

And:

try with --encryption_type=none first ?

if you have firewall setup you need to use ports that are open

also try with -num_ports=1 to simplify at first (you get better throughput with more obviously)

constantinevassil commented 7 years ago

ping time to this server is 0.544ms just tried with --encryption_type=none - got the same result added -num_ports=1 - got the same result

about the firewall - I didn't changed any setup in Application firewall - Allowed the pop up for wdt - that is all

ldemailly commented 7 years ago

can u send the log with 1 port and no encryption

also can you find out (use nc (netcat) for instance) why a tcp connection takes 6s ?

constantinevassil commented 7 years ago

I added the log: encryption_type_none_numports_1.txt

could you please send me example about how to use nc (netcat) to check the tcp connection?

ldemailly commented 7 years ago

You still have

E0615 20:41:24.104460 222154752 ClientSocket.cpp:55] wdt> Failed getaddrinfo hatc1.melissadata.com , 63497 : 8 : nodename nor servname provided, or not known

please use a working -hostname

for netcat you can do nc --help or google examples

ldemailly commented 7 years ago

@topconnector can you close the issue if it's resolved? thanks!