Kademlia

[yolanjoy]

Hi,

I coouldn't find a reference to get Kademlia working with mldonkey. I've enabled it but I see this message

Kademlia is enabled | Kademlia Connectivity: NOT enough online peers ,there maybe a problem with incoming udp packets

I understand it needs boot peers and there is an option to put a list of IP addresses in servers.ini. I don't know where it get those. Looking online , nodes.dat exist but I don't know how to use/load it in mldonkey.

Please advise, Thanks

[Testerprivate]

The network is decentralized. Here is an example from my servers.ini, but IPs change all the time. ` Kademlia = {

(* List of IP addresses to use to boot Kademlia networks *)

boot_peers = [ ("62.57.217.152", 1985); ("95.95.225.94", 22503); ("83.33.123.110", 14346); ("217.160.131.161", 15178); ("79.156.179.50", 4672); ("69.163.35.60", 4816); ("147.91.20.66", 18356); ("79.116.48.99", 16965); ("176.185.128.201", 5264); ("90.77.185.202", 60185); ("37.134.73.139", 4880); ("219.78.91.107", 24672); ("194.2.0.20", 53); ("88.5.45.153", 25020); ("90.20.210.98", 4672); ("189.100.246.23", 41441); ("94.177.167.104", 4953); ("94.34.119.100", 4511); ("89.67.166.67", 6693); ("95.21.51.65", 21464); ("212.233.42.166", 16519); ("86.153.20.223", 6772); ("81.36.96.175", 1493); ("46.117.117.7", 4672); ("84.77.152.221", 4672); ("98.128.144.15", 4670); ("88.26.86.24", 4321); ("99.232.32.218", 16903); ("81.36.194.115", 21185); ("91.163.0.14", 61390); ("89.186.145.226", 16897); ("85.215.165.180", 4672); ("220.125.49.62", 4803); ("93.42.74.82", 4672); ("80.103.210.13", 1590); ("78.198.183.112", 9561); ("135.23.188.133", 37136); ("80.129.159.146", 9672); ("91.117.192.160", 23138); ("185.74.223.229", 10932); ("88.21.169.160", 4672); ("79.8.134.164", 62296); ("88.18.52.244", 4672); ("2.138.150.107", 1828); ("84.77.66.63", 1400); ("191.190.199.209", 4672); ("213.195.164.128", 4677); ("95.23.24.216", 13926); ("37.120.167.200", 13451); ("88.12.9.189", 41556); ("88.12.251.245", 54616); ("88.14.254.237", 6969); ("85.127.56.38", 4672); ("79.116.156.3", 2291); ("82.53.14.178", 7981); ("159.196.88.17", 7810); ("93.55.93.148", 4672); ("37.134.232.102", 4672); ("188.78.86.160", 6999); ("85.57.209.70", 4672); ("77.27.92.1", 1961); ("188.127.166.68", 7346); ("37.59.62.216", 12156); ("79.255.7.183", 37742); ("81.56.13.170", 16394); ("62.99.9.185", 4535); ("81.35.105.66", 20040); ("86.235.26.69", 7121); ("193.84.70.82", 24197); ("88.6.57.97", 47949); ("79.144.96.173", 3333); ("95.250.69.133", 4237); ("173.243.42.52", 1178); ("1.36.15.121", 6709); ("78.112.172.240", 4672); ("85.84.66.24", 4672); ("192.234.79.143", 7230); ("87.15.177.6", 6419); ("159.196.89.240", 4127); ("90.153.25.173", 4672); ("90.77.130.171", 4670); ("90.73.217.130", 5742); ("59.10.44.158", 60205); ("62.37.140.235", 40002); ("93.209.52.155", 12226); ("114.35.54.30", 24115); ("37.134.140.147", 4672); ("87.223.147.85", 5070); ("152.171.143.182", 3612); ("88.17.204.207", 51281); ("213.152.161.35", 22792); ("88.24.184.190", 62575); ("104.244.78.5", 20849); ("185.121.35.49", 18715); ("89.154.205.30", 6306); ("219.248.117.24", 4713); ("83.38.100.23", 4511); ("190.220.163.92", 17971); ("85.193.204.171", 52218); ("88.172.200.225", 16334); ("104.160.44.228", 11031); ("185.50.119.172", 4671); ("93.34.147.242", 39207); ("81.32.118.211", 5060); ("175.213.87.189", 4803); ("91.115.77.49", 17966); ("88.26.150.130", 1985); ("87.5.206.20", 3737); ("68.100.212.168", 57717); ("83.51.140.201", 62110); ("79.200.153.208", 4672); ("81.33.126.84", 4672); ("211.200.65.244", 4803); ("62.57.241.133", 4672); ("83.39.156.47", 50000); ("79.151.138.140", 60287); ("83.42.115.83", 1985); ("79.117.29.174", 5128); ("91.117.236.11", 52109); ("81.33.228.194", 61117); ("134.249.120.23", 17011); ("200.103.175.100", 34663); ("85.54.140.253", 14927); ("93.176.161.183", 1985); ("82.84.17.183", 16626); ("93.43.222.52", 54496); ("81.217.14.69", 52446); ("37.11.100.238", 62482); ("88.6.126.87", 51567); ("2.37.63.48", 4672); ("77.26.5.87", 17275); ("81.169.224.76", 3725); ("94.73.36.171", 33964); ("89.130.231.70", 54520); ("46.6.8.71", 44672); ("79.158.149.49", 4676); ("79.24.105.235", 17597); ("79.159.227.205", 45982); ("50.127.112.178", 53682); ("186.158.200.84", 12885); ("174.102.65.145", 4672); ("207.188.180.192", 4672); ("79.154.47.191", 20278); ("37.134.58.51", 12307); ("87.216.65.82", 7346); ("85.137.151.68", 5886); ("2.239.16.85", 39878); ("82.58.88.73", 4672); ("87.13.230.13", 7040); ("82.60.148.151", 4672); ("93.46.119.24", 4672); ("211.218.157.56", 4672); ("91.116.168.125", 5556); ("109.227.152.26", 18880); ("84.75.67.109", 59388); ("83.43.153.23", 1485); ("213.194.152.92", 15155); ("121.129.204.27", 4803); ("87.20.12.206", 37094); ("90.25.9.5", 4672); ("81.43.187.253", 4672); ("188.26.198.147", 4672); ("95.19.211.59", 27540); ("188.134.72.54", 21809); ("46.136.127.238", 1985); ("31.150.248.253", 4672); ("2.138.150.107", 41550); ("79.116.0.185", 44044); ("80.137.78.206", 28268); ("93.149.40.130", 20241); ("88.218.11.2", 1985); ("81.109.121.90", 28886); ("2.140.75.109", 62111); ("109.227.148.164", 4572); ("37.35.162.237", 4672); ("79.9.138.156", 63370); ("79.151.139.181", 46631); ("83.200.109.102", 8330); ("82.58.160.206", 6419)] }` Of course, you should open ports, even if you are behind a NAT provider.

[yolanjoy]

Thank you for providing the list.

I suppose I only have to open Kad UDP port and not individual ports of each IPs above. Am I right? Do you have to update these IPs manually whenever there is a change? Where should I look for getting these IPs? Is there any URL for the list? When I loaded the nodes.dat it gave quite a few overnet peers. I don't think these are the kad boot peers but overnet which is inactive?

Thanks

[ogarcia]

I have the feeling that the Kad network is not working as it should. In my particular case it appears to be operational, with enough peers and a good list of boot servers and yet it doesn't seem to find anything on that network.

I have made the test of installing an eMule in a virtual machine, put to download a file that in MLDonkey appears to me without sources and in eMule at the moment I found someone in the Kad network that had it. Then I made the verification of downloading a file from myself and in eMule my MLDonkey client appears as not connected to the Kad network.

And as I said before, in the statistics section -> Kademilia it says: Kademlia is enabled | Kademlia Connectivity: Got enough online peers

[yolanjoy]

For me, MLDonkey shows this message even when I used the IPs given above.

Kademlia is enabled | Kademlia Connectivity: NOT enough online peers ,there maybe a problem with incoming udp packets

Maybe MLDonkey is not setup correctly in my case. Does it work for anyone at all?

@ygrek can you please confirm if Kad client in MLDonkey does work ? If it does, I will appreciate if you could provide some guidelines to set it up properly. Thanks

@ogarcia I wonder if you have tried to investigate the issue by enabling debug ? At least in your case Kad client can connect to the peers. Can you share your config ? I will investigate it as well. Thanks

[HinTak]

FWIW, did you just read the messge? "there maybe a problem with incoming udp packets". Are you getting any incoming UDP packets? Firewall / NAT settings etc.

[ogarcia]

@yolanjoy The truth is that there is nothing strange about my configuration:

 Kademlia = {

    (* Set this option to false if you don't want to receive new Overnet peers *)
   update_nodes = true

    (* The MD4 of the Overnet client *)
   md4 = "4608B2DBF75A72383EB74BB6BCC20B7B"

    (* Interval (in seconds) before republish files *)
   republish = 10800.

    (* allow extended search to search on overnet *)
   search_keyword = true

    (* Size of the filename storage used to answer queries *)
   store_size = 2000

    (* port for Kademlia *)
   port = 14868
}

The port is open in the firewall and the NAT rule is correctly configured in the router.

Full donkey.ini

```ini port = 14860 max_connected_servers = 3 reliable_sources = true ban_identity_thieves = true server_black_list = [] force_high_id = false force_client_high_id = false update_server_list_server = false update_server_list_server_met = true update_server_list_client = false max_sources_per_file = 5000 client_md4 = 4608B2DBF75A72383EB74BB6BCC20B7B client_private_key = "a key" enable_sui = true min_users_on_server = 0 max_published_files = 200 login = "" old_files = [] Overnet = { protocol_connectreply_version = 44 protocol_connect_version = 1044 update_nodes = true md4 = "4608B2DBF75A72383EB74BB6BCC20B7B" republish = 10800. search_keyword = true store_size = 2000 port = 12540 } Kademlia = { update_nodes = true md4 = "4608B2DBF75A72383EB74BB6BCC20B7B" republish = 10800. search_keyword = true store_size = 2000 port = 14868 } max_xs_packets = 30 check_client_connections_delay = 180. client_timeout = 40. keep_best_server = true connect_only_preferred_server = false max_walker_servers = 1 walker_server_lifetime = 300 log_clients_on_console = false propagate_sources = false keep_sources = true black_list = true port_black_list = [] queued_timeout = 1800. upload_timeout = 600. upload_lifetime = 90 upload_full_chunks = true upload_complete_chunks = false dynamic_upload_lifetime = false dynamic_upload_threshold = 10 connected_server_timeout = 1800. upload_power = 5 max_server_age = 2 remove_old_servers_delay = 900. min_left_servers = 20 servers_walking_period = 6 keep_cancelled_in_old_files = false keep_downloaded_in_old_files = false send_warning_messages = false ban_queue_jumpers = true use_server_ip = true ban_period = 1 good_client_rank = 500 options_version = 4 ```

NOTE: All MD4s are invented (not real)

[HinTak]

"incoming udp packets". You need to set up port forwarding at the router if you are behind NAT. Many consumer grade routers don't even give you the option to set up port forwarding at NAT'ed routers. (It is more a hosting feature for hosting for virtual server farms, which many consumer network explicitly disallows in their service agreement) . If you don't comprehend any of this, you probably have not set up your router correctly, or your router is of limited functionality and does not provide this option. Also, your ISP might block "incoming" UDP packets of a certain kind at their discretion. (To test that you'll basically need to move your computer to a different network...). Anyway, too much already.

[yolanjoy]

I've port forwarding setup on the router. I Installed eMule v070a and have kad is working within it. Having both running and testing out "search"on both. I get a feel that emule network is kinda flaky... probably same situation with Kad network I guess.

I think Mldonkey is much more flexible and have more features but many of them have not been documented ( at least I couldn't find ) .

[RigacciOrg]

I just installed mldonkey 3.2.1 compiled from sources, it seems that I cannot get responses from the Kademlia network. If I execute the kad_stats I get the Kademlia Connectivity: NOT enough online peers ,there maybe a problem with incoming udp packets response.

I executed a kad_load nodes.dat using a nodes.dat file copied from a well running aMule installation: I can see the outgoing UDP packets, but none of the hosts respond to me (the mldonkey host is a GNU/Linux machine with public IP address and the UDP port is open).

I also tried a kad_boot [ip] [port] using the IP address and port of the running aMule host; I can see the UDP packet arriving to the aMule host, but aMule does not respond either.

May be the outgoing UDP packets are not well formed?