search

perfsonar / mesh-config

Centralized configuration framework for measurement points and GUIs
Apache License 2.0
2 stars 0 forks source link

IPv6 connection refused errors in meshconfig-agent 4.0.1 #104

Open akurov opened 6 years ago

akurov commented 6 years ago

Hi all,

I added IPv6 Throughput tests in my perfsonar host 2001:db8::1 but no one of them running.

$ rpm -q perfsonar-meshconfig-agent perfsonar-meshconfig-agent-4.0.1-1.el7.centos.noarch

There are such errors in /var/log/perfsonar/meshconfig-agent.log

2017/11/10 17:54:32 (1392) WARN> perfsonar_meshconfig_agent:430 main:: - Problem adding test throughput(2001:db8::1->psmp-gn-bw-01-fra-de.geant.net), continuing with rest of config: 500 Can't connect to 2001:db8::1:443: Can't connect to 2001:db8::1:443

Connection refused at /usr/share/perl5/LWP/Protocol/http.pm line 51.

But telnet can connect to 2001:db8::1 port 443.

Also I can run command-line bwctl or owping tests with IPv6 hosts that I configured for scheduled tests.

arlake228 commented 6 years ago

What do you get when you run pscheduler task --debug throughput --source 2001:db8::1 --dest psmp-gn-bw-01-fra-de.geant.net? If that fails its possible youa re hitting a bug fixed in the forthcoming 4.0.2.

Also, is that actually your IPv6 address you are using or did you just substitute that in place of the real one for the sake of reporting? That's a reserved address so more shocked it works at all if that is indeed your address.

akurov commented 6 years ago

I replaced real address with 2001:db8::1 for reporting here.

$ pscheduler task --debug throughput --source 2001:db8::1 --dest psmp-gn-bw-01-fra-de.geant.net 2017-11-16T22:01:26 Debug signal ignored; already not debugging 2017-11-16T22:01:26 Debug discontinued 2017-11-16T22:01:26 Assistance is from localhost 2017-11-16T22:01:26 Forcing default slip of PT5M 2017-11-16T22:01:26 Converting to spec via https://localhost/pscheduler/tests/throughput/spec Submitting task... 2017-11-16T22:01:27 Fetching participant list 2017-11-16T22:01:27 Spec is: {"dest": "psmp-gn-bw-01-fra-de.geant.net", "source": "2001:db8::1", "schema": 1} 2017-11-16T22:01:27 Got participants: {u'participants': [u'2001:db8::1', u'psmp-gn-bw-01-fra-de.geant.net']} 2017-11-16T22:01:27 Lead is 2001:db8::1 2017-11-16T22:01:27 Pinging https://[2001:db8::1]/pscheduler/ 2017-11-16T22:01:27 2001:db8::1 is up 2017-11-16T22:01:27 Posting task to https://[2001:db8::1]/pscheduler/tasks 2017-11-16T22:01:27 Data is {"test": {"type": "throughput", "spec": {"dest": "psmp-gn-bw-01-fra-de.geant.net", "source": "2001:db8::1", "schema": 1}}, "schedule": {"slip": "PT5M"}, "schema": 1} Task URL: https://[2001:db8::1]/pscheduler/tasks/25e28ac5-0eea-4775-b7d7-70b3b73bad25 2017-11-16T22:01:35 Posted https://[2001:db8::1]/pscheduler/tasks/25e28ac5-0eea-4775-b7d7-70b3b73bad25 Running with tool 'iperf3' Fetching first run... 2017-11-16T22:01:35 Fetching https://[2001:db8::1]/pscheduler/tasks/25e28ac5-0eea-4775-b7d7-70b3b73bad25/runs/first 2017-11-16T22:01:37 Handing off: pscheduler watch --format text/plain --debug 'https://[2001:db8::1]/pscheduler/tasks/25e28ac5-0eea-4775-b7d7-70b3b73bad25' 2017-11-16T22:01:38 Debug signal ignored; already not debugging 2017-11-16T22:01:38 Debug discontinued 2017-11-16T22:01:38 Fetching https://[2001:db8::1]/pscheduler/tasks/25e28ac5-0eea-4775-b7d7-70b3b73bad25

Next scheduled run: https://[2001:db8::1]/pscheduler/tasks/25e28ac5-0eea-4775-b7d7-70b3b73bad25/runs/5cb8c560-6cf8-4618-81fe-f005ab119823 Starts 2017-11-16T22:01:45+02:00 (~6 seconds) Ends 2017-11-16T22:02:04+02:00 (~18 seconds) Waiting for result...

Summary Interval Throughput Retransmits
0.0 - 10.0 806.62 Mbps 1

No further runs scheduled.

I added 9 IPv6 Throughput scheduled tests but results of only two of them shown (both in one direction only), other tests including psmp-gn-bw-01-fra-de.geant.net still missing. Most of configured hosts do not appear in /var/log/pscheduler/pscheduler.log, only two of running tests there.

akurov commented 6 years ago

Problem still in perfSONAR 4.0.2.

akurov commented 6 years ago

Hi, I was hoping that setting up reverse DNS can fix errors "Can't connect to 2001:db8::1:443" but errors still occur after setting up DNS. Maybe something wrong with perfSONAR installation (it was installed in minimal CentOS 7 with yum install perfsonar-toolkit)? Is there any hints what should be checked in this installation?

akurov commented 6 years ago

Hi,

Yet one problem is error "Error loading test listing: Internal Server Error" when I open perfSONAR web interface via IPv6 address http://[2001:db8::1]/toolkit/ but no such problem with IPv4.

arlake228 commented 6 years ago

I was finally able to recreate this. This appears to be a bug in the perl module LWP (or maybe a lower-level library it uses)when creating IPv6 HTTP connections. The perfSONAR code is properly bracketing the URL but when it hands it to LWP, the brackets get stripped and the connection fails. This is not the first time a version of LWP has given us IPv6 headaches, so I put together a change that replaces it with a newer library that handles IPv6 properly. Its a big enough change that we likely won't release the change until 4.1 so we have time to test whether there are any unintended side effects. See perfsonar/psconfig#2 for a discussion on this issue (though i think I hit the major points here) and references to the commits that fixes it.