search

Parsl / parsl

Parsl - a Python parallel scripting library
http://parsl-project.org
Apache License 2.0
506 stars 195 forks source link

Reduce verbosity of address helper warnings #2309

Open benclifford opened 2 years ago

benclifford commented 2 years ago

Michael Tynes 22:39 Hi All, I was working with Parsl today and encountered another situation that leads to >100 line error messages when ultimately the code still runs. In this case I was running Parsl from behind an outbound web proxy and Parsl could not see out to the www, leading to the error messages. This was OK for my use case, since all of my resources were on the same network. My primary objective is simply to make you aware of this case. I am also curious if there is a way to configure Parsl to see out to the www from behind such a proxy, but there is no urgency for this. I have attached the error messages as a text file, with the code that generated it marked with >>> , including version info. tynesParslErrors-2022-06-13

import parsl parsl.version '1.3.0-dev' Click to expand in-line (198 lines)

Logan Ward 22:42 I chatted to Michael about this earlier today, and suggested he post the message here. Is there already an Issue and/or policy around Parsl's aggressive error messages?

Yadu Babuji 22:47 Hi Michael, the default mechanism that parsl uses to find the address of the host on which it runs is based on trial and error using a bunch of different methods. It looks like all of those methods failed. In these situations, you can explicitly set the address to work around the limitations of the auto-detection code. Basically you have to do this -> Config( executors = [ HighThroughputExecutor ( address = "" ) ] ) 22:47 Since everything is on the same machine, you could try seting address="127.0.0.1" 22:48 @Logan Ward Thanks for directing Michael to the channel :slightly_smiling_face:

Logan Ward 22:50 Is there any thought on toning down these error messages? I doubt Michael's the only user to be mislead into thinking Parsl is not working correctly by these log lines

Yadu Babuji 22:51 I think that;ll be useful to do. The full trace doesn't help anyone, it would be just as useful to summarize and report that all methods failed to find a viable address. 22:51 I'll add an issue on this

benclifford commented 2 years ago

I am also curious if there is a way to configure Parsl to see out to the www from behind such a proxy, but there is no urgency for this.

I think this would be the wrong thing to do: the code in question is trying to figure out IP addresses that are usable to contact the submitting host, and routing via a proxy would give the address of the proxy. This would be the same unusable address as when sitting behind IP-layer NAT.