Problem downloading evio on cdaql1

[MarkKJones]

With the changes to podd/site_scons/construct.py to use curl to download the evio library, scons fails on cdaql1. When running curl , it can not connect to github.com .

[brash99]

Maybe it’s a firewall/https proxy issue?

Incidentally, is there any way to log into cdaql1 from offsite now? I mean, via some other CUE machine like jlabl1 or so? Given the issues that have come up recently with installing on the counting house machines, I wanted to be able to test things there.

Cheers, E.

On Oct 9, 2017, at 11:18 AM, Mark K Jones notifications@github.com wrote:

With the changes to podd/site_scons/construct.py to use curl to download the evio library, scons fails on cdaql1. When running curl , it can not connect to github.com .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_JeffersonLab_hcana_issues_268&d=DwMCaQ&c=lz9TcOasaINaaC3U7FbMev2lsutwpI4--09aP8Lu18s&r=QQI4wgZ48DvzGull4QAPAA&m=Ms_A6abRTBV0uHEQdSVO7v-9DkjSPDVNwuVcUIgKEUg&s=Etks5GLW4zZuD2QsR0MD9MlXI-jnek19Dn7IKICGQ9k&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AE1Pu-5FZIEH9-2DKytRnesHY00llGIbUs8Mks5sqjlAgaJpZM4Pypka&d=DwMCaQ&c=lz9TcOasaINaaC3U7FbMev2lsutwpI4--09aP8Lu18s&r=QQI4wgZ48DvzGull4QAPAA&m=Ms_A6abRTBV0uHEQdSVO7v-9DkjSPDVNwuVcUIgKEUg&s=t6z_Odg70sM1cq_pA5B3K6p7-lnsbikSCr2Q00hwhSM&e=.

[sawatzky]

Actually wget https://github.com/JeffersonLab/hallac_evio/archive/evio-4.4.6.tar.gz is timing out for me on cdaq@cdaql1 too, so it's not just curl.

FWIW, I was able to successfully build hcana from scratch on cdaq@cdaql1 as of Oct 2... I think that was after the switch to curl, so something else may be going on here.

I'll follow up with CNI...

-- Brad

On Mon, 09 Oct 2017, Mark K Jones wrote:

With the changes to podd/site_scons/construct.py to use curl to download the evio library, scons fails on cdaql1. When running curl , it can not connect to github.com .

-- Brad Sawatzky, PhD brads@jlab.org -<>- Jefferson Lab / Hall C / C111 Ph: 757-269-5947 -<>- Fax: 757-269-5235 -<>- Pager: brads-page@jlab.org The most exciting phrase to hear in science, the one that heralds new discoveries, is not "Eureka!" but "That's funny..." -- Isaac Asimov

[sawatzky]

You just ssh to hallgw first, then jump from there. (Just the same as you do on-site.)

-- Brad

On Mon, 09 Oct 2017, brash99 wrote:

Maybe it’s a firewall/https proxy issue?

Incidentally, is there any way to log into cdaql1 from offsite now? I mean, via some other CUE machine like jlabl1 or so? Given the issues that have come up recently with installing on the counting house machines, I wanted to be able to test things there.

Cheers, E.

[sawatzky]

I realized that if it was a global proxy, then 'git clone' would fail... It is proxy-related though. If I set the appropriate shell variable then both wget and curl work. (Ex.'https_proxy=jprox.jlab.org:8081')

Our /home/cdaq/.gitconfig sets that proxy automatically which is why the 'git clone', etc commands work. I don't know why curl and wget are now broken... Could a change to Sconstruct change whether/how that proxy envar propagates to child shells?

-- Brad

% wget % https://github.com/JeffersonLab/hallac_evio/archive/evio-4.4.6.tar.gz

On Mon, 09 Oct 2017, Brad Sawatzky wrote:

Actually wget https://github.com/JeffersonLab/hallac_evio/archive/evio-4.4.6.tar.gz is timing out for me on cdaq@cdaql1 too, so it's not just curl.

FWIW, I was able to successfully build hcana from scratch on cdaq@cdaql1 as of Oct 2... I think that was after the switch to curl, so something else may be going on here.

I'll follow up with CNI...

-- Brad

On Mon, 09 Oct 2017, Mark K Jones wrote:

With the changes to podd/site_scons/construct.py to use curl to download the evio library, scons fails on cdaql1. When running curl , it can not connect to github.com .

-- Brad Sawatzky, PhD brads@jlab.org -<>- Jefferson Lab / Hall C / C111 Ph: 757-269-5947 -<>- Fax: 757-269-5235 -<>- Pager: brads-page@jlab.org The most exciting phrase to hear in science, the one that heralds new discoveries, is not "Eureka!" but "That's funny..." -- Isaac Asimov

[hansenjo]

Downloading EVIO in the SCons build with curl works perfectly fine with me on the Hall A counting house systems.

My environment defines

ole@aonl1 > env | grep proxy http_proxy=http://jprox.jlab.org:8082 ftp_proxy=http://jprox.jlab.org:8082 https_proxy=http://jprox.jlab.org:8082

Ole

On 10/09/2017 12:30 PM, Brad Sawatzky wrote:

Actually wget https://github.com/JeffersonLab/hallac_evio/archive/evio-4.4.6.tar.gz is timing out for me on cdaq@cdaql1 too, so it's not just curl.

FWIW, I was able to successfully build hcana from scratch on cdaq@cdaql1 as of Oct 2... I think that was after the switch to curl, so something else may be going on here.

I'll follow up with CNI...

[MarkKJones]

On cdaql1 , after doing setenv https_proxy jprox.jlab.org:8081

then scons run fine.

[hansenjo]

So, as suspected, it's merely a firewall issue.

In the Hall A counting house, we have EVIO installed centrally, so no downloads are normally necessary. That saves students from potential pitfalls with the networking. All they have to do for a private installation of the analysis software is to fetch a tarball from the hallaweb server, which you can do without going through a proxy. They can get the analysis software to work without having to define extra environment variables.

Ole

On 10/09/2017 01:17 PM, Mark K Jones wrote:

On cdaql1 , after doing setenv https_proxy jprox.jlab.or:8081

then scons run fine.

[sawatzky]

Yes, that's right. The question in my last email was what changed?

The http*_proxy variables have not been in the general cdaq environment for a while. (IIRC, they broke something else (non analyzer related) in a weird way -- can't remember what now though...)

So, that left something in the scons config as most likely.

-- Brad

On Mon, 09 Oct 2017, Ole Hansen wrote:

So, as suspected, it's merely a firewall issue.

In the Hall A counting house, we have EVIO installed centrally, so no downloads are normally necessary. That saves students from potential pitfalls with the networking. All they have to do for a private installation of the analysis software is to fetch a tarball from the hallaweb server, which you can do without going through a proxy. They can get the analysis software to work without having to define extra environment variables.

Ole

On 10/09/2017 01:17 PM, Mark K Jones wrote:

On cdaql1 , after doing setenv https_proxy jprox.jlab.or:8081

then scons run fine.

-- Brad Sawatzky, PhD brads@jlab.org -<>- Jefferson Lab / Hall C / C111 Ph: 757-269-5947 -<>- Fax: 757-269-5235 -<>- Pager: brads-page@jlab.org The most exciting phrase to hear in science, the one that heralds new discoveries, is not "Eureka!" but "That's funny..." -- Isaac Asimov

[MarkKJones]

I only saw scons work for hcana when it was getting the evio library using wget from drupal. I think this only came up when scons started to get the evio library from github. So it seems reasonable that the proxy needs to be set since we are using "curl" and not "git clone". Could we use "git clone" to get the evio library?

[hansenjo]

Aha. Yes, accessing a JLab-internal server works without the proxy, so that explains it. What changed is that we now access an external server.

If you want to use "git clone", you require everyone to have git installed in order to build the analyzer. Admittedly, that's probably the case nowadays, but on older Macs it wasn't, for example.

In any case, why is the ability to download EVIO from the HAll C counting house so important. Why not just install EVIO centrally? Saves disk space, too.

Ole

[sawatzky]

That must be it. The http_proxy variables have been added to the cdaq shell environment now. Unless we hit a problem with something else, that works for me.

My vote is still to make evio a submodule dependency. I'm happy to listen, but I don't see the advantage in it being an external library either. Is it mostly to avoid a git dependency if podd is built from the tarball?

-- Brad

On Mon, 09 Oct 2017, Mark K Jones wrote:

I only saw scons work for hcana when it was getting the evio library using wget from drupal. I think this only came up when scons started to get the evio library from github. So it seems reasonable that the proxy needs to be set since we are using "curl" and not "git clone". Could we use "git clone" to get the evio library?

-- Brad Sawatzky, PhD brads@jlab.org -<>- Jefferson Lab / Hall C / C111 Ph: 757-269-5947 -<>- Fax: 757-269-5235 -<>- Pager: brads-page@jlab.org The most exciting phrase to hear in science, the one that heralds new discoveries, is not "Eureka!" but "That's funny..." -- Isaac Asimov