search

ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

renew certificate fails #63

Closed laubblatt closed 7 years ago

laubblatt commented 7 years ago

Hi, I installed synda freshly on Ubuntu and successfully downloaded CMIP5 output. Yet a few files where missing / are on the waiting list but the daemon is not running. trying: synda certificate renew Error occurs while renewing certificate (Certificate generation failure.)

How can I resolve this issue to get the missing files? Thanks! Maik

ghost commented 7 years ago

Hi, Can you provide the log files last lines ? There should be more info about the error in them (in /var/log/synda/sdt)

Regarding the daemon, it can be started using 

$ sudo service synda start

but it will not work as long as the certificate error is not fixed.

Regards, Jerome

laubblatt commented 7 years ago

Hi Jerome,

I installed synda locally. Here is the output of transfer.log (hoping that is the log file you are looking for). tail ~/sdt/log/transfer.log

2017-04-12 16:12:47,910 INFO SDDAEMON-001 Daemon starting ... 2017-04-12 16:12:47,910 INFO SDTSCHED-533 Connected to /home/mrenner/sdt/db/sdt.db 2017-04-12 16:12:47,910 INFO SDTSCHED-993 Starting watchdog.. 2017-04-12 16:12:48,125 INFO SDMYPROX-002 Renew certificate.. 2017-04-12 16:12:49,421 ERROR SYDLOGON-012 Error occured while retrieving certificate from myproxy server (Certificate generation failure.)

In the moment I can search for files using synda, but not get them. Best, Maik

ghost commented 7 years ago

Hi Maik, Here is another command to test certificate renewal. It should provide more info about the error

python ~/sdt/lib/python2.7/site-packages/myproxy/script.py logon -b -T -s pcmdi.llnl.gov -l foobar -o /tmp/cert.pem

(you need to replace 'foobar' user and 'pcmdi.llnl.gov' host accordingly to your openid)

Regards Jerome

laubblatt commented 7 years ago

Hi Jerome,

still no success.

python ~/sdt/lib/python2.7/site-packages/myproxy/script.py logon -b -T -s esgf-data.dkrz.de -l mrenner -o /tmp/cert.pem

`Enter password for user 'mrenner' on MyProxy server 'esgf-data.dkrz.de': Error retrieving credentials: Certificate generation failure.

Best, Maik

ghost commented 7 years ago

Hi Maik,

Given the last test, I think the error is on the remote (server) side.

You can ask the ESGF team by sending a message to "esgf-user@lists.llnl.gov" mailing list with the full command line and output of last test (script.py logon ..)

Regards, Jerome

beyerle commented 7 years ago

Hi,

I still see the same error. Has somebody report this to esgf-user@lists.llnl.gov?

Cheers,

Urs

On 04/13/2017 10:08 AM, jripsl wrote:

Hi Maik,

Given the last test, I think the error is on the remote (server) side.

You can check that by asking ESGF team, by sending a message to "esgf-user@lists.llnl.gov mailto:esgf-user@lists.llnl.gov" mailing list with the full command line and output of previous test (script.py logon ..)

Regards, Jerome

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Prodiguer/synda/issues/63#issuecomment-293822151, or mute the thread https://github.com/notifications/unsubscribe-auth/AJx7IUtmXhFU18zgn8tPce4_D0DJMAveks5rvdf5gaJpZM4M7iuX.

SebastienDenvil commented 7 years ago

Here is DKRZ's answer that has been given on esgf-user@lists.llnl.gov

Indeed Synda is using the Python MyProxyClient.

Hi Maik, hi Jerome,

We recently installed a couple of CentOS patches on esgf-data that have broken MyProxyLogon using the Python MyProxyClient.

So in case Synda uses this module as well this might be the reason.

In that case changing

MESSAGE_DIGEST_TYPE = "md5" to MESSAGE_DIGEST_TYPE = "sha256"

in myproxy/client.py should do the trick.

Lukasz has already sent this fix to Phil so I hope we will have a new version of MyProxyClient soon.

Thanks, Katharina

ghost commented 7 years ago

Hi,

After patching myproxy/client.py as described in the previous message, you may face error below

"ValueError: No such digest method"

In this case, the problem can be fixed by installing myproxyclient 1.4.4 version

~/sdt/bin/pip install myproxyclient==1.4.4

Regards Jerome

laubblatt commented 7 years ago

Hi Jerome,

in the esgf email-list esgf-user@lists.llnl.gov Phil announced a new release of MyProxyClient:

I’ve made a new release of MyProxyClient - 2.0.1 including a patch to use the more up to date hashing algorithm. Thanks for spotting this issue Lukasz! Cheers, Phil

So I just updated MyProxyClient using: ~/sdt/bin/pip install myproxyclient==2.0.1

and changed in myproxy/client.py: MESSAGE_DIGEST_TYPE = "md5" to MESSAGE_DIGEST_TYPE = "sha256"

And now it works like a charm! Thanks to Jerome, Katharina and Phil! Best Maik

@jripsl Finally will these changes be updated in the next version of synda?

ghost commented 7 years ago

Hi Maik,

Good to hear it's working.

Regarding synda 3.8+ new installation, it should work as is, as the myproxyclient version is not hardcoded in synda installation script (i.e. the 2.0.1 version will be installed). Regarding .rpm and .deb synda 3.8, I added a manual patch in the installation procedures. Regarding .rpm and .deb synda 3.9+, packages will be generated with the correct myproxyclient version (2.0.1).

Thanks, Jerome