search

radical-cybertools / radical.saga

A Light-Weight Access Layer for Distributed Computing Infrastructure and Reference Implementation of the SAGA Python Language Bindings.
http://radical-cybertools.github.io/saga-python/
Other
83 stars 34 forks source link

File permission err for non-default .radical/ path in ptyshell #838

Closed CalmingBreeze closed 2 years ago

CalmingBreeze commented 3 years ago

Hi !

I'm working on a web interface wrapping saga but I have configuration problem. I'm running my app with django on apache. Without any conf, my home is /var/www/ , so /var/www/.radical/* raise permission error I tried to set RADICAL_BASE and it partially worked, as well as RADICAL_SAGA_LOG_TGT in /var/www/html/

But when I tried to initiate a shell connection I got an error about /var/www/.radical/ again. With further tracing I've ended finding this :

https://github.com/radical-cybertools/radical.saga/blob/ed6634ff8d7b31ff02b5f23d2aad1189d0f73360/src/radical/saga/utils/pty_shell.py#L228-L230

Is it possible to override this ? Maybe using conf file ?

Besides, is there a way to display all configs values ?

andre-merzky commented 3 years ago

Hi @CalmingBreeze,

Is it possible to override this ? Maybe using conf file ?

that's a great catch! We should make sure that RADICAL_BASE is also interpreted for that location, I'll take this as a bug report, and we should be able to fix this shortly.

Besides, is there a way to display all configs values ?

SAGA is indeed a bit under-documented in that respect, apologies for that. Let me try to collect what information we have, will ping back.

CalmingBreeze commented 3 years ago

Thank you for your quick response !

I think I've found another related issue : I managed to connect myself to my target server through ssh, but I can't get any job done due to missing .radical directory on target (used for state sync ?) :

1633092655.502 : radical.saga.pty     : 15596 : 140527094773504 : DEBUG    : Got initial shell prompt (6) (Last login: Fri Oct  1 14:50:11 2021 from xxxxxx.xxxxx.fr

With some debug enabled I got :

1633092657.252 : radical.saga.pty     : 15596 : 140527094773504 : DEBUG    : run_sync: find /var/www/.radical/saga/adaptors/sge_job/ -type f -mtime +30 -print -delete | wc -l
1633092657.252 : radical.saga.pty     : 15596 : 140527094773504 : DEBUG    : write: [   28] [   88] (find /var/www/.radical/saga/adaptors/sge_job/ -type f -mtime +30 -print -delete | wc -l\n)
1633092657.255 : radical.saga.pty     : 15596 : 140527094773504 : DEBUG    : read : [   28] [   80] (find: ‘/var/www/.radical/saga/adaptors/sge_job/’: No such file or directory\n0\n)
1633092657.255 : radical.saga.pty     : 15596 : 140527094773504 : DEBUG    : read : [   28] [   10] (PROMPT-0->)
1633092657.256 : radical.saga.cpi     : 15596 : 140527094773504 : INFO     : Purged find: ‘/var/www/.radical/saga/adaptors/sge_job/’: No such file or directory

Correct me if I'm wrong but wasn't saga supposed to try create .radical/ in my $HOME user on the target machine ? To be sure, I checked by connecting myself remotely with the same credentials, echo $HOME is correctly set as /auto/sshlogin

CalmingBreeze commented 3 years ago

Found the answer myself : In my case, there two json conf files hardcoded to $home :

https://github.com/radical-cybertools/radical.saga/blob/f453cccea4de594fd48e831a520a17dbf6083a25/src/radical/saga/configs/adaptors_sgejob.json#L13

https://github.com/radical-cybertools/radical.saga/blob/f453cccea4de594fd48e831a520a17dbf6083a25/src/radical/saga/configs/adaptors_shell_job.json#L18

A configurable variable like 'RADICAL_BASE_TGT' would be nice =)

CalmingBreeze commented 2 years ago

Hello Andre ! Any news about this env vars ? It would be nice to use the use the official package instead of my modified one !

andre-merzky commented 2 years ago

Oh boy, I did drop the ball on this one, didn't I? Dammit....

Please have a look at PR #841 - can you give that branch a try and let us know how it goes? Thanks!

CalmingBreeze commented 2 years ago

Hello, Andre, I'm currently working from home so I won't have access to the infrastructure for another week and can't test the PR.

May I ask if you have implemented a way to specify the distant sync directory ?

For example we will work to a distant infrastructure with a scratch partition, will I be able to specify RADICAL_BASE $home for my machine, and a sync directory outside of my own on the target (on the /scratch partition for example)