Interface to pbook broken in recent panda versions

[miholzbo]

Hi, I've noticed that in recent panda versions (current default is 1.5.62), the handy re-submission scripts such as panda-resub-taskid don't work anymore, I see messages like

Permission denied: taskID=34092415 is not owned by None
Permission denied: taskID=34086113 is not owned by None
Permission denied: taskID=34086060 is not owned by None

I have to go back to panda 1.5.46 to make these work as usual, then I see the following output

PBook user: Michael Holzbock
INFO : command=retry is registered. will be executed in a few minutes
INFO : command=retry is registered. will be executed in a few minutes
INFO : command=retry is registered. will be executed in a few minutes

It seems like the user name isn't retrieved correctly anymore in the more recent panda clients .. does anybody know a workaround for this?

[matthewfeickert]

Thanks for reporting this, @miholzbo. The panda-client team has broken the API before in patch releases (c.f. PR #58). We'll probably have to do diffs between panda-client v1.5.46 and newer releases to figure out what changed (e.g. https://github.com/PanDAWMS/panda-client/compare/1.5.46...1.5.47).

Can you verify that panda-client v1.5.46 works but panda-client v1.5.47 doesn't, or what is the minimum version of panda-client to still work?

I am traveling now so I might not be able to get to this soon, but perhaps @dguest can.

[miholzbo]

Thanks for the quick response! I've so far only tested the panda versions that I can setup via lsetup:

>> showVersions panda

panda versions;
 --> 1.5.40
 --> 1.5.41
 --> 1.5.42
 --> 1.5.44
 --> 1.5.46
 --> 1.5.50
 --> 1.5.60
 --> 1.5.61  previous
 --> 1.5.62  default   testing

I subsequently went back the versions above and 1.5.46 was the first verson which worked again.

[matthewfeickert]

Okay that already helps as even before testing locally we can isolate things to https://github.com/PanDAWMS/panda-client/compare/1.5.46...1.5.50

[matthewfeickert]

Okay that already helps as even before testing locally we can isolate things to https://github.com/PanDAWMS/panda-client/compare/1.5.46...1.5.50

[rbarrue]

Heads up: unfortunately, panda 1.5.46 is no longer available (even when doing setupATLAS -2) ...

[dguest]

I guess we should move to python 3 finally!

[rbarrue]

I guess we should move to python 3 finally!

Not sure this is the issue you wanted to comment on (maybe https://github.com/dguest/pandamonium/issues/23 ?) , not sure if the issues with the pbook API are related to the python version.

[dguest]

ok maybe not, I'd assumed maybe rucio had deprecated the ancient APIs with the python version but I probably confused myself.

[matthewfeickert]

Heads up: unfortunately, panda 1.5.46 is no longer available (even when doing setupATLAS -2) ...

What versions are available when doing so? Is it possible to just install a version of panda-client compatible with the lower bound

https://github.com/dguest/pandamonium/blob/8a019bb1182a0e24ee6ec98547aa7977650c2504/setup.cfg#L43-L44

(though you also need panda given that's what you mentioned is missing)?

[rbarrue]

Heads up: unfortunately, panda 1.5.46 is no longer available (even when doing setupATLAS -2) ...

What versions are available when doing so? Is it possible to just install a version of panda-client compatible with the lower bound

https://github.com/dguest/pandamonium/blob/8a019bb1182a0e24ee6ec98547aa7977650c2504/setup.cfg#L43-L44

(though you also need panda given that's what you mentioned is missing)?

lxplus950[~]$ showVersions panda

panda versions; --> 1.5.68 --> 1.5.70 --> 1.5.71 --> 1.5.73 previous --> 1.5.74 default testing Type lsetup "panda " to use panda

I now installed with pip, forcing panda-client version to 1.5.46 and it works as expected. Since I don't need `panda`, that would do enough for me. Think it's worth to force panda-client to this version ?

PS: when I have some time, I can go version by version to determine where they introduced the changes that are breaking this piece of code.

[dguest]

Thanks a lot, yeah probably worth excluding versions that don't work. Maybe @matthewfeickert has opinions here. But anyway yeah the best fix would be for someone to make it work with the most recent version, I bet it won't take that much time (but yeah I know none of us have much time).

[matthewfeickert]

I now installed with pip, forcing panda-client version to 1.5.46 and it works as expected. Since I don't need panda, that would do enough for me. Think it's worth to force panda-client to this version ?

@rbarrue can you either verify that v1.5.46 is the minimum required or give us a reproducible example for you that you're using so that we can verify it?

If you want an easy way to setup a virtual environment that works with CVMFS LCG views to test this (might not be necessary for you, so feel free to ignore), you can use cvmfs-venv.

[rbarrue]

I now installed with pip, forcing panda-client version to 1.5.46 and it works as expected. Since I don't need panda, that would do enough for me. Think it's worth to force panda-client to this version ?

@rbarrue can you either verify that v1.5.46 is the minimum required or give us a reproducible example for you that you're using so that we can verify it?

If you want an easy way to setup a virtual environment that works with CVMFS LCG views to test this (might not be necessary for you, so feel free to ignore), you can use cvmfs-venv.

I used as command panda-kill-taskid 39770381 (old task in done state). The last working version is 1.5.47, so something between 1.5.47 and 1.5.48 (https://github.com/PanDAWMS/panda-client/compare/1.5.47...1.5.48) is causing this.

1.5.48 introduces panda-client in batch mode.

[rbarrue]

Found the issue: while before the check for proxy and extraction of username was done when you created a PBookCore object (from the initializer), now they default the username to None and to get it we have to call the init() function.

Added a pbook.init() after the creation of the pbook objects in both panda-kill-taskid and panda-resub-taskid and it seems to work (with 1.5.98).

Will test on the latest version of panda-client and do the PR.

Edit: PR is below.