cli: regex/glob or similar for `reana-client ls`

[lukasheinrich]

for large workspaces it would be nice to either be able to glob / regex filter the files

reana-client ls 'mydir'
reana-client ls 'mydir/mysubdir'
reana-client ls 'mydir/mysubdir/*.root'

[tiborsimko]

For reana-client upload and reana-client download the globbing is already plugged in. It would be indeed good to do it for reana-client ls as well. Until then, one can do extra grep:

$ reana-client ls -w bsm-search.serial | grep 'merged.root'
merge/merged.root                                                           14268     2019-01-14T15:47:44
all_bkg_mc/merge/merged.root                                                12364     2019-01-14T15:47:35
data/mergeall/merged.root                                                   5114      2019-01-14T15:46:54
data/select_signal_merge_0/merged.root                                      51625     2019-01-14T15:45:04
data/select_control_merge_0/merged.root                                     153387    2019-01-14T15:45:11
data/merge_0/merged.root                                                    413053    2019-01-14T15:39:35
signal/hist_merge/merged.root                                               4450      2019-01-14T15:46:15
signal/select_merge_0/merged.root                                           297326    2019-01-14T15:43:04
signal/merge_0/merged.root                                                  297768    2019-01-14T15:39:31
all_bkg_mc/run_mc_1/mergeallvars/merged.root                                8034      2019-01-14T15:47:16
all_bkg_mc/run_mc_1/mergeshapes/merged.root                                 5551      2019-01-14T15:46:59
all_bkg_mc/run_mc_1/mergeweights/merged.root                                6355      2019-01-14T15:46:56
all_bkg_mc/run_mc_1/select_signal_merge_0/merged.root                       616540    2019-01-14T15:45:16
all_bkg_mc/run_mc_1/merge_1/merged.root                                     305601    2019-01-14T15:39:39
all_bkg_mc/run_mc_1/merge_0/merged.root                                     305545    2019-01-14T15:39:42
all_bkg_mc/run_mc_0/mergeallvars/merged.root                                8270      2019-01-14T15:47:18
all_bkg_mc/run_mc_0/mergeshapes/merged.root                                 5643      2019-01-14T15:47:00
all_bkg_mc/run_mc_0/mergeweights/merged.root                                6499      2019-01-14T15:46:52
all_bkg_mc/run_mc_0/select_signal_merge_0/merged.root                       548314    2019-01-14T15:45:16
all_bkg_mc/run_mc_0/merge_1/merged.root                                     298575    2019-01-14T15:39:33
all_bkg_mc/run_mc_0/merge_0/merged.root                                     298632    2019-01-14T15:39:38
all_bkg_mc/run_mc_1/select_signal_shapevars_1/merge_0/merged.root           599359    2019-01-14T15:45:31
all_bkg_mc/run_mc_1/select_signal_shapevars_0/merge_0/merged.root           611915    2019-01-14T15:45:16
all_bkg_mc/run_mc_0/select_signal_shapevars_1/merge_0/merged.root           491593    2019-01-14T15:45:33
all_bkg_mc/run_mc_0/select_signal_shapevars_0/merge_0/merged.root           565841    2019-01-14T15:45:27

[tiborsimko]

Two notes:

(1) The list command is now called ls, I have amend the original description to make it clearer.

(2) We do offer some globbing for the rm command, see:

$ reana-client ls -w hello
NAME                    SIZE   LAST-MODIFIED      
code/helloworld.py      3253   2021-03-22T14:56:31
data/names.txt          20     2021-03-22T14:56:32
results/greetings.txt   34     2021-03-22T14:56:46

$ reana-client rm -w hello 'data/*'    
File data/names.txt was successfully deleted.
20 bytes freed up.

$ reana-client ls -w hello
NAME                    SIZE   LAST-MODIFIED      
code/helloworld.py      3253   2021-03-22T14:56:31
results/greetings.txt   34     2021-03-22T14:56:46

It would be good to replicate the same functionality to ls command.... and see also download (#402) and all the other file manipulating commands where such an optional argument would be useful.

[mvidalgarcia]

(2) We do offer some globbing for the rm command, see:

The current rm globbing is relying on the files you have in your pwd, which IMO is wrong. To reproduce:

$ rm data/names.txt
$ reana-client rm -w hello data/*                           
zsh: no matches found: data/*

How should the reana-client ls globbing behave?

[tiborsimko]

Oh, I did not realise rm does local and not remote globbing?! It should definitely happen on the server-side, for all commands, rm and ls etc. When workflow produces N files, they will exist only remotely, not locally, for all these operations (except upload obviously).

[mvidalgarcia]

Err, this only happens if there are no quotes. Wildcard logic to remove files here.

[tiborsimko]

All good then :wink: Without single quotes, or escaping backslash, the local shell expansion takes over. So single quotes are necessary, as in my example above.

[mvidalgarcia]

Closed by linked PRs.