reanahub / reana-client

REANA command-line client
http://reana-client.readthedocs.io/
MIT License
10 stars 46 forks source link

cli: directories under `files` should be reported back to user, and vice versa #633

Closed audrium closed 1 year ago

audrium commented 2 years ago

Listing directories under files should be reported back to user, and vice versa.

(Alternatively, we can respect both, so if a user uses only files and puts directories there, we can "silently" accept them. This is something that could pave the way towards using only one directive in the future.)

BTW, regarding listing files and directories "unnecessarily", I have another observation. If a uses specifies the same file twice, it'll be uploaded twice, for example:

$ git diff
diff --git a/reana.yaml b/reana.yaml
index 4c69293..75d70e8 100644
--- a/reana.yaml
+++ b/reana.yaml
@@ -1,6 +1,11 @@
 version: 0.6.0
 inputs:
   files:
+    - code/gendata.C
+    - code/gendata.C
+    - code/gendata.C
+    - code/gendata.C
+    - code/gendata.C
     - code/gendata.C
     - code/fitdata.C
   parameters:

leads to:

$ reana-client run -w test
...
==> Uploading files...
==> Detected .gitignore file. Some files might get ignored.
==> SUCCESS: File /code/gendata.C was successfully uploaded.
==> SUCCESS: File /code/gendata.C was successfully uploaded.
==> SUCCESS: File /code/gendata.C was successfully uploaded.
==> SUCCESS: File /code/gendata.C was successfully uploaded.
==> SUCCESS: File /code/gendata.C was successfully uploaded.
==> SUCCESS: File /code/gendata.C was successfully uploaded.
==> SUCCESS: File /code/fitdata.C was successfully uploaded.
==> Starting workflow...

It would be nice to first make a list of all user-desired files and directories and walk through them, collect all the uploadable objects (avoiding symlinks), and then make a set out of them, so that we don't do any unnecessary upload the same object many times.

_Originally posted by @tiborsimko in https://github.com/reanahub/reana-client-go/pull/133#discussion_r982097814_