ascli shares - transfer-spec is not utilized

[thadu]

Hi! I am connecting to a Aspera Shares server and want to upload files. I can already upload files and folders, but using the --ts command doesn't result in the behaviour that I would expect. I used following command on Windows, escaping all " with \:

ascli shares files upload .\testFolder --to-folder="/destinationFolder" --ts=@json:'{\"overwrite\":\"never\",\"delete_source\":true}'

Content of local testFolder:

testFolder
- testfile.txt

Content of destinationFolder:

destinationFolder
- testFolder
-- testfile.txt 

Expected behaviour:

• "overwrite":"never" - As textfile.txt already exists in destinationFolder, it should not be replaced with the local file.
• "delete_source": true - after upload local testFolder should be deleted.

Observed behaviour:

• testfile.txt at destination is replaced by local testfile.txt
• local testFolder is not deleted

Are there any issues with how I wrote the command? Are there any limitations with Aspera Shares? Thanks in advance!

[laurent-martin]

Did you refer to the manual, see:

https://github.com/IBM/aspera-cli#shell-parsing-for-windows

and dev version of manual:

https://github.com/IBM/aspera-cli/tree/develop#shell-parsing-for-windows

explanations are there.

In short: command line parsing on Windows is ... not as standardized as on unix-like systems...

And ascli command line tokens is parsed by the ruby interpreter itself, which is similar to Linux, and you do not need to escape double quotes...

In long: mix of old stuff from DOS, plus Windows API based on full unparsed command line execution makes that...

Backslashes are not needed, should be removed.

Now, concerning the behaviour, ascli executes ascp underneath, you can observe that with option --log-level=debug.

Shares manages the access authorization, but then transfer is direct between ascp and the HSTS (transfer server) where server side ascp also runs.

So, for a manual of ts , you may refer to the manual of ascli, there is a table there, and also using ascli

ascli conf ascp spec

Then you can refer to the manual of ascp:

https://www.ibm.com/docs/en/ahts/4.4?topic=line-ascp-command-reference

[laurent-martin]

I did quick tests with Shares as you specified:

Right, source files are not deleted if they are not transfered because skipped (see ascp manual)

To delete source files even if only skipped, add: remove_skipped, so ts option would be:

{"overwrite":"never","delete_source":true,"remove_skipped":true}

Note: To get ascp logs add option : --transfer-info=@json:'{"ascp_args":["-DDL-"]}'

[thadu]

Thanks, I removed the backslash and the command still works. I still can not get the overwrite option to work.

Using --log-level=debug, I can see that the overwrite option is passed to ascp: Under the env_args: Following options can be found:

Still, the file is uploaded to the share server and not skipped. The same file is already at the source destination. ascp logs confirm this:

I also tested it with uploading only the single file to the target destination. The file at destination will always be overwritten. As the file is uploaded, "remove_skipped" should not be needed and "delete_source" should delete the file at source, which it also does not.

[laurent-martin]

Note that the question is not related to ascli it self, but to the behaviour of ascp which is part of the IBM Aspera SDK.

[thadu]

Hi, is this a problem on my side, or is this an issue with ascp? Is there any information that I can provide to help to find a solution? Thanks!

[laurent-martin]

Generally speaking, ascli is a front-end for ascp: For transfers , ascli interracts with the web ap (Shares, Faspex, etc..) to finally generate a transfer spec, with transfer authorization (token), and then it spawns ascp by translating this transfer spec into command line arguments to ascp. The way ascp will manage restarts, deletions, etc.. is not managed by ascli, only ascp following the arguments provided. And this project is not about ascp, it's ascli. Maybe write a specification of what you want to achieve, with some clear examples.

did you try this : ?

{"overwrite":"never","delete_source":true,"remove_skipped":true}

[laurent-martin]

This tiocket will be closed, if there is no news

[thadu]

Hi, yes, this is my Windows cmd command:

ascli shares files upload --ts=@json:'{"overwrite":"never","delete_source":true,"remove_skipped":true}' .\testFolder --to-folder="/destinationFolder" --log-level=debug

Files from testFolder always overwrite files in destinationFolder when uploaded. See explanation in the original comment.