pipeline doesn't work: ERROR: too many url strings specified

gridcf / gct

Grid Community Toolkit

Apache License 2.0

46 stars 30 forks source link

pipeline doesn't work: ERROR: too many url strings specified #203

Closed zhibo-yan closed 1 year ago

zhibo-yan commented 1 year ago

I want to use the pipeline feature of gridftp. I run globus-gridftp-server as a daemon in two servers and use globus-url-copy and -pp or -pipeline to achieve third-party transfer at one of the servers. But errors occur: too many url strings specified. How can I use the pipeline feature? Thanks.

zhibo-yan commented 1 year ago

[root@bl197 ~]# globus-gridftp-server -version
globus_gridftp_server: 13.24 (1653033972-1)

[root@bl197 ~]# globus-url-copy -version
globus-url-copy: 10.12

fscheiner commented 1 year ago

@yanzhibo-nju: Please also provide the used commandline.

zhibo-yan commented 1 year ago

Server1: globus-gridftp-server -control-interface 192.168.110.1 -aa -p 60060 - anonymous-user 1040 -home-dir /home/data -auth-level 0 -allow-root

Server2: globus-gridftp-server -control-interface 192.168.110.2 -aa -p 60060 - anonymous-user 1040 -home-dir /home/data -auth-level 0 -allow-root

then still on server2: globus-url-copy -vb -pp 2 -r ftp://192.168.110.1:60060/home/dtn/200MB/ ftp://192.168.110.2:60060/home/dtn/200MB/,where 200MB is my file folder.

fscheiner commented 1 year ago

Thanks for the details.

Two points here:

-pp is a flag (see https://github.com/gridcf/gct/blob/aed61ccce4b376c5a0107d72fec777b1824e3461/gass/copy/source/globus_url_copy.c#L900) and does not have an argument, so please try without 2 after it first and if that works add -ppq 2 (which is undocumented, but actually seems to exist: https://github.com/gridcf/gct/blob/aed61ccce4b376c5a0107d72fec777b1824e3461/gass/copy/source/globus_url_copy.c#L912)
~I am unsure if this works with the standard FTP protocol at all, so if the above mentioned doesn't work, try with gsiftp:// instead~
=> pipelining works with FTP protocol as well, see https://github.com/gridcf/gct/issues/203#issuecomment-1321310987

Hope this helps, please let me know if it works or not.

zhibo-yan commented 1 year ago

Hi fscheiner, thank you very much for your reply. I have tried the two ways, -pp as a flag and -ppq with an argument, they both work! But I still use ftp:// rather than gsi:ftp//, I believe for gsi:ftp it will work as well.

Thanks for you help again!

fscheiner commented 1 year ago

@yanzhibo-nju Good to know and thanks for your report.