search

x-atlas-consortia / clt

The X Atlas Consortia Command Line Transfer application
0 stars 0 forks source link

Update atlas-consortia-clt to enabled download locations outside of home directory #9

Open DerekFurstPitt opened 6 days ago

DerekFurstPitt commented 6 days ago

Globusconnectpersonal on windows/mac/linux has settings to give permissions for different directories but we still haven't seemed to find how to get gcp to let us set a location outside of the home directory.

There are certain use cases where users may wish to transfer their files outside of home.

bherr2 commented 6 days ago

Yeah, we had a related use case over a year ago with hubmap-clt. This is what we did:

Ahh, well, we may have figured it out though! I will go over gory details below and still verifying with Abhiroop if he can reproduce the fixes I made to make it work. But it worked for me when I tested it on Friday.

Essentially our problem was that we needed to download data with a headless server to a path not in the user's home directory (and no symbolic links don't work w/ globus).

For demonstration purposes, let's say the target directory is at /foo

Basically, we had to make to:

In hubmap-clt there is an assumption that files will be downloaded to the user's home directory (it adds ~/ to every path), so I removed that. (see https://github.com/hubmapconsortium/hubmap-clt/blob/main/hubmap_clt/__main__.py#L175-L182)

  • In the globus-connect-personal application, we had to add "/foo,0,1" to the ~/.globusonline/lta/config-paths file (see https://docs.globus.org/how-to/globus-connect-personal-linux/#config-paths)
  • Then when we run hubmap-clt transfer manifest.txt -d /foo it works. It'd be great to get this use case documented with the hubmap-clt and adding an option to let absolute paths through without appending ~/.

Thanks, Bruce

It would be great to have better support for this (or explicit directions in the documentation)

DerekFurstPitt commented 5 days ago

Thanks for that @bherr2

Yes, it seems its not as simple as adding new installation paths in the access tab. I would prefer if we could find a way to do it that didn't involve users needing to modify config files. In the mean time, our documentation as-is caused some confusion so its been amended until we have a less clunky solution implemented.