search

google / skicka

Command-line utility for working with Google Drive. Join the mailing list at https://groups.google.com/forum/#!forum/skicka-users.
Apache License 2.0
1.3k stars 118 forks source link

symbolic links are all ignored #55

Closed cjangrist closed 9 years ago

cjangrist commented 9 years ago

if a directory is composed of a lot of symbolic links skicka will fail to resolve them. It will ignore them all.

https://github.com/google/skicka/blob/31f6c0c2596011588d38c27a51e174fa3cbbb027/upload.go#L680-L682

ideally there should be a parameter:

-L Follow symbolic links.

As there is in the GNU program find.

screen shot 2015-03-02 at 5 37 40 pm

mmp commented 9 years ago

This was originally disallowed just to avoid having to write code to handle the case of detecting cycles in sym links, which are a bit of a bother to deal with. Would you be ok with a (user-configurable) fixed limit of how many sym links could be followed?

Related, it's probably worth supporting uploading/downloading sym links without following them. This could be done fairly easily by creating a zero-length file on Drive and adding a custom property to store the symlink target.

cjangrist commented 9 years ago

Fixed limit of depth of symbolic links that can be followed sounds like a great idea! A value of zero could simulate the current behavior, which some users might want.

As for supporting sym links without following (sentinel file of sorts?), perhaps this would be a good option to implement as a separate feature, maybe with a "maximum depth tree" parameter of "-1" or something like that. I don't think this should be the default behavior, however, as some users like myself use this program to move file between servers and this would break all the sym links (likely silently). Between defaulting to this option and no support for symlinks, I think I'd prefer the latter as long as it throws a warning (as it does now).

mmp commented 9 years ago

Ok, 'upload' now takes a -follow-symlinks option that takes a maximum depth. It seems to work well with tests here, but if you try it out, please double-check that its behavior matches your expectations!

cjangrist commented 9 years ago

It works marvellously! Just finished uploading 750GB of nested and complex symlink folders/data.

I'm testing the code to see how it handles loops/missing links/links with wrong permissions and such and will update here once I finish this.

mmp commented 9 years ago

Excellent! Happy to hear it's working as expected!

mmp commented 9 years ago

Upon further reflection, I don't think it's a good idea to have an option to maintain symbolic links as links (e.g. by encoding information in Google Drive file properties.) First, this could be an unhappy surprise for someone who thought they had a file uploaded but inadvertently only had a symlink that didn't point to actual data. Second, this would basically make the symlinks only useful to someone using skicka, which seems a generally undesirable direction.

So, closing this out as fixed, since the 'follow the links' code seems to be working well.