odeke-em
commented
8 years ago Hello there @ErraticEagle, thanks for the questions and welcome to drive!
So in regards to your questions, here are some thoughts:
1
Most importantly, how fast/agile is the community for bugs and updates? The biggest concern here is if Google does change something, which causes the client to break, in the future how long would this be down before there is a fix or a patch?
This project is run on volunteer time so the progress depends entirely on how much time folks can sacrifice. So, very unpredictable, depending on whoever has free time.
There are a certain class of issues that cause the client to break in ways out of our control: Currently there is still an issue that Google hasn't yet fixed and has been open for almost a year: https://github.com/odeke-em/drive/issues/480 left at the Google Drive team's discretion to solve. I personally don't work for Google nor do I know people who do work on Google Drive so fixing such is tricky.
In the past, whenever I noticed a random issue, I'd report it a few Google developer advocates or make mitigations for drive, but those issues would be patched up real quick by Google's infrastructure teams.
2
`At the moment we are currently using this client to push documents up to a drive and access/view the data there however we have sometimes noticed a duplication in names. Our current solution for this looks something like the following: cat lastRun/IRAK4_KS.sdf.gz \ | drive push -piped -force "$gDriveIRAK4/IRAK4 shared with WX/IRAK4_Selectivity.sdf.gz"
Is there a better way to push a file to drive and avoid name duplication?`
- What version of drive are y'all using?
- How many machines is this running on?
- How often does network outage happen at this place?
I'd recommend running one instance because with multiple instances, depending on if Google's backend responds properly, the client might potentially think a file doesn't exist and push it up -- however I think we've worked on making sure that this doesn't happen, but I'll double check.
There are also commands to deduplicate files. But I guess what is causing the piping and the --force is the lack of the ability to specify an alternate name for a file on the remote.
3
Can this client support scp commands if ran in a script? For example if we ran an instrument which followed a script, are we able to push the data to drive once completed using scp commands. If not, is this something that might be implemented in the future?
Someone in the past asked the same question here https://github.com/odeke-em/drive/issues/600. Unfortunately drive doesn't speak scp/ssh so you'll need to first copy those files to your local medium. This makes things awkward though copying files first from a server that you want to scp from, so even though it is a feature request, I agree that it would be a nice feature to have.
In the days before drive, I'd have to scp across multiple machines at school so I understand your plight.
However, I think if you have access to write and execute to an account on which you run scp, then you can definitely install drive on the machines that you wanna scp from. drive is a Go binary, Go binaries are static and moreover can be cross compiled -- see https://github.com/odeke-em/drive/blob/master/Makefile.
You'll need just credentials to a Google Drive account and the location of the binary -> chmod +x drive && ./drive
In fact that's the solution that I use across multiple machines if I want to push contents to Google Drive say on Lab machines after doing assignments.
The solution would be a composite command of different Unix utilities that can be run in a single ssh request, or after writing a bash script locally then uploading and running it (whichever works best for you). I'd use nohup to ensure that the ssh connection even if it times out, the payload would be deployed and working.
Here is a rough hack of something vague that I'd do (please check for errors before copying or running script, am not a lawyer):
$ scp -r drive_binary school_google_drive odeke@uofa:~/lab_workspace || exit -1
$ ssh odeke@uofa "chmod +x drive_binary && echo 'export PATH=\$HOME/lab_workspace:\$PATH' >> ~/.bashrc && cd \~/lab_workspace/school_google_drive && drive_binary push -m ~/workspace/Fall2015 school/labs" You just need the drive binary itself and the credentials which in the above script would be inside school_google_drive/.gd.
Synopsis
Basically most side projects keep evolving as the need for features springs up but also on the time available to hack on them. I don't know many people who are paid to work on their side projects so that factor alone means that people spend time making their living doing paid work and only tend to the side project when they are free. However, drive volunteers and users are very generous with their time :)
Spin off
Your questions got me think of two/three potential nice things we could add to drive:
- Specifying an alternate name for an uploaded file which would help solve the need
for
cat aname_on_disk | drive push --force --piped alternate_name_on_remote - Interoperability with other protocols, sources for files -- some sort of plugin system for scp, ftp.
- Improving the wiki and encouraging the community to share scripts or hacks and how they'd do a few things.
Thank you very much for the questions and please let me know what you think.
Hi @odeke-em just had some quick questions about this drive client.
1) Most importantly, how fast/agile is the community for bugs and updates? The biggest concern here is if Google does change something, which causes the client to break, in the future how long would this be down before there is a fix or a patch?
2) At the moment we are currently using this client to push documents up to a drive and access/view the data there however we have sometimes noticed a duplication in names. Our current solution for this looks something like the following:
Is there a better way to push a file to drive and avoid name duplication?
3) Can this client support scp commands if ran in a script? For example if we ran an instrument which followed a script, are we able to push the data to drive once completed using scp commands. If not, is this something that might be implemented in the future?
ie. copy a file from local directory to gdrive
copy set of files from local directory to gdrive
copy gdrive file to local folder or server
copy gdrive sheet file to local folder or server and convert to csv (tab would also be nice)
This one would just be a nice feature for us but not hugely needed.
Thanks,