After upgrade from 3.1 to 3.2, Hangs on mysterious file to be changed

[colinkeenan]

/home/colin/Google Drive %drive push Resolving... M /� Modification count 1 Proceed with the changes? [Y/n]: ^C% /home/colin/Google Drive %

I don't know what the file is that supposedly needs to be modified, but it never completes, even hours later. If there are other new files or changes, there's a progress bar that completes, then it hangs. If there aren't any as in the example above, the progress bar never even starts.

[odeke-em]

Hello there @colinkeenan, Thank you for reporting this issue, welcome to drive and my apologies for the hang. So for starters would you mind giving me the result of 'go get github.com/odeke-em/drive/drive-gen && drive-gen && drive version' Also is it possible for you to do a 'drive stat' on that file?

[odeke-em]

Also I will be away from and a physical computer for about 7 hours till I get back home, but we can try debugging in the mean time

[colinkeenan]

result of go get github.com/odeke-em/drive/drive-gen && drive-gen && drive version is package github.com/odeke-em/drive/drive-gen: cannot download, $GOPATH not set. For more details see: go help gopath. I have installed drive on Arch Linux from https://aur.archlinux.org/packages/drive/

I cannot do anything with that file that causes the hang because it's not showing up when I do ls -a and the name of the file as reported by drive is showing up as a question mark inside a box. I am using Guake as the terminal and Ubuntu Mono as the font.

[odeke-em]

Please see https://github.com/odeke-em/drive#requirements for how to set your $GOPATH, don't forget to reload your shell.

To examine that file, as I had previously mentioned you'll need to stat like this 'drive stat ' When you get information about it e.g fileId, then try something like 'drive open --id '

[odeke-em]

Even before setting your GOPATH what does 'drive version' say?

[colinkeenan]

drive version: 0.3.2 Commit Hash: '848497f4ba68050820f32e1e54266e1eba2d5189' Go Version: go1.5.1 OS: linux/amd64 BuildTime: 2015-11-08 16:16:14.810927216 -0600 CST

[colinkeenan]

Now with $GOPATH defined, I get drive version: 0.3.2 Commit Hash: '848497f4ba68050820f32e1e54266e1eba2d5189' Go Version: go1.5.1 OS: linux/amd64 BuildTime: 2015-11-08 16:16:14.810927216 -0600 CST

which is the same result as drive version

[odeke-em]

Awesome thanks. So any results from 'drive stat', 'drive list'? What does the file listing look like when you examine it with the Google Drive Ui?

[colinkeenan]

/home/colin/Google Drive %drive stat
/
Filename                  My Drive                      
FileId                    0AFX0bdl44fGjUk9PVA           
Bytes                     0                             
Size                      0.00B                         
DirType                   folder                        
VersionNumber             310115                        
MimeType                  application/vnd.google-apps.folder
Etag                      "QEqb14agVr9EIVEHCQ-sEEtYDkU/MTQyNzA1MzgyNTg2Ng"
ModTime                   2015-03-22 19:50:26 +0000 UTC 
LastViewedByMe            0001-01-01 00:00:00 +0000 UTC 
Shared                    false                         
Owners                    Colin Keenan                  
LastModifyingUsername     Colin Keenan                  
OriginalFilename                                        
Starred                   false                         
Viewed                    false                         
Trashed                   false                         
ViewersCanDownload        false                         

*
Name: Colin Keenan <colinnkeenan@gmail.com>
Role                 owner                         
AccountType          user                          
*
 /home/colin/Google Drive %

[colinkeenan]

 /home/colin/Google Drive %drive list
/11-6-2015_Sold_2003-Pontiac-Montana.pdf 
/Finances 
/backup 
/store.json 
/Mom-Household 
/For_App_On_Oct5-2015.pdf 
/RefractiveErrors.abw 
/1990Topps&1000YardFootball.abw 
/1991FleerFootball.abw 
/1989ToppsFootball.abw 
/Sharp_CarouselII_Convection_Microwave_R-8260.pdf 
/agreement.pdf 
/caregiver 
/Pictures 
/used-stove.mp4 
/3835 
/usr_local_bin 
/IMG_1716.JPG 
/gnote_in_share_subfolder_of_.local 
/content 
/DVR Tracker 
/Resume-Job Application.docx 
/Resume-Job Application 
/Buzz-0001.pdf 
 /home/colin/Google Drive %

[colinkeenan]

I do not see the mysterious file using drive list. Does the results of drive stat mean the mysterious file is "My Drive"?

[odeke-em]

Ah I see, Thank you very much for the listing, as we can see the mysterious file is not present. So the next trial might seem far fetched but it might make sense if you are able to reproduce it https://github.com/odeke-em/drive/issues/406 contains a weird issue that has appeared on Linux only and so has this weird character. BTW is it a visible character or what? Because in your issue posting I can't make what the character is.

[colinkeenan]

It is a black diamond with a question mark inside. I think that just means the character is not in the font (Ubuntu Mono) that I'm using. If I try to copy and paste it, I get a simple question mark.

[odeke-em]

Aha I see, gotcha. I think this might almost be the same elusive bug in that linked issue. And this issue sticks for whatever push you make e.g in sub directories?

[colinkeenan]

I haven't tried a subdirectory, but if I specify the file to be pushed, it doesn't happen. I will check if it happens in a subdirectory...

It does not happen in subdirectories.

[colinkeenan]

Although the issue may be related to the linked issue, there's a difference because he knew what files had strange characters in them. For me, it almost seems as though it is just trying to update the date on the root directory itself or some file used by drive itself.

[colinkeenan]

By the way, I always see 3 of those black diamonds with question marks in the "Resolving..." animation that occurs on drive push. Two of those question mark diamonds alternate with a slash and a dash, and the 3rd diamond just stays in place after the flashing ones. What is that animation supposed to look like?

[odeke-em]

The spinner is supposed to be alternating '/', '-', '\', '|' rotating fast.

[colinkeenan]

It's the | that's not displaying in that spinner, but I don't know why it wouldn't work. That symbol is very important and used frequently in linux terminals without trouble.

[colinkeenan]

Actually, on closer examination, it's the - that's missing in the spinner. I think it's probably not a -, but a long dash instead.

[odeke-em]

Yes I agree, the pipe character and even any one of those characters are all quite useful. One other thing to try, could you run this command 'go get -u -v github.com/odeke-em/drive/drive-gen && drive-gen' then retry to see if the mysterious file returns on push , just to rule out if the AUR provided binary is trippy.

[colinkeenan]

AUR doesn't provide binaries. It provides a PKGBUILD that makepkg can use to download the source and build the package. Here is the PKGBUILD in the AUR for drive:

# Maintainer: John Jenkins <twodopeshaggy@gmail.com>

pkgname=drive
pkgver=0.3.2
pkgrel=1
pkgdesc="Pull or push Google Drive files"
arch=('x86_64' 'i686' 'arm' 'armv6h' 'armv7h')
url="http://github.com/odeke-em/drive"
license=('Apache')
depends=('hicolor-icon-theme' 'gtk-update-icon-cache')
makedepends=('go' 'git')
conflicts=('drive-git')
options=('!strip' '!emptydirs')
install=$pkgname.install
source=("https://github.com/odeke-em/$pkgname/archive/v$pkgver.tar.gz")
sha256sums=('3ea0fe4ea8e42f3ad5ba085cc827de6017095240958b4decc57433325b9de1fc')

prepare() {
 mkdir -p "$srcdir/go"
 export GOPATH="$srcdir/go"
 go get github.com/odeke-em/drive/cmd/drive
 go get github.com/odeke-em/drive/config
 go get github.com/odeke-em/command
 go get github.com/odeke-em/drive/src
 go get github.com/odeke-em/ripper/src
 go get github.com/odeke-em/xon/pkger/src
 export GOPATH="$srcdir/$pkgname-$pkgver"
 go get github.com/odeke-em/drive/drive-gen
 cd $srcdir/$pkgname-$pkgver/bin/
 ./drive-gen
 go get github.com/odeke-em/rsc/qr
 go get github.com/martini-contrib/binding
 cd $srcdir/$pkgname-$pkgver/drive-server
 go build -o drive-server
}

package() {
  cd $srcdir/$pkgname-$pkgver/bin/
  install -Dm755 "$pkgname" "$pkgdir/usr/bin/$pkgname"
  install -p -m755 $srcdir/$pkgname-$pkgver/drive-server/drive-server "$pkgdir/usr/bin"
  cd "$srcdir/$pkgname-$pkgver"
  mkdir -p $pkgdir/usr/share/licenses/$pkgname
  install -m 0644 LICENSE $pkgdir/usr/share/licenses/$pkgname/
  mkdir -p "$pkgdir/usr/share/icons/hicolor/128x128/mimetypes"
  cp $srcdir/$pkgname-$pkgver/icons/*.png $pkgdir/usr/share/icons/hicolor/128x128/mimetypes
  mkdir -p "$pkgdir/usr/share/icons/hicolor/scalable/mimetypes"
  cp $srcdir/$pkgname-$pkgver/icons/*.svg $pkgdir/usr/share/icons/hicolor/scalable/mimetypes
  rm -r  "$srcdir/go"
}
# vim:set ts=2 sw=2 et:

[odeke-em]

Ah I see, thanks for the clarification. For now I have no idea on what could be causing this as it runs alright on Mac OSX, Ubuntu machines that I have access to :(

[colinkeenan]

I have solved the diamond-question mark issue. I had to edit /etc/environment. Here is a cat of that file now that I've edited it:

 /home/colin/Google Drive %cat /etc/environment 
#
# This file is parsed by pam_env module
#
# Syntax: simple "KEY=VAL" pairs on separate lines
#
LANG=en_US.UTF-8

After doing that and rebooting, when I do drive push, I get the correct spinner, and I can see what the mysterious file is:

 /home/colin/Google Drive %drive push
Resolving...
M /
Modification count 1
Proceed with the changes? [Y/n]:n
 /home/colin/Google Drive %

It is as I suspected. drive is trying to update the modification date on the root of the Google Drive directory itself. I have no idea why simple characters like - and / aren't the same in ASCII and UTF-8.

[colinkeenan]

I think the issue may have something to do with the fact that ls -a which shows all files includes . which is the current directory. drive should be set up to not to update the file that represents the current directory.

[odeke-em]

That's a nice investigation, I kindly wanted to ask if you could make a wiki entry on your diagnosis, that's useful information that others could use. So does the push succeed?

[colinkeenan]

Actually, / was displayed before I changed /etc/environment, but now with the correct environment, the diamond-question mark shows as blank space. I wonder what it really is? Anyway, I think drive is trying to push a file that should not be tracked that represents the current root directory of "My Drive".

[colinkeenan]

I didn't try it. I assume it fails as before. It's just displaying the characters correctly. I will try again though.

[colinkeenan]

It hangs as before. I'm pretty sure that on linux, for some reason, it's trying to update the root directory file itself. There's no reason to do that. It's a bug of this version. Maybe previous versions had the same bug, but timed out after a minute, while this version doesn't time out.

[odeke-em]

Great hypothesis actually it is true, root shouldn't b mutable. If you do a drive push --verbose you should be able see the retry. So to quickly get you out of this hang, just do drive pull --depth 1 and this should set the ModTime of your local to the one upstream, and the issue shouldn't return.

[colinkeenan]

I have added a section to the wiki.

[odeke-em]

Awesome, thanks for the entry.

[colinkeenan]

drive pull --depth 1 Worked! Thanks. I will close this thread since the problem has been resolved. Maybe there is some way for drive to detect this issue and fix itself?

[odeke-em]

I mentioned something about the latest change to ensure modTimes match up even for directories https://github.com/odeke-em/drive/issues/471

[odeke-em]

The way for this to be resolved would be to add a check for a 403, File not mutable error in the retryableErrorCheck code and that should sort it out. Let me hop onto these library computers and make this patch. Thanks for the investigation @colinkeenan, I learnt something that is going to help resolve th previously linked issue as well as some locale education.

[colinkeenan]

Your welcome. I really appreciate this package and your quick followup on this bug.

[odeke-em]

Awesome! I've also created a PR https://github.com/odeke-em/drive/pull/473 to avoid the exponential backoff a retry for the change that's forbidden irrespective of retires.

[colinkeenan]

I have drive 3.4 installed and am still occasionally needing to drive pull --depth 1 in order to stop it from hanging on thinking it needs to sync the change of the root directory itself.

[odeke-em]

Please build from source as you might be using stale code like you had reported in https://github.com/odeke-em/drive/issues/547

[colinkeenan]

I had built from source, but turns out it was running an older version located earlier in the path. Will work this out myself. Thanks.