bash on windows has error

[freshcn]

I use glide in bash on windows. when i exec glide install.i got the follow error message

[INFO]  --> Exporting golang.org/x/net
[INFO]  --> Exporting golang.org/x/crypto/acme/autocert
[INFO]  --> Exporting golang.org/x/crypto
[INFO]  Replacing existing vendor dependencies
[ERROR] remove /tmp/glide-vendor220918663/vendor/github.com/google/go-github/github: directory not empty
[ERROR] Unable to export dependencies to vendor directory: remove /mnt/c/Users/Administrator/go/src/github.com/freshcn/go-id-builder/vendor/github.com/google/go-github/github: directory not empty

there has the same error on v0.12.3 and 0.13.0-dev

[bacongobbler]

do you have an example glide.yaml and glide.lock that can reproduce this failure?

[freshcn]

this is my glide.yaml and glide.lock. they can work on ubuntu and mac.

glide.yaml

package: git.coding.net/dolist/api.git
import:
- package: github.com/freshcn/async
- package: github.com/freshcn/log
- package: github.com/freshcn/tools
  subpackages:
  - uuid
  - vaildate
- package: github.com/go-sql-driver/mysql
  version: ^1.2.0
- package: github.com/go-xorm/core
  version: ^0.5.6
- package: github.com/go-xorm/xorm
  version: ^0.5.6
- package: github.com/namsral/flag
- package: gopkg.in/ini.v1
  version: ^1.21.1
- package: gopkg.in/iris-contrib/middleware.v5
  version: ^5.0.0
  subpackages:
  - logger
- package: gopkg.in/kataras/iris.v5
  version: ^5.0.0
- package: gopkg.in/redis.v5
  version: ^5.0.1

glide.lock

hash: f3c7280548e096dae033969bab54d1ebd8925c7bb7bd1758ba5b223b78b5f9f0
updated: 2017-02-18T02:01:18.477596607+08:00
imports:
- name: github.com/fatih/structs
  version: dc3312cb1a4513a366c4c9e622ad55c32df12ed3
- name: github.com/freshcn/async
  version: c3953276c8fc222c13d454dee39e2eed6ba3f5ed
- name: github.com/freshcn/log
  version: 98911aaee264b8c69ffa041c63a71574d951c8bc
- name: github.com/freshcn/tools
  version: cdcebbb7db5ed8db845c644c4e9197e11778a0d5
  subpackages:
  - uuid
  - vaildate
- name: github.com/geekypanda/httpcache
  version: 76ba6c68462ae362cda7564c44492b95322b363a
  subpackages:
  - internal
  - internal/fhttp
  - internal/fhttp/rule
  - internal/nethttp
  - internal/nethttp/rule
  - internal/server
- name: github.com/go-sql-driver/mysql
  version: a0583e0143b1624142adab07e0e97fe106d99561
- name: github.com/go-xorm/builder
  version: 867edcc549127f15667ece8876c3a60093c9634b
- name: github.com/go-xorm/core
  version: 5bf745d7d163f4380e6c2bba8c4afa60534dd087
- name: github.com/go-xorm/xorm
  version: d75356fc733fce7683c6d961fc2aeb16ededc8ef
- name: github.com/google/go-github
  version: f7fcf6f52ff94adf1cc0ded41e7768d2ad729972
  subpackages:
  - github
- name: github.com/google/go-querystring
  version: 9235644dd9e52eeae6fa48efd539fdc351a0af53
  subpackages:
  - query
- name: github.com/gorilla/websocket
  version: 2aff870ef83507abb46472eae82449f7a4c464c5
- name: github.com/hashicorp/go-version
  version: e96d3840402619007766590ecea8dd7af1292276
- name: github.com/imdario/mergo
  version: 50d4dbd4eb0e84778abe37cefef140271d96fade
- name: github.com/iris-contrib/formBinder
  version: 81b6a071e35797b83562caf4b3cad24dc01912dc
- name: github.com/iris-contrib/lego
  version: 095d7f6459c501cb15319aa2754afa221b81a3ec
  subpackages:
  - acme
- name: github.com/iris-contrib/letsencrypt
  version: 1a3e5c619a13b307df3b1b4da7cb7e57d2e156dd
- name: github.com/kataras/go-errors
  version: 0f977b82cc78d5d31bb75fb6f903ad9e852c8bbd
- name: github.com/kataras/go-fs
  version: c029e113d9faaf558b730f06041c8bf9545a3502
- name: github.com/kataras/go-options
  version: 23b556c1b935c594ec6d71ff81ead4dbeec3aa8d
- name: github.com/kataras/go-serializer
  version: 0bd874a15c70db74ef2e668e5eeda27041f03b81
  subpackages:
  - data
  - json
  - jsonp
  - markdown
  - text
  - xml
- name: github.com/kataras/go-sessions
  version: 697680208eb3db6a76240f2556eaa7d4909f2556
- name: github.com/kataras/go-template
  version: 457f21178102f4688603eccbb4f2e8d5ae1023bf
  subpackages:
  - html
- name: github.com/klauspost/compress
  version: 8df558b6cb6f9b445f9586446cfe7223e7d8bd6b
  subpackages:
  - flate
  - gzip
  - zlib
- name: github.com/klauspost/cpuid
  version: 09cded8978dc9e80714c4d85b0322337b0a1e5e0
- name: github.com/klauspost/crc32
  version: cb6bfca970f6908083f26f39a79009d608efd5cd
- name: github.com/microcosm-cc/bluemonday
  version: f77f16ffc87a6a58814e64ae72d55f9c41374e6d
- name: github.com/miekg/dns
  version: 58f52c57ce9df13460ac68200cef30a008b9c468
- name: github.com/namsral/flag
  version: 91bb473a91a4a5e1ea554fb8ddce84f832be9aca
- name: github.com/russross/blackfriday
  version: 5f33e7b7878355cd2b7e6b8eefc48a5472c69f70
- name: github.com/satori/go.uuid
  version: b061729afc07e77a8aa4fad0a2fd840958f1942a
- name: github.com/shurcooL/sanitized_anchor_name
  version: 1dba4b3954bc059efc3991ec364f9f9a35f597d2
- name: github.com/square/go-jose
  version: 63d7cfa0284d0bc9bf41d58f802037559c45ce8f
  subpackages:
  - cipher
  - json
- name: github.com/valyala/bytebufferpool
  version: e746df99fe4a3986f4d4f79e13c1e0117ce9c2f7
- name: github.com/valyala/fasthttp
  version: 4ee6517626b0549c4a4c85ad7b91acdec1d2af24
  subpackages:
  - fasthttpadaptor
  - fasthttputil
- name: golang.org/x/crypto
  version: bed12803fa9663d7aa2c2346b0c634ad2dcd43b7
  subpackages:
  - acme/autocert
  - ocsp
- name: golang.org/x/crypto/acme/autocert
  version: bed12803fa9663d7aa2c2346b0c634ad2dcd43b7
- name: golang.org/x/net
  version: 41bba8d80bbfab43231ffdf4c210037baae5f6a3
  subpackages:
  - context
  - html
  - html/atom
  - publicsuffix
- name: golang.org/x/time
  version: f51c12702a4d776e4c1fa9b0fabab841babae631
  subpackages:
  - rate
- name: gopkg.in/ini.v1
  version: e3c2d47c61e5333f9aa2974695dd94396eb69c75
- name: gopkg.in/iris-contrib/middleware.v5
  version: eff973db50aaea60190e75f8f5dc343f7de34b38
  subpackages:
  - logger
- name: gopkg.in/iris-contrib/websocket.v5
  version: bc14117ad25db88dd667f03170dc6db1e5c1f78f
- name: gopkg.in/kataras/go-errors.v0
  version: d2aaffdebcb40c69c4cb6a95e436430cde192435
- name: gopkg.in/kataras/go-fs.v0
  version: ecf3587588f7218a21a991a1c0a9f4bdebd20cc1
- name: gopkg.in/kataras/go-websocket.v0
  version: cb1681badd5582ccaebaf02f452e2d5a98f1a440
- name: gopkg.in/kataras/iris.v5
  version: 547c5360493789dc4f0e65894f14af086c62d03c
  subpackages:
  - utils
- name: gopkg.in/redis.v5
  version: 2fe9c5cc0a8da9faeba3a45511a4ece8f7e37dc2
  subpackages:
  - internal
  - internal/consistenthash
  - internal/hashtag
  - internal/pool
  - internal/proto
testImports: []

[mattfarina]

Is this bash installed with git? What windows version and bash version?

[joesteele]

I believe this is related to #654.

Using the example docker glide.yaml provided in the comments, I'm able to reproduce this on my Windows machine (on the recent Creator's Update) using the WSL (bash v4.3.11(1)-release).

Using this same glide.yaml on my mac laptop works fine with the same glide and go versions (glide v0.12.3 and go v1.8).

package: .
import:
- package: github.com/docker/docker
  version: master

[eliothedeman]

I see the same thing running WSL on kernal 4.4.0-43-Microsoft on most recent update of windows 10 glide 0.12.3 and go 1.8.1

[djensen47]

Is this bash installed with git? What windows version and bash version?

No, this is on "Bash on Ubuntu for Windows" also known as Windows Subsystem for Linux (WSL).

It looks like some folks are seeing this issue on Vagrant as well.

[kumarharsh]

I get this same error on powershell too. It's not a WSL problem, but rather something to do with windows filesystem locking some file while glide is doing it's thing. And it's always random repositories, it's not just due to any one dependency. So there's not real way to stop this from happening apart from this:

rm -r ./vendor; glide install

which sucks!

[thomastaylor312]

I was able to duplicate this problem as well when doing a glide install in the Helm repo using WSL. Mine seems to hang on the same repo every time /tmp/glide-vendor123664865/vendor/golang.org/x/sys/unix

[vidaoaaron]

Also seeing this consistently on the same repo

$ glide -v
glide version 0.13.0-dev

[ERROR] remove /tmp/glide-vendor361549384/vendor/google.golang.org/api: directory not empty                                          
[ERROR] Unable to export dependencies to vendor directory: remove /root/gopath/src/github.com/xx/yy/vendor/google.golang.org/api: directory not empty

[kumarharsh]

Replying to https://github.com/Masterminds/glide/issues/778#issuecomment-287961222, @mattfarina this happens even with the latest Git for Windows version (2.13.0 at the time of writing this), and with powershell or cmd or git bash (mingw64 bash). Glide version is the latest (but it's still lacking those windows fixes merged into master) (0.12.1 at the time of writing).

To easily reproduce the issue, running a simple glide update after glide install on this glide.yaml should throw the error on Windows.

[djensen47]

I'm using glide version v0.12.1-80-g4b407b3 from master and I'm still seeing these issues.

[BhallalDev]

I'm also using glide version v0.12.3 from master and facing this issue consistently

[djensen47]

Are there any workarounds?

I noticed this only happens for me if a dependency that I'm importing has files in it's /vendor dir.

[djensen47]

Also, I know that other folks said they couldn't get this working on Windows with Git Bash. After clearing my glide cache, it worked for me on Git Bash!

Should we file something against microsoft/BashOnWindows?

I ask because in the early days of WSL (BashOnWindows) there was an issue in nodejs where you could not watch files and it turned out to be a WSL issue IIRC related to file descriptors.

[kumarharsh]

Should we file something against microsoft/BashOnWindows?

@djensen47 - not sure about that. I get same errors using powershell or cmd or git's mingw bash. But I also don't know if WSL would affect normal powershell either, so ¯_(ツ)_/¯

[djensen47]

@kumarharsh Did you try clearing your glide cache? I was having issues with Git (mingw) Bash at first but clearing the cache fixed it.

[kumarharsh]

@djensen47 I tried clearing cache and running glide update again. Still the same error.

[ERROR] Unable to export dependencies to vendor directory: remove C:\Workspace...\vendor\github.com\chzyer\readline\example: The directory is not empty.

[mattfarina]

I've been able to reproduce on projects with a large number of dependencies. Projects I've tried with a smaller set of dependencies have not exhibited the behavior. Is this how it's worked for others?

[mattfarina]

The issue appears to be that os.RemoveAll is unable to remove some files. This had previously been an issue in Go and some other projects detect Windows to handle this case separately. The issue is closed.

Unless someone beats me to it, I'll be:

• Filing an issue against Go about this
• Implementing a work around

[kumarharsh]

@mattfarina Yes, it happens consistently with large number of dependencies.

[thomastaylor312]

@mattfarina That seems to be the case for me. Thanks for looking into it!

[mattfarina]

Quick update... I think I have a fix but testing it has been a little hard. On Windows (cmd.exe, etc) I'm having trouble reproducing the original issue to prove it's fixed with the change. It's intermittent for me. Under Bash... the linux subsystem setup to work with the filesystem is really slow which has made testing on projects with lots of files (where this shows up) and proving the fix worked to be slow.

It's just taking a little longer than I expected.

[kumarharsh]

@mattfarina - I have a system where I always get this error. Maybe if you can share the code/glide binary, I can try it out and report if all goes well

[mattfarina]

When I return home, in a little bit, I'll post the code for review. Thanks.

[thomastaylor312]

I can test as well

[djensen47]

@mattfarina I can also test. If you push your updates to a branch, I'll compile and test. Thanks!

[mattfarina]

Sorry for the delay. Took a little longer to get to the bottom of this than I expected.

First, the Go issue is at https://github.com/golang/go/issues/20841 along with details to reproduce it outside of Glide.

Second, there is a pull request that needs testing at https://github.com/Masterminds/glide/pull/857.

Feedback is requested. Is there a better way to handle this?

[marct83]

I can test this I'm just not sure how install your fix on my system. I'm running windows linux subsystem.

[djensen47]

$ git clone git@github.com:Masterminds/glide.git $GOPATH/src/github.com/Masterminds/glide
$ cd $GOPATH/src/github.com/Masterminds/glide
$ git checkout -b windows-many-files origin/fix/windows-many-files
$ make build
$ mv glide $GOPATH/bin

Go to your project and use the new glide but be sure that you're using the new one...

$ which glide

Next, I cleared everything away.

$ rm -rf vendor
$ glide cc

@mattfarina My first attempt at glide cc failed with this error:

[ERROR] Unable to clear the cache: remove .glide/cache/src/https-github.com-fatih-color/vendor/golang.org/x/sys/unix: directory not empty

I tried again and it worked the second time.

The moment of truth ...

$ glide update

🥁 (to be continued, that's supposed to be a drumroll while I'm waiting for the command to finish)

[djensen47]

SUCCESS! It worked! 🕺 🎉 🎆

@mattfarina Thank you! I feel like I should buy you a beer, or coffee, or artisan water, or whatever is your beverage of choice.

[marct83]

@djensen47

I'm running into this issue when I try the clone:

Cloning into '/root/Code/go/github.com/Masterminds/glide'... Permission denied (publickey). fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

[djensen47]

Oh, clone using the https address instead.

git clone https://github.com/Masterminds/glide.git

On Sat, Jul 1, 2017, 10:49 PM marct83 notifications@github.com wrote:

@djensen47 https://github.com/djensen47

I'm running into this issue when I try the clone:

Cloning into '/root/Code/go/github.com/Masterminds/glide'... Permission denied (publickey). fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Masterminds/glide/issues/778#issuecomment-312472568, or mute the thread https://github.com/notifications/unsubscribe-auth/AAgN-f2ehDfk6Zc9MWv0j2meCUip06EIks5sJy9ngaJpZM4MXDzl .

[marct83]

Closer...

make build go build -o glide -ldflags "-X main.version=v0.12.1-81-ga7ffb17" glide.go glide.go:24:2: cannot find package "github.com/Masterminds/glide/action" in any of: /usr/lib/go-1.6/src/github.com/Masterminds/glide/action (from $GOROOT) /root/marcb/Code/go/src/github.com/Masterminds/glide/action (from $GOPATH) glide.go:25:2: cannot find package "github.com/Masterminds/glide/cache" in any of: /usr/lib/go-1.6/src/github.com/Masterminds/glide/cache (from $GOROOT) /root/marcb/Code/go/src/github.com/Masterminds/glide/cache (from $GOPATH) glide.go:26:2: cannot find package "github.com/Masterminds/glide/msg" in any of: /usr/lib/go-1.6/src/github.com/Masterminds/glide/msg (from $GOROOT) /root/marcb/Code/go/src/github.com/Masterminds/glide/msg (from $GOPATH) glide.go:27:2: cannot find package "github.com/Masterminds/glide/path" in any of: /usr/lib/go-1.6/src/github.com/Masterminds/glide/path (from $GOROOT) /root/marcb/Code/go/src/github.com/Masterminds/glide/path (from $GOPATH) glide.go:28:2: cannot find package "github.com/Masterminds/glide/repo" in any of: /usr/lib/go-1.6/src/github.com/Masterminds/glide/repo (from $GOROOT) /root/marcb/Code/go/src/github.com/Masterminds/glide/repo (from $GOPATH) glide.go:29:2: cannot find package "github.com/Masterminds/glide/util" in any of: /usr/lib/go-1.6/src/github.com/Masterminds/glide/util (from $GOROOT) /root/marcb/Code/go/src/github.com/Masterminds/glide/util (from $GOPATH) glide.go:31:2: cannot find package "github.com/codegangsta/cli" in any of: /usr/lib/go-1.6/src/github.com/codegangsta/cli (from $GOROOT) /root/marcb/Code/go/src/github.com/codegangsta/cli (from $GOPATH) Makefile:8: recipe for target 'build' failed make: *** [build] Error 1

[djensen47]

My instructions had a typo. I was missing /src after $GOPATH. I updated the instructions and they should work now. You'll have to start from the top though.

[marct83]

Ok so I was able to get it built. When I do a which glide I'm getting /usr/bin/glide. How do I change it to use the new glide? Thanks for the help.

[marct83]

FYI I got this to work. Tested working.

[kumarharsh]

It works for me too on Windows proper (on powershell)!

[thomastaylor312]

Thanks for patching this @mattfarina!