dep ensure fails with grouped write of manifest, lock and vendor: cannot delete

hectorj2f commented 6 years ago

What version of `dep` are you using (`dep version`)?

v0.3.2

What `dep` command did you run?

dep ensure -v

I always get this error one out of 2 runs.

(20/22) Wrote github.com/pelletier/go-toml@v1.0.1
(21/22) Wrote gopkg.in/mattn/go-isatty.v0@v0.0.3
(22/22) Wrote gopkg.in/cheggaaa/pb.v2@v2.0.5
grouped write of manifest, lock and vendor: cannot delete /pkg/src/github.com/project/kubernetes/cli/vendor: remove /pkg/src/github.com/project/kubernetes/cli/vendor/golang.org/x/sys/unix: directory not empty

sdboyer commented 6 years ago

hi! any chance you're running on WSL? #1327

hectorj2f commented 6 years ago

Nope, I am running on Ubuntu

sdboyer commented 6 years ago

hmm, that's a first.

can you provide more detailed reproduction steps?

hectorj2f commented 6 years ago

I have a clean project without any vendor directory, so I run dep ensure. Everything goes smoothly. Then I run it again and I get the mentioned error and I can see a new directory vendor.orig. If I run it a third time, everything works as when I ran it at the beginning.

sdboyer commented 6 years ago

"i have a clean project" is not a step i can reproduce. i need to know what imports exist, what's in the Gopkg.toml, etc. reporting the output of dep hash-inputs would probably be sufficient, though if you're working on a public project you could point me to, that would be preferable.

hectorj2f commented 6 years ago

"i have a clean project" is not a step i can reproduce

Unfortunately I cannot point you to the our private project.

Output of Gopkg.toml

[[constraint]]
  name = "github.com/mesosphere/dcos-commons"
  version = "0.30.3"

[[constraint]]
  branch = "v3-unstable"
  name = "gopkg.in/alecthomas/kingpin.v3-unstable"

[[constraint]]
  name = "github.com/yudai/gojsondiff"
  version = "v1.0.0"

[[constraint]]
  name = "gopkg.in/cheggaaa/pb.v1"
  version = "v1.0.18"

Output of Gopkg.lock


[[projects]]
  branch = "master"
  name = "github.com/alecthomas/units"
  packages = ["."]
  revision = "2efee857e7cfd4f3d0138cc3cbb1b4966962b93a"

[[projects]]
  name = "github.com/davecgh/go-spew"
  packages = ["spew"]
  revision = "346938d642f2ec3594ed81d874461961cd0faa76"
  version = "v1.1.0"

[[projects]]
  name = "github.com/mattn/go-runewidth"
  packages = ["."]
  revision = "9e777a8366cce605130a531d2cd6363d07ad7317"
  version = "v0.0.2"

[[projects]]
  name = "github.com/mesosphere/dcos-commons"
  packages = ["cli","cli/client","cli/commands","cli/config"]
  revision = "7ae5d99f5e48817e1bdd9a68c3c6179e377ad74a"
  version = "0.30.3"

[[projects]]
  name = "github.com/nicksnyder/go-i18n"
  packages = ["i18n","i18n/bundle","i18n/language","i18n/translation"]
  revision = "0dc1626d56435e9d605a29875701721c54bc9bbd"
  version = "v1.10.0"

[[projects]]
  name = "github.com/pelletier/go-toml"
  packages = ["."]
  revision = "16398bac157da96aa88f98a2df640c7f32af1da2"
  version = "v1.0.1"

[[projects]]
  name = "github.com/pmezard/go-difflib"
  packages = ["difflib"]
  revision = "792786c7400a136282c1664665ae0a8db921c6c2"
  version = "v1.0.0"

[[projects]]
  branch = "master"
  name = "github.com/sergi/go-diff"
  packages = ["diffmatchpatch"]
  revision = "1744e2970ca51c86172c8190fadad617561ed6e7"

[[projects]]
  name = "github.com/stretchr/testify"
  packages = ["assert","require","suite"]
  revision = "69483b4bd14f5845b5a1e55bca19e954e827f1d0"
  version = "v1.1.4"

[[projects]]
  name = "github.com/yudai/gojsondiff"
  packages = ["."]
  revision = "7b1b7adf999dab73a6eb02669c3d82dbb27a3dd6"
  version = "1.0.0"

[[projects]]
  branch = "master"
  name = "github.com/yudai/golcs"
  packages = ["."]
  revision = "ecda9a501e8220fae3b4b600c3db4b0ba22cfc68"

[[projects]]
  branch = "v3-unstable"
  name = "gopkg.in/alecthomas/kingpin.v3-unstable"
  packages = ["."]
  revision = "c9d19ded11d4843b42b60700409da7259460cb65"

[[projects]]
  name = "gopkg.in/cheggaaa/pb.v1"
  packages = ["."]
  revision = "657164d0228d6bebe316fdf725c69f131a50fb10"
  version = "v1.0.18"

[[projects]]
  branch = "v2"
  name = "gopkg.in/yaml.v2"
  packages = ["."]
  revision = "287cf08546ab5e7e37d55a84f7ed3fd1db036de5"

[solve-meta]
  analyzer-name = "dep"
  analyzer-version = 1
  inputs-digest = "0273ecdf202ad698ab023fb64f91dd9e472ed2bbd9e61fafdc17cea113b3d869"
  solver-name = "gps-cdcl"
  solver-version = 1

Output of dep hash-inputs:

dep  hash-inputs
-CONSTRAINTS-
github.com/mesosphere/dcos-commons
svc-^0.30.3
github.com/yudai/gojsondiff
svc-^1.0.0
gopkg.in/alecthomas/kingpin.v3-unstable
b-v3-unstable
gopkg.in/cheggaaa/pb.v1
svc-^1.0.18
-IMPORTS/REQS-
github.com/mesosphere/dcos-commons/cli
github.com/mesosphere/dcos-commons/cli/client
github.com/mesosphere/dcos-commons/cli/commands
github.com/mesosphere/dcos-commons/cli/config
github.com/stretchr/testify/assert
github.com/stretchr/testify/suite
github.com/yudai/gojsondiff
gopkg.in/alecthomas/kingpin.v3-unstable
gopkg.in/cheggaaa/pb.v1
-IGNORES-
-OVERRIDES-
-ANALYZER-
dep
1

hectorj2f commented 6 years ago

@sdboyer can you reproduce it ?

xjtian commented 6 years ago

I'm running into this issue as well. It seems somewhat non-deterministic, but only happens if I run dep ensure with an existing vendor/ directory. Happens once every 2-3 runs I'd say, with the same error:

cannot delete .../src/PROJECT/vendor: remove .../src/PROJECT/vendor/golang.org/x/sys/unix: directory not empty

System Info:

Ubuntu 16.04 VM, managed by Virtualbox/Vagrant (bento/ubuntu-16.04 box)
MacOS 10.12.6 host
VM environment has DEPNOLOCK=1 set (https://github.com/golang/dep/issues/947)

Gopkg.toml:

required = ["github.com/golang/protobuf/protoc-gen-go", "github.com/go-swagger/go-swagger/cmd/swagger"]

[[constraint]]
  name = "github.com/go-swagger/go-swagger"
  version = "0.12.0"

[[constraint]]
  branch = "master"
  name = "github.com/go-openapi/strfmt"

[[constraint]]
  branch = "master"
  name = "github.com/golang/glog"

[[constraint]]
  branch = "master"
  name = "github.com/golang/protobuf"

[[constraint]]
  name = "github.com/labstack/echo"
  version = "3.2.5"

[[constraint]]
  branch = "master"
  name = "github.com/lib/pq"

[[constraint]]
  name = "github.com/mattn/go-sqlite3"
  version = "1.4.0"

[[constraint]]
  name = "github.com/prometheus/client_golang"
  version = "0.8.0"

[[constraint]]
  branch = "master"
  name = "github.com/prometheus/client_model"

[[constraint]]
  name = "github.com/stretchr/testify"
  version = "1.1.4"

[[constraint]]
  branch = "master"
  name = "golang.org/x/net"

[[constraint]]
  name = "google.golang.org/grpc"
  version = "1.8.0"

dep hash-inputs:

-CONSTRAINTS-
github.com/go-openapi/strfmt
b-master
github.com/go-swagger/go-swagger
svc-^0.12.0
github.com/golang/glog
b-master
github.com/golang/protobuf
b-master
github.com/labstack/echo
svc-^3.2.5
github.com/lib/pq
b-master
github.com/mattn/go-sqlite3
svc-^1.4.0
github.com/prometheus/client_golang
svc-^0.8.0
github.com/prometheus/client_model
b-master
github.com/stretchr/testify
svc-^1.1.4
golang.org/x/net
b-master
google.golang.org/grpc
svc-^1.8.0
-IMPORTS/REQS-
github.com/go-openapi/errors
github.com/go-openapi/strfmt
github.com/go-openapi/swag
github.com/go-openapi/validate
github.com/go-swagger/go-swagger/cmd/swagger
github.com/golang/glog
github.com/golang/protobuf/jsonpb
github.com/golang/protobuf/proto
github.com/golang/protobuf/protoc-gen-go
github.com/golang/protobuf/ptypes
github.com/golang/protobuf/ptypes/any
github.com/golang/protobuf/ptypes/duration
github.com/golang/protobuf/ptypes/timestamp
github.com/golang/protobuf/ptypes/wrappers
github.com/labstack/echo
github.com/lib/pq
github.com/mattn/go-sqlite3
github.com/prometheus/client_golang/prometheus
github.com/prometheus/client_model/go
github.com/stretchr/testify/assert
github.com/stretchr/testify/mock
golang.org/x/net/context
google.golang.org/grpc
google.golang.org/grpc/codes
google.golang.org/grpc/metadata
google.golang.org/grpc/status
-IGNORES-
-OVERRIDES-
-ANALYZER-
dep
1

Let me know if you need any more info. Thanks for looking into this!

cove commented 6 years ago

I'm able to consistently reproduce this issue.

Env:

Windows 10 Pro (10.0.16299 Build 16299)
Docker 17.12.0-ce-win47 (15139) using Linux containers
Go builder container (golang:1.9.2)

Steps:

Download https://github.com/AccelByte/dep-issue-1407
Type "make build" twice

The 2nd build results in the following error:

docker run --rm -e PROJECT_ROOT=depbug -v C:/ab/depbug:/go/src/depbug -w /go/src/depbug -v /var/run/docker.sock:/var/run/docker.sock  depbug dep ensure -v
Gopkg.lock was already in sync with imports and Gopkg.toml
(1/3) Wrote github.com/go-ini/ini@v1.32.0
(2/3) Wrote github.com/jmespath/go-jmespath@0b12d6b5
(3/3) Wrote github.com/aws/aws-sdk-go@v1.12.70
grouped write of manifest, lock and vendor: cannot delete /go/src/depbug/vendor: remove /go/src/depbug/vendor/github.com/aws/aws-sdk-go/models/apis: directory not empty
make: *** [build] Error 1

djumpen commented 6 years ago

@sdboyer Any chance to get answer on this issue? I faced this issue under vagrant box. On host machine dep works OK, but from under the vagrant box ($GOPATH mounted to Ubuntu box) dep causes errors and creates vendor.orig. The only workaround is to remove vendor folder before every dep ensure

djumpen commented 6 years ago

The problem was caused by interaction of Go and virtualbox fs. NFS is a solution config.vm.synced_folder "~", "~", :nfs => { :mount_options => ["dmode=777","fmode=777"] }

JesseRhoads commented 6 years ago

I have the same environment as xjtian (macos host, virtualbox vm, DEPNOLOCK=1) and this breaks my build every time.
What is bizarre to me is that I can literally delete the files before running ensure and it still hits it, so it's almost like there's an intermediate step in the ensure process that causes this. And sorry, but removing the vendor directory each time or using NFS mounts are not great solutions.

sinramyeon commented 6 years ago

macOs Sierra 10.12.6 dep installed by brew and got same problem

export PATH=$PATH:/usr/local/opt/go/libexec/bin
kimseolhwaui-Mac-mini:engine ksh$ dep ensure
grouped write of manifest, lock and vendor: could not stat file that VerifyVendor claimed existed: stat /Users/ksh/dev/workspace/go/src/engine/vendor/gopkg.in/mgo.v2: no such file or directory

joonas-fi commented 6 years ago

Yeah same error here with Win10 + Vagrant-Virtualbox shared folder mounted inside a Docker container in Ubuntu guest.

Virtualbox shared folders have been notoriously buggy (probably file locks) many years for me no matter using Windows 7 or 10 as a host.

0x7061 commented 6 years ago

Same here on Ubuntu, however I feel like it only happens inside Docker!?!?

rebrendov commented 5 years ago

Hey, I stumbled on a very similar problem and for me the culprit was the chown root vendor made by one of the tools I used. Besides that I would check if the user has a write permission for vendor directory.

markzeras commented 5 years ago

What solved for me was removing /vendor

kamal-github commented 5 years ago

But is this the permanent solution @mrkmat ?

woyoutlz commented 5 years ago

solved by removing /vendor

papag00se commented 5 years ago

Hey, I stumbled on a very similar problem and for me the culprit was the chown root vendor made by one of the tools I used. Besides that I would check if the user has a write permission for vendor directory.

This is what worked for me. In my case I had to open Git Bash on Window as Administrator

golang / dep