search

Masterminds / glide

Package Management for Golang
https://glide.sh
Other
8.15k stars 540 forks source link

Trying to understand glide on example of `glide get github.com/aws/aws-sdk-go` #213

Open arvenil opened 8 years ago

arvenil commented 8 years ago

So... for the first time I'm trying out glide (and also experimental vendor flag) and first run is a bit confusing for me. Here goes example:

bash-3.2$ glide create
[INFO] Generating a YAML configuration file and guessing the dependencies
[INFO] Attempting to import from other package managers (use --skip-import to skip)
bash-3.2$ glide get github.com/aws/aws-sdk-go
[INFO] Preparing to install 1 package.
[INFO] Importing github.com/aws/aws-sdk-go
[INFO] Walking the dependency tree to calculate versions
[INFO] Fetching updates for github.com/aws/aws-sdk-go.
[INFO] Scanning github.com/aws/aws-sdk-go for dependencies.
[INFO] ==> Unknown golang.org/x/tools (golang.org/x/tools/go/loader)
[INFO] ==> Unknown github.com/stretchr/testify (github.com/stretchr/testify/assert)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/stretchr/testify (github.com/stretchr/testify/assert)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] ==> Unknown github.com/lsegal/gucumber (github.com/lsegal/gucumber)
[INFO] Fetching updates for github.com/go-ini/ini.
[INFO] Fetching updates for golang.org/x/tools.
[INFO] Fetching updates for github.com/stretchr/testify.
[INFO] Fetching updates for github.com/lsegal/gucumber.
[INFO] Fetching updates for github.com/jmespath/go-jmespath.
[INFO] Scanning github.com/jmespath/go-jmespath for dependencies.
[INFO] Scanning github.com/go-ini/ini for dependencies.
[INFO] Scanning golang.org/x/tools for dependencies.
[INFO] Fetching updates for golang.org/x/net.
[INFO] Scanning golang.org/x/net for dependencies.
[INFO] ==> Unknown golang.org/x/text (golang.org/x/text/encoding)
[INFO] ==> Unknown golang.org/x/text (golang.org/x/text/encoding/charmap)
[INFO] ==> Unknown golang.org/x/text (golang.org/x/text/encoding/htmlindex)
[INFO] ==> Unknown golang.org/x/text (golang.org/x/text/transform)
[INFO] ==> Unknown golang.org/x/crypto (golang.org/x/crypto/ssh/terminal)
[INFO] Fetching updates for golang.org/x/crypto.
[INFO] Fetching updates for golang.org/x/text.
[INFO] Scanning golang.org/x/text for dependencies.
[INFO] Scanning golang.org/x/text for dependencies.
[INFO] Scanning golang.org/x/text for dependencies.
[INFO] Scanning golang.org/x/text for dependencies.
[INFO] Scanning golang.org/x/crypto for dependencies.
[INFO] Found Godeps.json file.
[INFO] Found Godeps.json file for "github.com/stretchr/testify"
[INFO] Fetching updates for github.com/stretchr/objx.
[INFO] Setting version for github.com/stretchr/objx to cbeaeb16a013161a98496fad62933b1d21786672.
[INFO] Fetching updates for github.com/davecgh/go-spew.
[INFO] Setting version for github.com/davecgh/go-spew to 2df174808ee097f90d259e432cc04442cf60be21.
[INFO] Fetching updates for github.com/pmezard/go-difflib.
[INFO] Setting version for github.com/pmezard/go-difflib to d8ed2627bdf02c080bf22230dbb337003b7aba2d.
[INFO] Scanning github.com/davecgh/go-spew for dependencies.
[INFO] Scanning github.com/pmezard/go-difflib for dependencies.
[INFO] Scanning github.com/stretchr/objx for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] ==> Unknown github.com/shiena/ansicolor (github.com/shiena/ansicolor)
[INFO] Fetching updates for github.com/shiena/ansicolor.
[INFO] Scanning github.com/shiena/ansicolor for dependencies.
[INFO] Found Godeps.json file.
[INFO] Found Godeps.json file for "github.com/stretchr/testify"
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Scanning github.com/lsegal/gucumber for dependencies.
[INFO] Setting version for github.com/davecgh/go-spew to 2df174808ee097f90d259e432cc04442cf60be21.
[INFO] Setting version for github.com/pmezard/go-difflib to d8ed2627bdf02c080bf22230dbb337003b7aba2d.
[INFO] Setting version for github.com/stretchr/objx to cbeaeb16a013161a98496fad62933b1d21786672.
[INFO] Project relies on 13 dependencies.
[INFO] Writing glide.lock file

My questions:

  1. Why glide get github.com/aws/aws-sdk-go ended up fetching 13 dependencies? Shouldn't it just download github.com/aws/aws-sdk-go and nothing else since this project has a vendor dir? From how it works section:

    It then recursively walks through the downloaded packages looking for those with a glide.yaml file (or Godep, gb, or GPM config file) that don't already have a vendor directory and installing their dependencies to their vendor directories. but it has vendor directory, shouldn't it then just get aws package and nothing else?

  2. Unknown github.com/lsegal/gucumber what this means and why it's repeated so many times (making output hard to read)
mattfarina commented 8 years ago

@arvenil I think I can answer your question. This is where vendor/ practices get confusing and I'm happy to take feedback here.

  1. github.com/aws/aws-sdk-go uses all those dependencies within its codebase. Now, it does have 2 dependencies in a vendor/ folder. Where things get difficult is that each vendor/ folder is a different location and the same dependency in two different vendor/ folders, even if the same version, cannot share instances of types and will be in the compiled binary each time. This can lead to build issues and binary bloat. So, the recommendation, which Glide does automatically, is to only have one vendor/ folder at the top level of an application. To flatten the dependencies. Do you have a suggestion on how to do this better? I'm happy to have a conversation on it. If you want to chat in realtime you can find me in the #masterminds room on Freenode IRC.
  2. That messaging is poor on our part. We're currently reworking this part of the logic and the next release should have better messaging.
arvenil commented 8 years ago
  1. My point is, the doc says that in case if there is already vendor dir it's not going to fetch any dependencies, just vendor dir. However glide currently works differently, it doesn't care if vendor dir is there, it will always scan everything for dependencies. I'm not saying behavior is wrong, I'm saying that either I don't understand doc or doc says it wrong.
  2. ok
  3. I have another problem, that something like glide get github.com/z/somepackage scans all subpackages for dependencies, while I not necessary might want it. Maybe I only want somepackage package, not any other subpackage. I think this is somehow related to #166 #164 I would expect to be able glide get github.com/z/somepackage or glide get github.com/z/somepackage/... or glide get github.com/z/somepackage/subpackage

(and I also run right now into a problem when glide get ended up in infinite dependency loop:/ but didn't figure out yet, why the hell:/ nvm, my fault)

mattfarina commented 8 years ago

@arvenil We're currently working to make #1 and #3 better. Optimization can be hard.

technosophos commented 8 years ago

@mattfarina Can we close now that the solution to #1 and #3 are merged onto master?

@arvenil If you feel like re-testing off of master, we'd appreciate any input. This new version should be much better at selectively importing what is necessary, not blindly importing everything.

arvenil commented 8 years ago

Wow, indeed it works better, fetches way less dependencies. 31 vs 51. Good job guys!

I still however have this remaining question for 1. The how it works section states:

It then recursively walks through the downloaded packages looking for those with a glide.yaml file (or Godep, gb, or GPM config file) that don't already have a vendor directory and installing their dependencies to their vendor directories.

but

  1. github.com/aws/aws-sdk-go has vendor directory and now I'm not sure if the new version works like the doc says, so github.com/aws/aws-sdk-go is not scanned at all for dependencies because it has a vendor dir, or it is still scanned and the doc is misleading?
  2. and installing their dependencies to their vendor directories is also false now? As glide installs dependencies to top level vendor dir?
arvenil commented 8 years ago

I also found out one inconsistency with fetching dependencies that is related to my questions. I think code is the simplest way to explain this :)

glide get github.com/aws/aws-sdk-go fetches only one dep... so it looks like it cares about vendor dir and doesn't scan it for dependencies?

bash-3.2$ glide create
[INFO] Generating a YAML configuration file and guessing the dependencies
[INFO] Attempting to import from other package managers (use --skip-import to skip)
bash-3.2$ glide get github.com/aws/aws-sdk-go
[INFO] Preparing to install 1 package.
[INFO] Importing github.com/aws/aws-sdk-go
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/aws/aws-sdk-go.
[INFO] Resolving imports
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/aws/aws-sdk-go.
bash-3.2$ cat glide.lock
hash: f0e0abd12e6cf5a81e2863ad24fcafedae90eef238a1108ed41c6efaa6bff74f
updated: 2016-02-10T14:16:53.052542434Z
imports:
- name: github.com/aws/aws-sdk-go
  version: 0d8e5fdff6a7f73e2c8b84c02330c3c030650b82
devImports: []

Now I'm trying to fetch dependency (github.com/uniplaces/osin-dynamodb) that depends on github.com/aws/aws-sdk-go (more precisely on github.com/aws/aws-sdk-go/service/dynamodb)

bash-3.2$ rm glide.lock glide.yaml
bash-3.2$ glide create
[INFO] Generating a YAML configuration file and guessing the dependencies
[INFO] Attempting to import from other package managers (use --skip-import to skip)
bash-3.2$ glide get github.com/uniplaces/osin-dynamodb
[INFO] Preparing to install 1 package.
[INFO] Importing github.com/uniplaces/osin-dynamodb
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/uniplaces/osin-dynamodb.
[INFO] Resolving imports
[INFO] Fetching updates for github.com/RangelReale/osin.
[INFO] Fetching updates for github.com/aws/aws-sdk-go.
[INFO] Fetching updates for github.com/pborman/uuid.
[INFO] Fetching updates for github.com/go-ini/ini.
[INFO] Fetching updates for github.com/jmespath/go-jmespath.
[INFO] Downloading dependencies. Please wait...
[INFO] Fetching updates for github.com/uniplaces/osin-dynamodb.
bash-3.2$ cat glide.lock
hash: f9b467e54af5991537817e9f40ef526311e41a8b51dfae4527f5fd17c2fa06c5
updated: 2016-02-10T14:32:49.801943376Z
imports:
- name: github.com/aws/aws-sdk-go
  version: 0d8e5fdff6a7f73e2c8b84c02330c3c030650b82
  subpackages:
  - aws
  - service/dynamodb
  - aws/awserr
  - aws/credentials
  - aws/awsutil
  - aws/client
  - aws/client/metadata
  - aws/request
  - private/protocol/jsonrpc
  - private/signer/v4
  - private/waiter
  - private/protocol/json/jsonutil
  - private/protocol/rest
  - private/protocol
- name: github.com/go-ini/ini
  version: 193d1ecb466bf97aae8b454a5cfc192941c64809
- name: github.com/jmespath/go-jmespath
  version: 0b12d6b521d83fc7f755e7cfc1b1fbdd35a01a74
- name: github.com/pborman/uuid
  version: 0f1a46960a86dcdf5dd30d3e6568a497a997909f
- name: github.com/RangelReale/osin
  version: 2d8139c3efb5d19649fea4215969840122a39f8e
- name: github.com/uniplaces/osin-dynamodb
  version: a96212a5fd9159d83d80ed956032204e238fe65e
devImports: []

Notice github.com/jmespath/go-jmespath in dependencies. This is purely dependency of github.com/aws/aws-sdk-go but why it shows if it's in github.com/aws/aws-sdk-go vendor dir? Especially that glide get github.com/aws/aws-sdk-go didn't fetch it. I'm just guessing that this is related to the fact that github.com/uniplaces/osin-dynamodb is dependent on subpackage of github.com/aws/aws-sdk-go and maybe so glide doesn't see a vendor dir because he doesn't check parent dir (or in other words root package).

Below is a proof that vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath is actually used, and that vendor/github.com/jmespath/go-jmespath is unnecessary fetched (and unused) dependency.

bash-3.2$ ls vendor/github.com/jmespath/go-jmespath
LICENSE         api_test.go     compliance_test.go  interpreter_test.go parser_test.go
Makefile        astnodetype_string.go   functions.go        lexer.go        toktype_string.go
README.md       cmd         fuzz            lexer_test.go       util.go
api.go          compliance      interpreter.go      parser.go       util_test.go

bash-3.2$ go build -v -x 2>&1 | grep --color go-jmespath
github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath
mkdir -p $WORK/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath/_obj/
cd /Users/kamil/go/src/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath
/Users/kamil/.gobrew/versions/1.5.3/pkg/tool/darwin_amd64/compile -o $WORK/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath.a -trimpath $WORK -p github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath -complete -buildid 4da3618a1da31cc431aadbdc9ac0d7338d5167e6 -D _/Users/kamil/go/src/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath -I $WORK -pack ./api.go ./astnodetype_string.go ./functions.go ./interpreter.go ./lexer.go ./parser.go ./toktype_string.go ./util.go
/Users/kamil/.gobrew/versions/1.5.3/pkg/tool/darwin_amd64/compile -o $WORK/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/aws/awsutil.a -trimpath $WORK -p github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/aws/awsutil -complete -buildid 42944d3fe42c2f8d7884f09360a5cc68c0d11f05 -importmap github.com/jmespath/go-jmespath=github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath -D _/Users/kamil/go/src/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/aws/awsutil -I $WORK -I /Users/kamil/go/pkg/darwin_amd64 -pack ./copy.go ./equal.go ./path_value.go ./prettify.go ./string_value.go

bash-3.2$ rm -rf vendor/github.com/jmespath/go-jmespath

bash-3.2$ go build -v -x 2>&1 | grep --color go-jmespath
github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath
mkdir -p $WORK/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath/_obj/
cd /Users/kamil/go/src/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath
/Users/kamil/.gobrew/versions/1.5.3/pkg/tool/darwin_amd64/compile -o $WORK/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath.a -trimpath $WORK -p github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath -complete -buildid 4da3618a1da31cc431aadbdc9ac0d7338d5167e6 -D _/Users/kamil/go/src/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath -I $WORK -pack ./api.go ./astnodetype_string.go ./functions.go ./interpreter.go ./lexer.go ./parser.go ./toktype_string.go ./util.go
/Users/kamil/.gobrew/versions/1.5.3/pkg/tool/darwin_amd64/compile -o $WORK/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/aws/awsutil.a -trimpath $WORK -p github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/aws/awsutil -complete -buildid 42944d3fe42c2f8d7884f09360a5cc68c0d11f05 -importmap github.com/jmespath/go-jmespath=github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/vendor/github.com/jmespath/go-jmespath -D _/Users/kamil/go/src/github.com/uniplaces/glide-vendor-test1/vendor/github.com/aws/aws-sdk-go/aws/awsutil -I $WORK -I /Users/kamil/go/pkg/darwin_amd64 -pack ./copy.go ./equal.go ./path_value.go ./prettify.go ./string_value.go

bash-3.2$ cat main.go
package main

import (
    _ "github.com/uniplaces/osin-dynamodb"
)

func main() {
    panic("test")
}
arvenil commented 8 years ago

I also wonder if there is a way to ignore vendor/.../vendor/ dirs to avoid some complications with including several times the same library... I know I could just rm -rf all vendor dirs under root vendor dir... but maybe glide provides some functionality to do that.

(beside, that, let me state this... first package manager for golang that actually works for me... great job)

mattfarina commented 8 years ago

@arvenil I can try to answer your questions here. And, there may be room for optimization in Glide.

First, Glide complements the go tool. So, building binaries and so forth are handled there. This is important because any ignoring of vendor/.../vendor/ to build a binary has to happen in the go tool. Glide can't affect that.

It's generally considered a bad practice for a library to have packages in the vendor/ folder unless they explicitly have a reason for it. For example, they are using a fork or it only works with a specific version. Otherwise you get into binary bloat and issues of sharing instances of types.

Glide does not check if a package is already in the vendor/ folder and skip downloading it. That could be another optimization.

I'm going to have to look at how glide get is generating the lock file. The lock should have github.com/aws/aws-sdk-go and any known dependencies. I'll check that for a bug. Thanks.

mattfarina commented 8 years ago

@arvenil I can explain what's happening now.

  1. When glide get downloads a package it inspects that packages and fetches it's dependencies. The root of github.com/aws/aws-sdk-go has no sub-packages to fetch so it doesn't. If you had should code that used it in your application it would walk the used imports and fetch more.
  2. When glide up is run to update it looks at the import path used in your code and follows those import paths to fetch everything imported. The parts of github.com/aws/aws-sdk-go and their dependencies will be fetched when used in your application code.

We can likely give this a better experience somehow.