Source Unit Name is overloaded

generalpiston commented 8 years ago

Parameters:

git clone https://github.com/leveros/leveros
cd leveros and Modify Srclibtoolchain with the contents below
src config and look out the output below

Problems:

Notice how leveros has no packages other than "." and vendorized deps. "." is listed several times.
Take a peek at the src api units output and notice how there is one unit that isn't vendorized ".". The dependencies from the other packages are completely missing. Example: api is a package that isn't listed. I've found it get's clobbered when writing to cache.

Srclibtoolchain

{
  "Tools": [
    {
      "Subcmd": "scan",
      "Op": "scan",
      "SourceUnitTypes": [
        "GoPackage"
      ]
    },
    {
      "Subcmd": "graph",
      "Op": "graph",
      "SourceUnitTypes": [
        "GoPackage"
      ]
    },
    {
      "Subcmd": "depresolve",
      "Op": "depresolve",
      "SourceUnitTypes": [
        "GoPackage"
      ]
    },
    {
      "Subcmd": "fmt",
      "Op": "fmt",
      "SourceUnitTypes": [
        "GoPackage"
      ]
    }
  ],
  "Bundle": {
    "Paths": [
    "Srclibtoolchain",
    ".bin",
    ".bin/srclib-go"
  ],
    "Commands": [
    "mkdir -p .bin",
    "GOPATH=go get github.com/kardianos/govendor && govendor sync && go build -o .bin/srclib-go"
  ],
    "Variants": [
    {"GOOS": "linux", "GOARCH": "amd64"},
    {"GOOS": "darwin", "GOARCH": "amd64"}
  ]
  }
}

Output

 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: .
 - GoPackage: github.com/Sirupsen/logrus
 - GoPackage: github.com/aerospike/aerospike-client-go
 - GoPackage: github.com/aerospike/aerospike-client-go/internal/lua
 - GoPackage: github.com/aerospike/aerospike-client-go/internal/lua/resources
 - GoPackage: github.com/aerospike/aerospike-client-go/logger
 - GoPackage: github.com/aerospike/aerospike-client-go/pkg/bcrypt
 - GoPackage: github.com/aerospike/aerospike-client-go/pkg/ripemd160
 - GoPackage: github.com/aerospike/aerospike-client-go/types
 - GoPackage: github.com/aerospike/aerospike-client-go/types/atomic
 - GoPackage: github.com/aerospike/aerospike-client-go/types/particle_type
 - GoPackage: github.com/aerospike/aerospike-client-go/types/rand
 - GoPackage: github.com/aerospike/aerospike-client-go/utils/buffer
 - GoPackage: github.com/codegangsta/cli
 - GoPackage: github.com/davecgh/go-spew/spew
 - GoPackage: github.com/fsouza/go-dockerclient
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/Sirupsen/logrus
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/opts
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/archive
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/fileutils
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/homedir
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/idtools
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/ioutils
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/longpath
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/pools
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/promise
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/stdcopy
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/docker/go-units
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/hashicorp/go-cleanhttp
 - GoPackage: github.com/fsouza/go-dockerclient/external/github.com/opencontainers/runc/libcontainer/user
 - GoPackage: github.com/fsouza/go-dockerclient/external/golang.org/x/net/context
 - GoPackage: github.com/fsouza/go-dockerclient/external/golang.org/x/sys/unix
 - GoPackage: github.com/golang/protobuf/proto
 - GoPackage: github.com/hashicorp/consul/api
 - GoPackage: github.com/hashicorp/go-cleanhttp
 - GoPackage: github.com/hashicorp/serf/coordinate
 - GoPackage: github.com/leveros/go-syslog
 - GoPackage: github.com/leveros/go-syslog/format
 - GoPackage: github.com/leveros/gods/containers
 - GoPackage: github.com/leveros/gods/lists
 - GoPackage: github.com/leveros/gods/lists/singlylinkedlist
 - GoPackage: github.com/leveros/gods/maps
 - GoPackage: github.com/leveros/gods/maps/treemap
 - GoPackage: github.com/leveros/gods/stacks
 - GoPackage: github.com/leveros/gods/stacks/linkedliststack
 - GoPackage: github.com/leveros/gods/trees
 - GoPackage: github.com/leveros/gods/trees/redblacktree
 - GoPackage: github.com/leveros/gods/utils
 - GoPackage: github.com/leveros/gods/utils/timsort
 - GoPackage: github.com/leveros/syslogparser
 - GoPackage: github.com/leveros/syslogparser/rfc3164
 - GoPackage: github.com/leveros/syslogparser/rfc5424
 - GoPackage: github.com/mgutz/ansi
 - GoPackage: github.com/miekg/dns
 - GoPackage: github.com/pmezard/go-difflib/difflib
 - GoPackage: github.com/stretchr/testify/assert
 - GoPackage: github.com/x-cray/logrus-prefixed-formatter
 - GoPackage: github.com/yuin/gopher-lua
 - GoPackage: github.com/yuin/gopher-lua/ast
 - GoPackage: github.com/yuin/gopher-lua/parse
 - GoPackage: github.com/yuin/gopher-lua/pm
 - GoPackage: golang.org/x/net/context
 - GoPackage: golang.org/x/net/http2
 - GoPackage: golang.org/x/net/http2/hpack
 - GoPackage: golang.org/x/net/internal/timeseries
 - GoPackage: golang.org/x/net/trace
 - GoPackage: golang.org/x/sys/unix
 - GoPackage: google.golang.org/grpc
 - GoPackage: google.golang.org/grpc/codes
 - GoPackage: google.golang.org/grpc/credentials
 - GoPackage: google.golang.org/grpc/grpclog
 - GoPackage: google.golang.org/grpc/internal
 - GoPackage: google.golang.org/grpc/metadata
 - GoPackage: google.golang.org/grpc/naming
 - GoPackage: google.golang.org/grpc/peer
 - GoPackage: google.golang.org/grpc/transport
 - CommonJSPackage: leveros-server
 - CommonJSPackage: leveros-common
 - CommonJSPackage: leveros
 - LicenseUnit: No_license_found
 - LicenseUnit: Apache-2.0
 - LicenseUnit: Google-BSD
 - LicenseUnit: BSD-3-Clause
 - LicenseUnit: BSD
 - LicenseUnit: See-URL

abbreviated src api units output

Abe|/tmp/leveros> cat /tmp/dat | grep -B 6 '"Type": "GoPackage"' | grep '"Name"'
        "Name": ".",
        "Name": "github.com/Sirupsen/logrus",
        "Name": "github.com/aerospike/aerospike-client-go",
        "Name": "github.com/aerospike/aerospike-client-go/internal/lua",
        "Name": "github.com/aerospike/aerospike-client-go/internal/lua/resources",
        "Name": "github.com/aerospike/aerospike-client-go/logger",
        "Name": "github.com/aerospike/aerospike-client-go/pkg/bcrypt",
        "Name": "github.com/aerospike/aerospike-client-go/pkg/ripemd160",
        "Name": "github.com/aerospike/aerospike-client-go/types",
        "Name": "github.com/aerospike/aerospike-client-go/types/atomic",
        "Name": "github.com/aerospike/aerospike-client-go/types/particle_type",
        "Name": "github.com/aerospike/aerospike-client-go/types/rand",
        "Name": "github.com/aerospike/aerospike-client-go/utils/buffer",
        "Name": "github.com/codegangsta/cli",
        "Name": "github.com/davecgh/go-spew/spew",
        "Name": "github.com/fsouza/go-dockerclient",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/Sirupsen/logrus",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/opts",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/archive",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/fileutils",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/homedir",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/idtools",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/ioutils",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/longpath",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/pools",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/promise",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/stdcopy",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/docker/go-units",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/hashicorp/go-cleanhttp",
        "Name": "github.com/fsouza/go-dockerclient/external/github.com/opencontainers/runc/libcontainer/user",
        "Name": "github.com/fsouza/go-dockerclient/external/golang.org/x/net/context",
        "Name": "github.com/fsouza/go-dockerclient/external/golang.org/x/sys/unix",
        "Name": "github.com/golang/protobuf/proto",
        "Name": "github.com/hashicorp/consul/api",
        "Name": "github.com/hashicorp/go-cleanhttp",
        "Name": "github.com/hashicorp/serf/coordinate",
        "Name": "github.com/leveros/go-syslog",
        "Name": "github.com/leveros/go-syslog/format",
        "Name": "github.com/leveros/gods/containers",
        "Name": "github.com/leveros/gods/lists",
        "Name": "github.com/leveros/gods/lists/singlylinkedlist",
        "Name": "github.com/leveros/gods/maps",
        "Name": "github.com/leveros/gods/maps/treemap",
        "Name": "github.com/leveros/gods/stacks",
        "Name": "github.com/leveros/gods/stacks/linkedliststack",
        "Name": "github.com/leveros/gods/trees",
        "Name": "github.com/leveros/gods/trees/redblacktree",
        "Name": "github.com/leveros/gods/utils",
        "Name": "github.com/leveros/gods/utils/timsort",
        "Name": "github.com/leveros/syslogparser",
        "Name": "github.com/leveros/syslogparser/rfc3164",
        "Name": "github.com/leveros/syslogparser/rfc5424",
        "Name": "github.com/mgutz/ansi",
        "Name": "github.com/miekg/dns",
        "Name": "github.com/pmezard/go-difflib/difflib",
        "Name": "github.com/stretchr/testify/assert",
        "Name": "github.com/x-cray/logrus-prefixed-formatter",
        "Name": "github.com/yuin/gopher-lua",
        "Name": "github.com/yuin/gopher-lua/ast",
        "Name": "github.com/yuin/gopher-lua/parse",
        "Name": "github.com/yuin/gopher-lua/pm",
        "Name": "golang.org/x/net/context",
        "Name": "golang.org/x/net/http2",
        "Name": "golang.org/x/net/http2/hpack",
        "Name": "golang.org/x/net/internal/timeseries",
        "Name": "golang.org/x/net/trace",
        "Name": "golang.org/x/sys/unix",
        "Name": "google.golang.org/grpc",
        "Name": "google.golang.org/grpc/codes",
        "Name": "google.golang.org/grpc/credentials",
        "Name": "google.golang.org/grpc/grpclog",
        "Name": "google.golang.org/grpc/internal",
        "Name": "google.golang.org/grpc/metadata",
        "Name": "google.golang.org/grpc/naming",
        "Name": "google.golang.org/grpc/peer",
        "Name": "google.golang.org/grpc/transport",

alexsaveliev commented 8 years ago

@abec, does it work better when replacing git clone ... srclib config with:

go get github.com/leveros/leveros
cd $GOPATH/src/github.com/leveros/leveros
srclib config

?

generalpiston commented 8 years ago

@alexsaveliev following up from slack, it does indeed work. srclib-go needs GOPATH and the directory structures to be setup properly to work. Thanks for your help. I doubt this is a bug any more, but I'd rather someone else who knows more about Go chime in on this.

beyang commented 8 years ago

Yes, srclib-go assumes that the repository being processed is in the GOPATH. You can look at how it sets up the environment in this Dockerfile for reference: https://sourcegraph.com/sourcegraph/sourcegraph@master/-/blob/services/worker/dockerfiles/Dockerfile.srclib-go

sourcegraph / srclib-go

Source Unit Name is overloaded #93