search

kamilchm / go2nix

Reproducible builds and development environment for Go
MIT License
94 stars 17 forks source link

problem with packaging #21

Closed qknight closed 7 years ago

qknight commented 8 years ago

i've been using this go2nix and created a deps.nix which i converted into a deps.json manually.

the package

default.nix

{ stdenv, buildGoPackage, fetchgit, fetchhg, fetchbzr, fetchsvn }:

buildGoPackage rec {
  name = "leaps-${version}";
  version = "20160626-${stdenv.lib.strings.substring 0 7 rev}";
  rev = "5cf7328a8c498041d2a887e89f22f138498f4621";

  goPackagePath = "github.com/jeffail/leaps";

  src = fetchgit {
    inherit rev;
    url = "https://github.com/jeffail/leaps";
    sha256 = "1qbgz48x9yi0w9yz39zsnnhx5nx2xmrns9v8hx28jah2bvag6sq7";
    fetchSubmodules = false;  
  };

  goDeps = ./deps.json;

  preConfigure = ''
    rm -Rf src
  '';

  # TODO: add metadata https://nixos.org/nixpkgs/manual/#sec-standard-meta-attributes
  meta = {
  };
}

deps.json

[
  {
    "goPackagePath" : "github.com/amir/raidman",
    "fetch": {
      "type": "git",
      "url": "https://github.com/amir/raidman",
      "rev":  "91c20f3f475cab75bb40ad7951d9bbdde357ade7",
      "sha256": "0pkqy5hzjkk04wj1ljq8jsyla358ilxi4lkmvkk73b3dh2wcqvpp"
    }
  },
  {
    "goPackagePath" : "github.com/elazarl/go-bindata-assetfs",
    "fetch" : {
      "type": "git",
      "url": "https://github.com/elazarl/go-bindata-assetfs",
      "rev": "57eb5e1fc594ad4b0b1dbea7b286d299e0cb43c2",
      "sha256": "1za29pa15y2xsa1lza97jlkax9qj93ks4a2j58xzmay6rczfkb9i"
    }
  },
  {
   "goPackagePath": "github.com/garyburd/redigo",
    "fetch": {
      "type": "git",
       "url": "https://github.com/garyburd/redigo",
       "rev": "8873b2f1995f59d4bcdd2b0dc9858e2cb9bf0c13",
       "sha256": "1lzhb99pcwwf5ddcs0bw00fwf9m1d0k7b92fqz2a01jlij4pm5l2"
    }
  },
  {
   "goPackagePath": "github.com/go-sql-driver/mysql",
    "fetch": {
      "type": "git",
       "url": "https://github.com/go-sql-driver/mysql",
       "rev": "7ebe0a500653eeb1859664bed5e48dec1e164e73",
       "sha256": "1gyan3lyn2j00di9haq7zm3zcwckn922iigx3fvml6s2bsp6ljas"
    }
  },
  {
   "goPackagePath": "github.com/golang/protobuf",
    "fetch": {
      "type": "git",
       "url": "https://github.com/golang/protobuf",
       "rev": "bf531ff1a004f24ee53329dfd5ce0b41bfdc17df",
       "sha256": "10lnvmq28jp2wk1xc32mdk4745lal2bmdvbjirckb9wlv07zzzf0"
    }
  },
  {
   "goPackagePath": "github.com/jeffail/gabs",
    "fetch": {
      "type": "git",
       "url": "https://github.com/jeffail/gabs",
       "rev": "ee1575a53249b51d636e62464ca43a13030afdb5",
       "sha256": "0svv57193n8m86r7v7n0y9lny0p6nzr7xvz98va87h00mg146351"
    }
  },
  {
   "goPackagePath": "github.com/jeffail/util",
    "fetch": {
      "type": "git",
       "url": "https://github.com/jeffail/util",
       "rev": "48ada8ff9fcae546b5986f066720daa9033ad523",
       "sha256": "0k8zz7gdv4hb691fdyb5mhlixppcq8x4ny84fanflypnv258a3i0"
    }
  },
  {
   "goPackagePath": "github.com/lib/pq",
    "fetch": {
      "type": "git",
       "url": "https://github.com/lib/pq",
       "rev": "3cd0097429be7d611bb644ef85b42bfb102ceea4",
       "sha256": "1q7qfzyfgjk6rvid548r43fi4jhvsh4dhfvfjbp2pz4xqsvpsm7a"
    }
  },
  {
   "goPackagePath": "github.com/satori/go.uuid",
    "fetch": {
      "type": "git",
       "url": "https://github.com/satori/go.uuid",
       "rev": "f9ab0dce87d815821e221626b772e3475a0d2749",
       "sha256": "0z18j6zxq9kw4lgcpmhh3k7jrb9gy1lx252xz5qhs4ywi9w77xwi"
    }
  },
  {
   "goPackagePath": "github.com/jeffail/leaps",
    "fetch": {
      "type": "git",
       "url": "https://github.com/jeffail/leaps",
       "rev": "5cf7328a8c498041d2a887e89f22f138498f4621",
       "sha256": "1qbgz48x9yi0w9yz39zsnnhx5nx2xmrns9v8hx28jah2bvag6sq7"
    }
  },
  {
   "goPackagePath": "golang.org/x/net",
    "fetch": {
      "type": "git",
       "url": "https://go.googlesource.com/net",
       "rev": "07b51741c1d6423d4a6abab1c49940ec09cb1aaf",
       "sha256": "12lvdj0k2gww4hw5f79qb9yswqpy4i3bgv1likmf3mllgdxfm20w"
    }
  }
]

the error

nix-env -f /home/joachim/Desktop/projects/nixos/nixpkgs/default.nix -I nixpkgs=/home/joachim/Desktop/projects/nixos/nixpkgs -iA leaps
installing ‘go1.6-leaps-20160626-5cf7328’
these derivations will be built:
  /nix/store/yishlw4wwp937zkb0vzd39qqmw7l10w5-go1.6-leaps-20160626-5cf7328.drv
building path(s) ‘/nix/store/fc1dn5fv2jiix62zfhm7d0n4wibmv8q8-go1.6-leaps-20160626-5cf7328’, ‘/nix/store/lwrndn6hyi9bl2n900v1s671na2bcwz9-go1.6-leaps-20160626-5cf7328-bin’
unpacking sources
unpacking source archive /nix/store/27vc3vfkb7cjpjjnbdni6bmajkbbnf1a-leaps-5cf7328
source root is leaps-5cf7328
patching sources
configuring
grep: Invalid range end
unpacking source archive /nix/store/kwf6af210nhpipxki0hffz52zsc4rj4v-raidman-91c20f3
unpacking source archive /nix/store/8i5kw5i82r10psg9fni49sdh736h7ag7-go-bindata-assetfs-57eb5e1
unpacking source archive /nix/store/l63kp458n182fymdc8w5wl3s41hw9bpc-redigo-8873b2f
unpacking source archive /nix/store/wcw7vfm7abz3bbmsyrwr392vkg6p6p7p-mysql-7ebe0a5
unpacking source archive /nix/store/dmnbwirpnbkaxb1gxr4z2j27iix70sq2-protobuf-bf531ff
unpacking source archive /nix/store/j0ipypgkbkfm0p9mri71w449jhjf2cvx-gabs-ee1575a
unpacking source archive /nix/store/cpk6ssqvb44jx6kg5vcp231kc0fkj89k-util-48ada8f
unpacking source archive /nix/store/0g6yrisq8dsid473ygkpgw7ywlvgzd3g-pq-3cd0097
unpacking source archive /nix/store/mmzrswsnnsg65i26m86395nrling058g-go.uuid-f9ab0dc
unpacking source archive /nix/store/27vc3vfkb7cjpjjnbdni6bmajkbbnf1a-leaps-5cf7328
unpacking source archive /nix/store/1ki1728win67jg7jj3rqiww9nxqv8gkb-net-07b5174
building
github.com/jeffail/leaps/vendor/github.com/elazarl/go-bindata-assetfs
github.com/jeffail/leaps/vendor/github.com/garyburd/redigo/internal
github.com/jeffail/leaps/vendor/github.com/jeffail/util/log
github.com/jeffail/leaps/vendor/github.com/go-sql-driver/mysql
github.com/jeffail/leaps/vendor/github.com/garyburd/redigo/redis
github.com/jeffail/leaps/vendor/github.com/satori/go.uuid
github.com/jeffail/leaps/vendor/github.com/lib/pq/oid
github.com/jeffail/leaps/vendor/github.com/lib/pq
github.com/jeffail/leaps/lib/util
github.com/jeffail/leaps/vendor/github.com/golang/protobuf/proto
github.com/jeffail/leaps/lib/acl
golang.org/x/net/proxy
github.com/jeffail/leaps/vendor/github.com/jeffail/gabs
golang.org/x/net/websocket
github.com/jeffail/leaps/lib/store
github.com/jeffail/leaps/vendor/github.com/amir/raidman/proto
github.com/jeffail/leaps/vendor/github.com/amir/raidman
github.com/jeffail/leaps/vendor/github.com/jeffail/util/metrics
github.com/jeffail/leaps/lib/binder
github.com/jeffail/leaps/lib/curator
github.com/jeffail/leaps/lib/http
github.com/jeffail/leaps/cmd/leaps
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/elazarl/go-bindata-assetfs
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/golang/protobuf/proto
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/gabs
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/amir/raidman/proto
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/amir/raidman
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/metrics
github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps
# github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps
go/src/github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps/leaps.go:114: cannot use logger (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to acl.NewFileExists:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps/leaps.go:118: cannot use logger (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to curator.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps/leaps.go:166: cannot use logger (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to "github.com/jeffail/leaps/lib/http".WebsocketHandler:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/garyburd/redigo/internal
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/garyburd/redigo/redis
github.com/jeffail/leaps/leaps-5cf7328/lib/acl
github.com/jeffail/leaps/leaps-5cf7328/lib/binder
github.com/jeffail/leaps/leaps-5cf7328/lib/curator
# github.com/jeffail/leaps/leaps-5cf7328/lib/curator
go/src/github.com/jeffail/leaps/leaps-5cf7328/lib/curator/curator.go:256: cannot use c.log (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to binder.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/lib/curator/curator.go:299: cannot use c.log (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to binder.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/lib/curator/curator.go:340: cannot use c.log (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to binder.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
github.com/jeffail/leaps/leaps-5cf7328/lib/http
github.com/jeffail/leaps/leaps-5cf7328/lib/register
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/go-sql-driver/mysql
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/lib/pq/oid
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/lib/pq
github.com/jeffail/leaps/leaps-5cf7328/lib/store
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/satori/go.uuid
github.com/jeffail/leaps/leaps-5cf7328/lib/util
github.com/jeffail/leaps/lib/register
builder for ‘/nix/store/yishlw4wwp937zkb0vzd39qqmw7l10w5-go1.6-leaps-20160626-5cf7328.drv’ failed with exit code 2
error: build of ‘/nix/store/yishlw4wwp937zkb0vzd39qqmw7l10w5-go1.6-leaps-20160626-5cf7328.drv’ failed

clue

according to golang on irc this might be because the go-modules/generic/default.nix applies various vendor patches i don't understand yet.

do you have a clue what has gone wrong?

qknight commented 8 years ago

in go-modules/generic/default.nix i removed this lines:

@@ -109,13 +111,16 @@ go.stdenv.mkDerivation (
       local d; local cmd;
       cmd="$1"
       d="$2"
       echo "$d" | grep -q "\(/_\|examples\|Godeps\)" && return 0
       [ -n "$excludedPackages" ] && echo "$d" | grep -q "$excludedPackages" && return 0
       local OUT
       if ! OUT="$(go $cmd $buildFlags "''${buildFlagsArray[@]}" -v $d 2>&1)"; then
         if ! echo "$OUT" | grep -q 'no buildable Go source files'; then
-          echo "$OUT" >&2
-          return 1
+          #echo "$OUT" >&2
+          return 0

and now it compiled & installed

qknight commented 8 years ago

if i reduce the deps it'll just work. so there seems to be a collision of some sort which is caught but i don't understand why go2nix outputs a deps.nix which then has too many entries.

deps.json

[
  {
   "goPackagePath": "golang.org/x/net",
    "fetch": {
      "type": "git",
       "url": "https://go.googlesource.com/net",
       "rev": "07b51741c1d6423d4a6abab1c49940ec09cb1aaf",
       "sha256": "12lvdj0k2gww4hw5f79qb9yswqpy4i3bgv1likmf3mllgdxfm20w"
    }
  }
]

compile output

nix-env -f /home/joachim/Desktop/projects/nixos/nixpkgs/default.nix -I nixpkgs=/home/joachim/Desktop/projects/nixos/nixpkgs -iA leaps.bin         ~/Desktop/projects/nixos/nixpkgs/pkgs
replacing old ‘go1.6-leaps-20160626-5cf7328’
installing ‘go1.6-leaps-20160626-5cf7328’
these derivations will be built:
  /nix/store/vfzq83lfws46wisjvx78vvc07i6d51rs-go1.6-leaps-20160626-5cf7328.drv
building path(s) ‘/nix/store/h01809qll0llrhh52jq4v7xjf4zkfrp1-go1.6-leaps-20160626-5cf7328’, ‘/nix/store/m88pyqrd5syzr8h70jnk6ag9yccx78jy-go1.6-leaps-20160626-5cf7328-bin’
unpacking sources
unpacking source archive /nix/store/27vc3vfkb7cjpjjnbdni6bmajkbbnf1a-leaps-5cf7328
source root is leaps-5cf7328
patching sources
configuring
preconfigure flux
grep: Invalid range end
preconfigure flux II
unpacking source archive /nix/store/1ki1728win67jg7jj3rqiww9nxqv8gkb-net-07b5174
building
===== getGoDirs 
fluuuuuuxii 2
--11- github.com/jeffail/leaps --11-
--22- github.com/jeffail/leaps/cmd/leaps
github.com/jeffail/leaps/lib/acl
github.com/jeffail/leaps/lib/binder
github.com/jeffail/leaps/lib/curator
github.com/jeffail/leaps/lib/http
github.com/jeffail/leaps/lib/register
github.com/jeffail/leaps/lib/store
github.com/jeffail/leaps/lib/util --22-
fluuuuuuxii 2
github.com/jeffail/leaps/cmd/leaps github.com/jeffail/leaps/lib/acl github.com/jeffail/leaps/lib/binder github.com/jeffail/leaps/lib/curator github.com/jeffail/leaps/lib/http github.com/jeffail/leaps/lib/register github.com/jeffail/leaps/lib/store github.com/jeffail/leaps/lib/util
go install  ... github.com/jeffail/leaps/cmd/leaps github.com/jeffail/leaps/lib/acl github.com/jeffail/leaps/lib/binder github.com/jeffail/leaps/lib/curator github.com/jeffail/leaps/lib/http github.com/jeffail/leaps/lib/register github.com/jeffail/leaps/lib/store github.com/jeffail/leaps/lib/util
===== buildGoDir install, github.com/jeffail/leaps/cmd/leaps github.com/jeffail/leaps/lib/acl github.com/jeffail/leaps/lib/binder github.com/jeffail/leaps/lib/curator github.com/jeffail/leaps/lib/http github.com/jeffail/leaps/lib/register github.com/jeffail/leaps/lib/store github.com/jeffail/leaps/lib/util
---flux 1---
---/flux 1---
---flux 11 ---
---/flux 11 ---
github.com/jeffail/leaps/vendor/github.com/garyburd/redigo/internal
github.com/jeffail/leaps/vendor/github.com/jeffail/util/log
github.com/jeffail/leaps/vendor/github.com/elazarl/go-bindata-assetfs
github.com/jeffail/leaps/vendor/github.com/garyburd/redigo/redis
github.com/jeffail/leaps/vendor/github.com/go-sql-driver/mysql
github.com/jeffail/leaps/vendor/github.com/satori/go.uuid
github.com/jeffail/leaps/vendor/github.com/lib/pq/oid
github.com/jeffail/leaps/vendor/github.com/lib/pq
github.com/jeffail/leaps/lib/util
github.com/jeffail/leaps/vendor/github.com/golang/protobuf/proto
github.com/jeffail/leaps/lib/acl
golang.org/x/net/proxy
github.com/jeffail/leaps/vendor/github.com/jeffail/gabs
golang.org/x/net/websocket
github.com/jeffail/leaps/lib/store
github.com/jeffail/leaps/lib/register
github.com/jeffail/leaps/vendor/github.com/amir/raidman/proto
github.com/jeffail/leaps/vendor/github.com/amir/raidman
github.com/jeffail/leaps/vendor/github.com/jeffail/util/metrics
github.com/jeffail/leaps/lib/binder
github.com/jeffail/leaps/lib/curator
github.com/jeffail/leaps/lib/http
github.com/jeffail/leaps/cmd/leaps
installing
/tmp/nix-build-go1.6-leaps-20160626-5cf7328.drv-0/go /tmp/nix-build-go1.6-leaps-20160626-5cf7328.drv-0
/tmp/nix-build-go1.6-leaps-20160626-5cf7328.drv-0
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/h01809qll0llrhh52jq4v7xjf4zkfrp1-go1.6-leaps-20160626-5cf7328
patching script interpreter paths in /nix/store/h01809qll0llrhh52jq4v7xjf4zkfrp1-go1.6-leaps-20160626-5cf7328
/nix/store/h01809qll0llrhh52jq4v7xjf4zkfrp1-go1.6-leaps-20160626-5cf7328/share/go/src/github.com/jeffail/leaps/vendor/github.com/golang/protobuf/ptypes/regen.sh: interpreter directive changed from "/bin/bash -e" to "/nix/store/d44582rghk696yw704sh5nbvbpnm69iv-bash-4.3-p42/bin/bash -e"
shrinking RPATHs of ELF executables and libraries in /nix/store/m88pyqrd5syzr8h70jnk6ag9yccx78jy-go1.6-leaps-20160626-5cf7328-bin
shrinking /nix/store/m88pyqrd5syzr8h70jnk6ag9yccx78jy-go1.6-leaps-20160626-5cf7328-bin/bin/leaps
stripping (with flags -S) in /nix/store/m88pyqrd5syzr8h70jnk6ag9yccx78jy-go1.6-leaps-20160626-5cf7328-bin/bin 
patching script interpreter paths in /nix/store/m88pyqrd5syzr8h70jnk6ag9yccx78jy-go1.6-leaps-20160626-5cf7328-bin
building path(s) ‘/nix/store/1b4l4csk97b51j2cxavydnhkdy9n1l2h-user-environment’
created 60 symlinks in user environment
kamilchm commented 8 years ago

Yep, you're right there's no need to analyze dependencies from vendor in go2nix. I prepared a workaround and tested it on leaps https://github.com/kamilchm/go2nix/commit/83d4e614d26d687f5d2ee05b488decf4739d90bb I'll try to make a test case for it later and do it then.

kamilchm commented 7 years ago

fixed in https://github.com/kamilchm/go2nix/releases/tag/v1.1.1