search

bamzi / jobrunner

Framework for performing work asynchronously, outside of the request flow
MIT License
1.04k stars 99 forks source link

Help with Dep #20

Closed ddavtian closed 4 years ago

ddavtian commented 4 years ago

First of all wanted to chime in and say that jobrunner is fantastic and I've been using it for some time now. Recently I seem to be having issues with trying to install the dependencies with dep after the v3 upgrade from cron

dep:
 version     : v0.5.4
 build date  : 2019-07-01
 git hash    : 1f7c19e
 go version  : go1.12.6
 go compiler : gc
 platform    : darwin/amd64
 features    : ImportDuringSolve=false
[[constraint]]
  branch = "master"
  name = "github.com/bamzi/jobrunner"

[[constraint]]
  branch = "master"
  name = "github.com/asaskevich/EventBus"

[[constraint]]
  branch = "master"
  name = "github.com/docker/docker"

[[constraint]]
  name = "github.com/enriquebris/goconcurrentqueue"
  version = "0.1.0"

[[constraint]]
  name = "github.com/fsnotify/fsnotify"
  version = "1.4.7"

[[constraint]]
  name = "github.com/labstack/echo"
  version = "3.3.10"

[[constraint]]
  name = "github.com/satori/go.uuid"
  version = "1.2.0"

[[constraint]]
  name = "github.com/sirupsen/logrus"
  version = "1.4.1"

[[constraint]]
  name = "github.com/spf13/viper"
  version = "1.4.0"

[[constraint]]
  branch = "master"
  name = "golang.org/x/net"

[[constraint]]
  name = "gopkg.in/resty.v1"
  version = "1.12.0"

[prune]
  go-tests = true
  unused-packages = true

On a clean install, so nothing in vendor and nothing in pkg/dep directory getting the following error when I run

dep ensure -v

Solving failure: No versions of github.com/robfig/cron met constraints:
    v3.0.0: Could not introduce github.com/robfig/cron@v3.0.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1.2.0: Could not introduce github.com/robfig/cron@v1.2.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1.1.0: Could not introduce github.com/robfig/cron@v1.1.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1.0.0: Could not introduce github.com/robfig/cron@v1.0.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v3.0.0-rc1: Could not introduce github.com/robfig/cron@v3.0.0-rc1, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    master: Could not introduce github.com/robfig/cron@master, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1: Could not introduce github.com/robfig/cron@v1, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v2: Could not introduce github.com/robfig/cron@v2, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v3: Could not introduce github.com/robfig/cron@v3, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v3-dev: Could not introduce github.com/robfig/cron@v3-dev, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master

All the imports in the code reference

import (
"github.com/bamzi/jobrunner"
"github.com/robfig/cron/v3"
)

Any help is very much appreciated and thank you

bamzi commented 4 years ago

have you tried go get -u github.com/bamzi/jobrunner AND go get github.com/robfig/cron/v3@v3.0.0

ddavtian commented 4 years ago

Hi @bamzi thank you very much for commenting back. Output of the commands below, seems like the second one did not succeed.

go get -u github.com/bamzi/jobrunner
go get github.com/robfig/cron/v3@v3.0.0
go: cannot use path@version syntax in GOPATH mode
ddavtian commented 4 years ago

@bamzi just for reference I have asked on the cron side of the world as well: https://github.com/robfig/cron/issues/242

ddavtian commented 4 years ago

Seem like I have exhausted most variations of imports and seeing this at the end

Solving failure: No versions of github.com/robfig/cron met constraints:
    v3.0.0: Could not introduce github.com/robfig/cron@v3.0.0, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v1.2.0: Could not introduce github.com/robfig/cron@v1.2.0, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v1.1.0: Could not introduce github.com/robfig/cron@v1.1.0, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v1.0.0: Could not introduce github.com/robfig/cron@v1.0.0, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v3.0.0-rc1: Could not introduce github.com/robfig/cron@v3.0.0-rc1, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    master: Could not introduce github.com/robfig/cron@master, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v1: Could not introduce github.com/robfig/cron@v1, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v2: Could not introduce github.com/robfig/cron@v2, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v3: Could not introduce github.com/robfig/cron@v3, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
    v3-dev: Could not introduce github.com/robfig/cron@v3-dev, as its subpackage github.com/robfig/cron/v3 is missing. (Package is required by github.com/bamzi/jobrunner@master.)
bamzi commented 4 years ago

I reproduced your situation and I can confirm it with my go version 1.13.1 and your experience is similar to this case: https://github.com/golang/go/issues/34184

bamzi commented 4 years ago

I resolved vendoring with the following way: go version: 1.13.1

Env: (.bash_profile or .zshrc)

export GOROOT=/usr/local/go  <--- make sure it the right path first!!!
export PATH=$PATH:$GOROOT/bin
export GOPATH=~/Go  <--- Make sure this is set to your GOPATH
export PATH=$PATH:$GOPATH/bin
export GO111MODULE=auto

Then:

>[project root directory]: go mod init
>[project root directory]: go run ***.go // to test the app still works

>[project root directory]: go get
>[project root directory]: go mod vendor
ddavtian commented 4 years ago

@bamzi thank you for taking the time in trying to figure out this issue. I wish I can say that my end results were as good as yours. I am providing all the steps that I took, at this point it's probably an operator error on my part :-( Apologies for the verbosity, trying to help in narrowing down the issue if it happens to be impacting others as well.

go version

go version go1.13.1 darwin/amd64

go env

GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/davtian/Library/Caches/go-build"
GOENV="/Users/davtian/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/davtian/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/g9/c2njm0x56tj9v40v8_6t2hnc0000gn/T/go-build234405501=/tmp/go-build -gno-record-gcc-switches -fno-common"

Environment Variables

env | grep GO

GOROOT=/usr/local/go
GOPATH=/Users/davtian/go
GO111MODULE=auto

Gopkg.toml

[[constraint]]
  branch = "master"
  name = "github.com/ddavtian/eventbus"

[[constraint]]
  branch = "master"
  name = "github.com/bamzi/jobrunner"

[[constraint]]
  branch = "master"
  name = "github.com/docker/docker"

[[constraint]]
  name = "github.com/enriquebris/goconcurrentqueue"
  version = "0.1.0"

[[constraint]]
  name = "github.com/fsnotify/fsnotify"
  version = "1.4.7"

[[constraint]]
  name = "github.com/labstack/echo"
  version = "3.3.10"

[[constraint]]
  name = "github.com/satori/go.uuid"
  version = "1.2.0"

[[constraint]]
  name = "github.com/sirupsen/logrus"
  version = "1.4.1"

[[constraint]]
  name = "github.com/spf13/viper"
  version = "1.4.0"

[[constraint]]
  branch = "master"
  name = "golang.org/x/net"

[[constraint]]
  name = "gopkg.in/resty.v1"
  version = "1.12.0"

[prune]
  go-tests = true
  unused-packages = true

Code Imports

import (
github.com/bamzi/jobrunner
github.com/robfig/cron/v3
)

Gopkg.lock

Snippets that relate to jobrunner/cron

[[projects]]
  branch = "master"
  digest = "1:bccd5db418e44d6031e9394ff11553d9bdc2f98f773136793e5b8d67b371f835"
  name = "github.com/bamzi/jobrunner"
  packages = ["."]
  pruneopts = "UT"
  revision = "852b7ca4d475a62a2c6f1e4bf6e1d458e2d6ae0f"

[[projects]]
  branch = "v2"
  digest = "1:e8c48aa61ea272e211858e52979ad4d114c60bfe6900d12431ac697482100b0a"
  name = "github.com/robfig/cron/v3"
  packages = ["."]
  pruneopts = "UT"
  revision = "be2e0b0deed5a68ffee390b4583a13aff8321535"

Mod Init

go mod init
go: creating new go.mod: module gandalfd
go: copying requirements from Gopkg.lock
go: converting Gopkg.lock: stat github.com/ddavtian/eventbus@fc9d54b5ebf44dd842ac4dea44adad3fb5f23108: unknown revision fc9d54b5ebf44dd842ac4dea44adad3fb5f23108

Observed Issues below

go get

go: finding gopkg.in/robfig/cron.v3 v3.0.0
go: finding gopkg.in/robfig/cron.v3 v3.0.0
build gandalfd: cannot load gopkg.in/robfig/cron.v3: cannot find module providing package gopkg.in/robfig/cron.v3

Grepping

grep -r "gopkg.in/robfig/cron.v3" .

./vendor/github.com/bamzi/jobrunner/jobrunner.go:   "gopkg.in/robfig/cron.v3"
./vendor/github.com/bamzi/jobrunner/stopjob.go:import "gopkg.in/robfig/cron.v3"
./vendor/github.com/bamzi/jobrunner/status.go:  "gopkg.in/robfig/cron.v3"
./vendor/github.com/bamzi/jobrunner/init.go:    "gopkg.in/robfig/cron.v3"
./vendor/github.com/bamzi/jobrunner/runjob.go:  "gopkg.in/robfig/cron.v3"

go mod vendor

go: finding gopkg.in/robfig/cron.v3 v3.0.0
go: finding gopkg.in/robfig/cron.v3 v3.0.0
gandalfd/pkg/attacks/bandwidth imports
    github.com/bamzi/jobrunner imports
    gopkg.in/robfig/cron.v3: cannot find module providing package gopkg.in/robfig/cron.v3

dep ensure -update -v

Solving failure: No versions of github.com/robfig/cron met constraints:
    v3.0.0: Could not introduce github.com/robfig/cron@v3.0.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1.2.0: Could not introduce github.com/robfig/cron@v1.2.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1.1.0: Could not introduce github.com/robfig/cron@v1.1.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1.0.0: Could not introduce github.com/robfig/cron@v1.0.0, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v3.0.0-rc1: Could not introduce github.com/robfig/cron@v3.0.0-rc1, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    master: Could not introduce github.com/robfig/cron@master, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v1: Could not introduce github.com/robfig/cron@v1, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v2: Could not introduce github.com/robfig/cron@v2, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v3: Could not introduce github.com/robfig/cron@v3, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
    v3-dev: Could not introduce github.com/robfig/cron@v3-dev, as its subpackage github.com/robfig/cron/v3 is missing. Package is required by:
    (root)
    github.com/bamzi/jobrunner@master
ddavtian commented 4 years ago

@bamzi A suggestive fix: https://github.com/robfig/cron/issues/242#issuecomment-539630086 is to remove the `/v3

As indicated in the comment

It might be a show-stopper, but I think you would have to update any library that imports cron to trim the /v3. It looks like jobrunner imports it as cron/v3 which is causing that error message.

bamzi commented 4 years ago

I think the problem is not the import but older version of robfig/cron package in your existing GOPATH

First -> go get -u github.com/robfig/cron so that your base lib is updated to latest and then go get github.com/robfig/cron/v3@v3.0.0 to get the latest tags

since this is more of an environment issue than functional, i'm going to close this issue