kubernetes / dns

Kubernetes DNS service
Apache License 2.0
930 stars 467 forks source link

Errors you may encounter when upgrading the library #385

Closed KateGo520 closed 3 years ago

KateGo520 commented 4 years ago

(The purpose of this report is to alert kubernetes/dns to the possible problems when kubernetes/dns try to upgrade the following dependencies)

An error will happen when upgrading library _prometheus/clientgolang:

github.com/prometheus/client_golang

-Latest Version: v1.7.1 (Latest commit b0cdec2 3 days ago) -Where did you use it: https://github.com/kubernetes/dns/search?q=prometheus%2Fclient_golang%2Fprometheus&unscoped_q=prometheus%2Fclient_golang%2Fprometheus -Detail:

github.com/prometheus/client_golang/go.mod


module github.com/prometheus/client_golang
require (
github.com/beorn7/perks v1.0.1
github.com/cespare/xxhash/v2 v2.1.1
…
)
go 1.11
>[github.com/prometheus/client_golang/prometheus/desc.go]( https://github.com/prometheus/client_golang/blob/v1.7.1/prometheus/desc.go#L22 )

package prometheus import ( "github.com/cespare/xxhash/v2" … )


This problem was introduced since **_prometheus /client_golang v1.2.0(committed 9a2ab94 on 16 Oct 2019)_** .Now you used version [v1.1.0](https://github.com/kubernetes/dns/blob/master/Gopkg.toml#L50). If you try to upgrade  **prometheus/client_golang** to version **v1.2.0** and **above**, you will get an error--- **_no package exists at " github.com/cespare/xxhash/v2 "_**

## I investigated the libraries  (**prometheus/client_golang >= v1.2.0.**)  release information and found the root cause of this issue is that---- 
1.  These dependencies all added [Go modules](https://github.com/golang/go/wiki/Modules) in the recent versions.

2.  They all comply with the specification of "Releasing Modules for v2 or higher" available in the Modules documentation. Quoting the specification:
> A package that has migrated to Go Modules _must_ include the major version in the import path to reference any v2+ modules. For example,  Repo github.com/my/module migrated to Modules on version v3.x.y.  Then this repo should declare its module path with MAJOR version suffix "/v3"  (e.g., module `github.com/my/module/v3`), and its downstream project should use `"github.com/my/module/v3/mypkg"` to import this repo’s package.

3.  This "github.com/my/module/v3/mypkg" is not the `physical path`. So earlier versions of Go (including those that don't have minimal module awareness) plus all tooling (like dep, glide, govendor, etc) don't have `minimal module awareness` as of now and therefore don't handle import paths correctly See [golang/dep#1962](https://github.com/golang/dep/issues/1962), [golang/dep#2139](https://github.com/golang/dep/issues/2139).
> Note: creating a new branch is not required. If instead you have been previously releasing on master and would prefer to tag v3.0.0 on master, that is a viable option. (However, be aware that introducing an incompatible API change in master can cause issues for non-modules users who issue a go get -u given the go tool is not aware of semver prior to Go 1.11 or when module mode is not enabled in Go 1.11+).
> Pre-existing dependency management solutions such as dep currently can have problems consuming a v2+ module created in this way. See for example dep#1962.
https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

## Solution
### 1. Migrate to Go Modules.
Go Modules is the general trend of ecosystem, if you want a better upgrade package experience, migrating to Go Modules is a good choice.

Migrate to modules will be accompanied by the introduction of virtual paths(It was discussed above).
> This "github.com/my/module/v3/mypkg" is not the `physical path`. So Go versions older than 1.9.7 and 1.10.3 plus all third-party dependency management tools (like dep, glide, govendor, etc) don't have `minimal module awareness` as of now and therefore don't handle import paths correctly.

Then the downstream projects might be negatively affected in their building if they are module-unaware (Go versions older than 1.9.7 and 1.10.3; Or use third-party dependency management tools, such as: Dep, glide, govendor…).

### 2. Maintaining v2+ libraries that use Go Modules in Vendor directories. 
If `kubernetes/dns` want to keep using the dependency manage tools (like dep, glide, govendor, etc), and still want to upgrade the dependencies, can choose this fix strategy.
Manually download the dependencies into the vendor directory and do compatibility dispose(materialize the virtual path or delete the virtual part of the path). Avoid fetching the dependencies by virtual import paths. This may add some maintenance overhead compared to using modules.

As the import paths have different meanings between the projects adopting module repos and the non-module repos, materialize the virtual path is a better way to solve the issue, while ensuring compatibility with downstream module users. A textbook example provided by repo `github.com/moby/moby` is here:
https://github.com/moby/moby/blob/master/VENDORING.md
https://github.com/moby/moby/blob/master/vendor.conf
In the vendor directory, `github.com/moby/moby` adds the **/vN** subdirectory in the corresponding dependencies.
**This will help more downstream module users to work well with your package.**

### 3. Request upstream to do compatibility processing.
The `prometheus/client_golang` have 1039 module-unaware users in github, such as: AndreaGreco/mqtt_sensor_exporter, seekplum/plum_exporter, arl/monitoring…
https://github.com/search?q=prometheus%2Fclient_golang+filename%3Avendor.conf+filename%3Avendor.json+filename%3Aglide.toml+filename%3AGodep.toml+filename%3AGodep.json

## Summary
You can make a choice when you meet this DM issues by balancing your own development schedules/mode against the affects on the downstream projects.

For this issue, Solution 1 can maximize your benefits and with minimal impacts to your downstream projects the ecosystem. 

## References
* https://github.com/golang/go/wiki/Modules#semantic-import-versioning
* https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning
* https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Do you plan to upgrade the libraries in near future?
Hope this issue report can help you ^_^
Thank you very much for your attention.

Best regards,
Kate
KateGo520 commented 4 years ago

@prameshj @pnovotnak Could you help me review this issue? Thx :p By the way, have you ever encountered such errors when upgading the libraries?

prameshj commented 4 years ago

Could you clarify what the issue you are seeing is? Are you trying to upgrade the prometheus client version in this repo?

KateGo520 commented 4 years ago

@prameshj Thanks for your reply. This report is a warm prompt for you to prevent or combat this issue. Just let you know the problem if you upgrade this dependency in near future.

Hope this can help you. Thanks again. Kate

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 3 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes/dns/issues/385#issuecomment-762834073): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
fejta-bot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close