rabbitmq / rabbitmq-server

Open source RabbitMQ: core server and tier 1 (built-in) plugins
https://www.rabbitmq.com/
Other
12.27k stars 3.91k forks source link

Use SemVer for Hex dependencies #2641

Open ericmj opened 5 years ago

ericmj commented 5 years ago

Hex requires pacakges to use semantic versioning. The packages amqp_client and rabbit_common uses strict version requirements on its dependencies which causes conflicts and ecosystem problems for users of those packages. Instead they should use the ~> version requirement operator that follows the rules of SemVer. ~> 1.0 will depend on any stable version >= 1.0.0 and < 2.0.0.

Current dependencies

amqp_client:
rabbit_common == 3.7.13

rabbit_common:
jsx == 2.9.0
lager == 3.6.5
ranch == 1.7.1
recon == 2.3.6

How dependencies should be declared

amqp_client:
rabbit_common ~> 3.7

rabbit_common:
jsx ~> 2.9
lager ~> 3.6
ranch ~> 1.7
recon ~> 2.3
michaelklishin commented 5 years ago

This client very likely will continue requiring an identical rabbit_common version. rabbit_common's dependencies perhaps can be relaxed. @dumbbell would there be any potential ramifications for the pipelines?

dumbbell commented 5 years ago

RabbitMQ does not use semantic versioning for its own components currently. As @michaelklishin said, the rabbit_common version pinning in amqp_client is strict for this reason.

About the pinning of third-party dependencies, we also want strict pinning for the following reasons:

dumbbell commented 5 years ago

Note that Erlang.mk, the build tool we use, does not have a "lock" mechanism like other build tools, so we can't rely on such a thing.

ericmj commented 5 years ago

About the pinning of third-party dependencies, we also want strict pinning for the following reasons:

This should not be be achieved by version pinning your package dependencies - what you want to do is pin your project dependencies. Rebar3 and mix support this out of the box with lock files.

If you use another build tool that does not support lock files I suggest changing the package metadata before publishing or listing project dependencies and package dependencies separately. I notice that you have built a custom tool for publishing to hex, maybe this can be added to that tool?

michaelklishin commented 5 years ago

What we need here is to relax rabbit_common dependencies in Hex metadata. @dumbbell can we add a Mix file to it just for dependency management at hex.pm release time? Pinning to a certain minor version of a dependency should work reasonably well in practice. Application developers can ping to a specific patch if needed.

dumbbell commented 5 years ago

If you use another build tool that does not support lock files I suggest changing the package metadata before publishing or listing project dependencies and package dependencies separately. I notice that you have built a custom tool for publishing to hex, maybe this can be added to that tool?

That's a good idea. Unfortunately, what we publish to Hex.pm still uses Erlang.mk as the build tool and it only accepts exact version pinning (it doesn't support the -> x.y syntax and logic).

Can we add a Mix file to it just for dependency management at hex.pm release time?

Mix is not the build time, even after rabbit_common is published to Hex.pm, so we are still limited by what Erlang.mk provides.

ericmj commented 5 years ago

That's a good idea. Unfortunately, what we publish to Hex.pm still uses Erlang.mk as the build tool and it only accepts exact version pinning (it doesn't support the -> x.y syntax and logic).

There's a difference what the package metadata is and what the package contents are. You can have ~> in the metadata and an exacter version in the Makefile config.

Mix is not the build time, even after rabbit_common is published to Hex.pm, so we are still limited by what Erlang.mk provides.

It seems like all the packages you publish support building with both Erlang.mk and rebar3, at least that's what is in the package metadata.

michaelklishin commented 5 years ago

@ericmj we understand what you are proposing. It's mostly a matter of finding a way to do that that won't be too fragile and/or require substantial changes to our build system or pipeline. I think it's doable, we'll investigate soon. Thank you for the suggestion.

ericmj commented 5 years ago

Thank you for looking into it!

dumbbell commented 5 years ago

It seems like all the packages you publish support building with both Erlang.mk and rebar3, at least that's what is in the package metadata.

Hmm, you're right, I was looking at the list of files in the output of rebar3 hex publish and didn't spot rebar.config. I assumed we didn't support rebar3 and forgot the Rebar files are not listed explicitely.

So yes, we should be able to patch the generated rebar.config to use relaxed pinning.