Support both cowboy 1.x and 2.x version

[gomoripeti]

Why would we support cowboy 1.x

• first of all because it requires OTP 19+ and uses maps and we would like to use xprof even on R16B
• there might be other projects that use newer Erlang version but their own app still depends on cowboy 1.x

Handling cowboy dependency

• Mix allows depending on multiple version {cowboy, "~> 1.1.2 or ~> 2.0"} then we need to add also a mix.exs to xprof
• rebar3 supports overrides so the app including xprof can set arbitrary cobwoy version We need to update instructions how to do this (and decide which cowboy version should be default in xprof, probably 2.0)
• maybe different rebar3 profiles (how to set profile from user's app?)
• we can create separate branch for legacy cowboy 1.x support we need to keep it up-to-date ( 👎 ) we can tag separate versions and publish them on hex.pm (xprof 2.0+cowboy1)

Compiling

• As a minimum we need a macro whether maps are supported and ommit code snippets (with ifdefs) which include maps This can be done automatically with platform_define in xprof's rebar.config
• The user can define env var/macro XPROF_COWBOY_1 to enforce cowboy 1.x usage otherwise cowboy 2.x will be used (at compile time!) rebar.config.script can adjust deps based on this (as well as unneeded code can be skipped with ifdef)

Runtime

• xprof_gui_app can detect at runtime cowboy version and use the appropriate cowboy API and handler module to start cowboy This is automatic and cool

To sum up there are probably two options

1. Explicit
   • user defines XPROF_COWBOY_1 to enforce version at compile time
   • rebar.config.script + ifdefs in code
2. Implicite/Automatice
   • mix.exs and rebar3 override to adjust dependency
   • automatice macro if maps are supported (+ifdefs in code)
   • xprof_gui_app detects verison at runtime

...and probably a third, simplest one I did not think of.

Sample WIP code for 1. can be seen here: xprof_gui_cowboy_handler.erl, xprof_gui_app.erl

[afronski]

3rd possible options would be split.

Let's call packages as following - xprof_core (matching, throttling, tracing etc.), xprof_api - which is cowboy 2.x wrapper, xprof_legacy_api - which is cowboy 1.x wrapper and xprof_front_end - which is UI package (with support for both REST APIs).

Then we can publish two packages on hex.pm:

• xprof = xprof_core + xprof_api + xprof_front_end - for cobwoy 2.x.
• xprof_legacy = xprof_core + xprof_legacy_api + xprof_front_end - for cowboy 1.x.

Benefits:

• In case of version bumps on both cowboy branches we can fix API independently.
• There is no implicit mechanism, no auto-detection, no compile time mechanism.
• Explicitness is required (TBH: I like that part the most - user needs to think which version should be used and needs to do it deliberately and consciously).

Drawbacks:

• xprof_core and xprof_front_end needs to be compatible with all requirements that are set for xprof_legacy (e.g. support for old Erlang versions - no maps, no fancy stuff added in later VM).
• xprof_front_end possibly will need to support 2 different APIs (e.g. v1 and v2 of REST API, or REST API and websockets API).

[gomoripeti]

good suggestion

• the REST API (and even the websocket API) should look exactly the same regardless of cowboy version so this is not a problem for xprof_front_end
• xprof_core needs to be compatible with old Erlang versions anyway so this is no different in any solution

I am a bit scared of introducing so many hex packages and OTP applications (even for a javascript-only component) but I should get over this :) . Maybe we could generate multiple hex packages (xprof_api/xprof_legacy_api) from the same OTP application and code base. (These are basically 1 module apps)

I wonder if there is any existing example in the hex world for such fine-grained packaging?

[afronski]

@gomoripeti Yup, it is possible e.g. phoenix is organized in such way - https://hex.pm/packages/phoenix - phoenix_pubsub is there by default, also there is phoenix_html, phoenix_ecto in your generated application and so on. I wouldn't name that fine-grained, but it is really close to what we want to achieve here.

[gomoripeti]

just a note that if we have xprof_gui (depending on cowboy 2.0.0) and xprof_legacy_gui (depending on cowboy 1.1.2) in the same repo/umbrella project, rebar3 will only fetch one of the cowboy versions. (Obviously, to avoid version conflict)

===> Skipping cowboy (from {pkg,<<"cowboy">>,<<"2.0.0">>,
                                      <<"A3B680BCC1156C6FBCB398CC56ADC35177037012D7DC28D8F7E7926D6C243561">>}) as an app of the same name has already been fetched

scetching up deps hell: repo1 (compiles with any Erlang version)

apps/xprof_legacy
├─ apps/xprof_core
└─ apps/xprof_legacy_gui (includes js and common rest module)
   └─ cowboy 1.1.2 (hex pkg)

repo2 (Erlang 19+ only):

apps/xprof
├─ xprof_core (hex pkg)
└─ apps/xprof_gui (overrides cowboy version)
   ├─ cowboy 2.0.0 (hex pkg)
   └─ xprof_legacy_gui (hex pkg)

[gomoripeti]

question of naming and what should be the default

naming

1. if the default is cowboy 2.x and OTP 19+ use _legacy suffix for old stuff
2. if the default is cowboy 1.x and old OTP support use _c2 or _ng (or you name it) suffix for new stuff

(which cowboy version and hence which gui variant to use)

• if you have separate gui node: cowboy version does not matter => use old one as it is compatible with any erlang version
• if you include gui in prod node and dont have cowboy 2.x => again, doesn’t matter, use old one
• if you include gui in prod node and have old Erlang version => have to use old one
• only if you include gui in prod node and already have cowboy 2.x => have to use new one

So it seems that cowboy 2.x is a special case, so the default should be the old one. On the other hand it's a good idea to have the new thing as default (in general; forward looking) as later other, new features could come also requiring eg. OTP 19+ (most notably the tracer NIF)

[afronski]

I'd vote for new stuff as a default. I know that will require more work for the older VMs, but that's the deal with supporting and using older stuff - if we're aiming into bigger adoption, the new should be our default.

[ppikula]

I am also for keeping new as default and hopefully dropping legacy stuff after some time.

[gomoripeti]

packaging and scenarios (maybe we don't want to support all of them)

• Elixir/Mix + hex package
  • cowboy 2 👍
  • cowboy 1 (Issue 1.)
• Erlang/rebar3 + hex package
  • cowboy 2 👍
  • cowboy 1 (Issue 2.)
• Erlang/rebar2/rebar3 + git dep (no hex for rebar2) could use raw and rsync providers, the solution "OS env var to choose cowboy version" #124 should work in this case
  • cowboy 2 👍
  • cowboy 1 👍

Issues

1. Mix does not support passing OS env when building rebar dependency before 1.5.3 (included in 1.6.0 - https://github.com/elixir-lang/elixir/pull/6933 )
2. rebar3 does not support multiple versions like "1.1.2 or ~> 2.0.0" (big bummer) a version like {cowboy, "~> 2.0.0 ir 1.1.2"} in rebar.config cause crash like

   ===> Uncaught error: {badmatch,[<<"2.0.0">>,<<"or">>,<<"1.1.2">>]}
   ===> Stack trace to the error location:
   [{rebar_app_utils,update_source,3,
                    [{file,"/Users/peter/git/rebar3/_build/default/lib/rebar/src/rebar_app_utils.erl"},
                     {line,259}]},

   multiple version in hex package dependency causes the dependency to be simply skipped

   ===> [xprof_gui:2.0.0-prealpha.2], Bad dependency version for cowboy: 1.1.2 or ~> 2.0.
   ...
        └─ xprof_gui─2.0.0-prealpha.2 (hex package)
           └─ jsone─1.3.1 (hex package)