search

OvermindDL1 / protocol_ex

Elixir Extended Protocol
https://hex.pm/packages/protocol_ex
49 stars 5 forks source link

Failing ProtocolEx consolidation during releases #59

Closed marcelotto closed 3 years ago

marcelotto commented 3 years ago

The RDF.ex library uses ProtocolEx for its extensible datatype system. In an app which uses RDF.ex, we're getting the following error during releases:

** (MatchError) no match of right hand side value: [
  "protocolex_releases_test/_build/prod/lib/rdf/ebin/Elixir.RDF.Literal.Datatype.Registry.Registration.beam", 
  "protocolex_releases_test/_build/prod/rel/protocolex_releases_test/lib/rdf-0.9.3/ebin/Elixir.RDF.Literal.Datatype.Registry.Registration.beam"]
    (protocol_ex 0.4.3) lib/protocol_ex.ex:426: ProtocolEx.consolidate/2
    (elixir 1.11.1) lib/enum.ex:1399: Enum."-map/2-lists^map/1-0-"/2
    (protocol_ex 0.4.3) lib/mix/tasks/compile.protocol_ex.ex:25: Mix.Tasks.Compile.ProtocolEx.run/1
    (mix 1.11.1) lib/mix/task.ex:394: Mix.Task.run_task/3
    (mix 1.11.1) lib/mix/tasks/compile.all.ex:90: Mix.Tasks.Compile.All.run_compiler/2
    (mix 1.11.1) lib/mix/tasks/compile.all.ex:70: Mix.Tasks.Compile.All.compile/4
    (mix 1.11.1) lib/mix/tasks/compile.all.ex:57: Mix.Tasks.Compile.All.with_logger_app/2
    (mix 1.11.1) lib/mix/tasks/compile.all.ex:35: Mix.Tasks.Compile.All.run/1

In the line causing this error, you're looking for the beam file of a module in the build path and expect it to return exactly one path:

[path] = Path.wildcard(glob) # TODO:  Add error reporting on this...

However, as you can see in the error message (showing two beam files for the same module), during releases it is normal to have the beam files in the build path in multiple places so that this pattern match will fail.

On a side note, I'd like to ask another question. When I first saw this error, I remembered your comment here and thought the missing compiler in our app using RDF.ex might be the cause.

[...] users might need to add compilers: Mix.compilers ++ [:protocol_ex], to their mix.exs file too.

— https://elixirforum.com/t/panpipe-an-elixir-wrapper-around-pandoc-and-its-markdown-ast/24774/4

As it turned out this was not the case. However, I'd like to get a confirmation if my assumption about this requirement is correct. Generally, the users of RDF.ex don't have to add the protocol_ex compiler, only when they have ProtocolEx-implementations of the respective ProtocolEx protocols in their app, right?

Steps to reproduce the problem

  1. mix new protocolex_releases_test
  2. Add {:rdf, "0.9.3"} as a dependency
  3. mix deps.get
  4. MIX_ENV=prod mix release
  5. Upgrade to {:rdf, "0.9.4"}
  6. mix deps.get
  7. MIX_ENV=prod mix release
OvermindDL1 commented 3 years ago

However, as you can see in the error message (showing two beam files for the same module), during releases it is normal to have the beam files in the build path in multiple places so that this pattern match will fail.

Yeah this isn't how releases used to be done, this is from the new elixir release system and ProtocolEx predates that... ^.^;

Basically I'll need to just have it ignore the rel directory... I think... Theoretically they are the same files though so I could just grab either... EDIT: Apparently they aren't the same file, elixir releases are working different than distillary releases, hmm...

As it turned out this was not the case. However, I'd like to get a confirmation if my assumption about this requirement is correct. Generally, the users of RDF.ex don't have to add the protocol_ex compiler, only when they have ProtocolEx-implementations of the respective ProtocolEx protocols in their app, right?

Eh, it's needed in the compiler list if it needs to compile the protocol. If the output beam files are included in the library directly and there is no user implementation that will need it to recompile then it's not needed, but that's not elixir's standard compiling method. You can always just output the generated source as a new source file and use it straight, you'd only need to call the compiler yourself if you needed to update that source file. I should probably add a way to write out source files directly for such out-of-band updating, I can see definite uses for that especially for libraries that use it but don't need to support external implementations...