:crypto.block_decrypt/4 is undefined in OTP24

[alappe]

With elixir v1.12 on OTP v24.0.1 I have the following issue:

iex(2)> :erlang.system_info(:otp_release)
'24'
iex(3)> jwk = JOSE.JWK.from(%{"k" => "STlqtIOhWJjoVnYjUjxFLZ6oN1oB70QARGSTWQ_5XgM", "kty" => "oct"})                   %JOSE.JWK{
  fields: %{},
  keys: :undefined,
  kty: {:jose_jwk_kty_oct,
   <<73, 57, 106, 180, 131, 161, 88, 152, 232, 86, 118, 35, 82, 60, 69, 45, 158,
     168, 55, 90, 1, 239, 68, 0, 68, 100, 147, 89, 15, 249, 94, 3>>}
}
iex(4)> JOSE.JWE.block_decrypt(jwk, "eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..jBt5tTa1Q0N3uFPEkf30MQ.Ei49MvTLLje7bsZ5EZCZMA.gMWOAmhZSq9ksHCZm6VSoA")
** (UndefinedFunctionError) function :crypto.block_decrypt/4 is undefined or private, use crypto:crypto_one_time/5, crypto:crypto_one_time_aead/6,7 or crypto:crypto_(dyn_iv)?_init + crypto:crypto_(dyn_iv)?_update + crypto:crypto_final instead
    (crypto 5.0.1) :crypto.block_decrypt(:aes_cbc, <<158, 168, 55, 90, 1, 239, 68, 0, 68, 100, 147, 89, 15, 249, 94, 3>>, <<140, 27, 121, 181, 54, 181, 67, 67, 119, 184, 83, 196, 145, 253, 244, 49>>, <<18, 46, 61, 50, 244, 203, 46, 55, 187, 110, 198, 121, 17, 144, 153, 48>>)
    (jose 1.11.1) src/jwe/jose_jwe_enc_aes.erl:167: :jose_jwe_enc_aes.block_decrypt/4
    (jose 1.11.1) src/jwe/jose_jwe.erl:198: :jose_jwe.block_decrypt/2
    (jose 1.11.1) lib/jose/jwe.ex:470: JOSE.JWE.block_decrypt/2
iex(4)>

I deleted deps/ and _build, fetched everything new, tried erlang-jose v1.11.1 and master. Is there something wrong with my setup?

https://github.com/alappe/jose_otp24 shows the issue:

$ elixir -v
Erlang/OTP 23 [erts-11.1.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

Elixir 1.11.2 (compiled with Erlang/OTP 21)
$ mix test                                                                                                                                                                                          13:43
==> jose
Compiling 106 files (.erl)
Compiling 8 files (.ex)
Generated jose app
==> jason
Compiling 8 files (.ex)
Generated jason app
==> jose_otp24
Compiling 1 file (.ex)
Generated jose_otp24 app
..

Finished in 0.04 seconds
1 doctest, 1 test, 0 failures

Randomized with seed 689982
$ 

$ asdf install
elixir 1.12.0 is already installed
erlang 24.0.1 is already installed
nodejs 14.17.0 is already installed

$ elixir -v
Erlang/OTP 24 [erts-12.0.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

Elixir 1.12.0 (compiled with Erlang/OTP 22)
$ mix test
==> jose
Compiling 106 files (.erl)
Compiling 8 files (.ex)
Generated jose app
==> jason
Compiling 8 files (.ex)
Generated jason app
==> jose_otp24
Compiling 1 file (.ex)
Generated jose_otp24 app

  1) test block_decrypt (JoseOtp24Test)
     test/jose_otp24_test.exs:5
     ** (UndefinedFunctionError) function :crypto.block_decrypt/4 is undefined or private, use crypto:crypto_one_time/5, crypto:crypto_one_time_aead/6,7 or crypto:crypto_(dyn_iv)?_init + crypto:crypto_(dyn_iv)?_update + crypto:crypto_final instead
     code: JOSE.JWE.block_decrypt(
     stacktrace:
       (crypto 5.0.1) :crypto.block_decrypt(:aes_cbc, <<158, 168, 55, 90, 1, 239, 68, 0, 68, 100, 147, 89, 15, 249, 94, 3>>, <<140, 27, 121, 181, 54, 181, 67, 67, 119, 184, 83, 196, 145, 253, 244, 49>>, <<18, 46, 61, 50, 244, 203, 46, 55, 187, 110, 198, 121, 17, 144, 153, 48>>)
       (jose 1.11.1) src/jwe/jose_jwe_enc_aes.erl:167: :jose_jwe_enc_aes.block_decrypt/4
       (jose 1.11.1) src/jwe/jose_jwe.erl:198: :jose_jwe.block_decrypt/2
       (jose 1.11.1) lib/jose/jwe.ex:470: JOSE.JWE.block_decrypt/2
       test/jose_otp24_test.exs:9: (test)

.

Finished in 0.04 seconds (0.00s async, 0.04s sync)
1 doctest, 1 test, 1 failure

Randomized with seed 783367
$

Probably unrelated, but I cannot compile the project to run the tests, mix test fails with

…
 C      libsodium_api.c
In file included from /project/deps/jose/deps/libsodium/c_src/libsodium_api.c:4:
In file included from /project/deps/jose/deps/libsodium/c_src/libsodium_api.h:7:
/project/deps/jose/deps/libsodium/c_src/libsodium_drv_common.h:9:10: fatal error: 'erl_interface.h' file not found
#include <erl_interface.h>
         ^~~~~~~~~~~~~~~~~
1 error generated.
make: *** [/project/deps/jose/deps/libsodium/c_src/libsodium_api.o] Error 1
===> Hook for compile failed!

** (Mix) Could not compile dependency :libsodium, "/Users/nd/.asdf/installs/elixir/1.12.0/.mix/rebar3 bare compile --paths /project/deps/jose/_build/test/lib/*/ebin" command failed. You can recompile this dependency with "mix deps.compile libsodium", update it with "mix deps.update libsodium" or clean it with "mix deps.clean libsodium"

[ericmj]

It looks like a few places were missed during the OTP 24 compatibility changes. Any contributions that fix up the last places would be appreciated. I have added OTP 24 to CI to make it easier to test the fixes.

Probably unrelated, but I cannot compile the project to run the tests, mix test fails with

I would recommend running the tests in docker. You can see examples in the CI.

[alappe]

Ok, I can try to look into it. Thanks for the feedback and the ci changes.

[alappe]

After some hours I had one line in to fix my error, but the test suite still has 37 fails. I conclude I have way to little knowledge on the topic to mess around in this code. :-(

[michaelklishin]

We need to get https://github.com/potatosalad/erlang-libsodium/pull/9 in in order for mix test to function on Erlang 24.

I no longer see crypto:block_decrypt/{3,4} used outside of of jose_crypto_compat. #113 addressed an edge case with compilation on Erlang 23 and running on 24 (many open source projects support both versions and distribute binary builds).

I'd be happy to take a look at any remaining issues but without a running test suite it's a bit difficult to assess what may be missing.

[alappe]

As Eric said, running the suite in docker worked fine for me.

e.g. in src/jwa/jose_jwa.erl, block_decrypt/3 is using Module:block_decrypt(BlockCipher, Key, CipherText) as a fallback which is using crypto:block_decrypt/3 then. I tried to add more pattern matching there, but I'm way too clueless about the algorithm combinations to finish this.

[pjk25]

I've made some progress with #118 (which is still a draft), dropping the number of failures from 37 to 4. I hope to have the remaining 4 solved soon enough.

[pjk25]

The remaining failures stem from jose_public_key:der_decode(AlicePrivateDER). behaving differently based on the erlang version, for the same value of AlicePrivateDER (<<48,46,2,1,0,48,5,6,3,43,101,112,4,34,4,32,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>).

The failures in fact match the added support for ed25519 & ed448 in Erlang 24. It appears to me that erlang-jose provided it's own implementation of these algorithms, so my suspicion currently is that built in decoding now short circuits a fallback.

[pjk25]

118 is now ready to go

[alappe]

Thank you! 🎉