curve25519-dalek zeroize dependency not compatible with other libraries

[samuelvanderwaal]

Problem

Solana v1.8.x relied on curve25519-dalek ^v2.1.0 which itself has a zeroize dependency of ^1.

However, Solana v1.9+ rely on curve25519-dalek ^v3.2.1 which limits zeroize to a range of >=1, <1.4 which is not compatible with many other cryptography libraries (or libraries which depend on cryptography libraries) which use ^1.4 or ^1.5. E.g.s:

• aws-types
• num-big-int-dig (which RSA relies on)
• jsonwebkey

It's unclear why curve25519-dalek hasn't bumped yet but there is an open PR to do so. Most other cryptography libraries seem to have bumped to v1.4+

Are there any known workarounds for this? If curve25519-dalek doesn't bump, is it possible for Solana to drop back down to ^v2.1?

Proposed Solution

[anthontaylor]

I'm currently running into the same issue with dependencies being incompatible with the outdated zeroize on curve25519-dalek.

[whymidnight]

this was blocking for me when i tried using RustCrypto/RSA

however, at least for rsa, i found sfackler/rust-openssl that binds to openssl OS package.

this seems to be a viable alternative in the meantime.

[whymidnight]

can confirm that i was able to implement RSA key generation and decryption using sfackler/rust-openssl without dependency issues.

[samuelvanderwaal]

can confirm that i was able to implement RSA key generation and decryption using sfackler/rust-openssl without dependency issues.

Thanks for sharing this!

[NorbertBodziony]

Any update on this ? Majority of packages already bumped zeroize.

[mschneider]

curve25519-dalek 3.2.1 seems to be a blocker here, a 4.0.0 release is imminent (rc.2 was published on March 26)

[juchiast]

4.0.0 is released

[hlgltvnnk]

Upgrading curve25519-dalek to version 4 will not solve the problem Because there is also ed25519-dalek-bip32 (v0.2.0), which depends on ed25519-dalek (v1.0.1), which also depends on curve25519-dalek (version = "3") There is an ed25519-dalek v2.0.0-rc.3 that uses curve25519-dalek "=4.0.0-rc.3", but there is no stable version yet, so ed25519-dalek-bip32 is still on ed25519-dalek 1.0.1 So the way out of this situation may be:

1. Wait for the release of the stable version of ed25519-dalek -> and then update ed25519-dalek-bip32 -> and then update deps in solana core
2. Replace the ed25519-dalek-bip32 crate - as far as I understand it is used only for generating a keypair using Bip32 Hierarchical Derivation

[juchiast]

2. Replace the ed25519-dalek-bip32 crate - as far as I understand it is used only for generating a keypair using Bip32 Hierarchical Derivation

solana also uses ed25519-dalek 1.01, so I don't think this is an option.

https://github.com/solana-labs/solana/blob/3dcb3827314517eeec6018cbe28e1ae535437f76/Cargo.toml#L182

[hlgltvnnk]

2. Replace the ed25519-dalek-bip32 crate - as far as I understand it is used only for generating a keypair using Bip32 Hierarchical Derivation

solana also uses ed25519-dalek 1.01, so I don't think this is an option.

https://github.com/solana-labs/solana/blob/3dcb3827314517eeec6018cbe28e1ae535437f76/Cargo.toml#L182

This is a temporary workaround (if you want solana to be compatible with other crates that use zeroize) to use the raw version of ed25519-dalek (Im using "2.0.0-rc.3" in my fork) and then replace the ed25519-dalek-bip32 (because I don't want to fork it as well). This is not stable and should only be used if it blocks development (for example, I need to use the async solana cli in my project, which is implemented in the latest versions of solana, so incompatibility with other packages is a big issue for me). The best option would be to wait until ed25519-dalek releases v2 and then 1) wait for ed25519-dalek-bip32 to update or 2) replace ed25519-dalek-bip32 (spl also needs to be updated)

[hlgltvnnk]

ed25519-dalek v2.0.0 is released!

[jbuckmccready]

There is another dependency that needs to be updated to eliminate the awkward zeroize <1.4 dependency problem: aes-gcm-siv needs to be updated to v0.11.1.

[mschneider]

can someone post a public example project for reproduction? hbu @samuelvanderwaal @whymidnight @hlgltvnnk @jbuckmccready?

[tomlinton]

https://github.com/tomlinton/solana-ed25519-conflict

[jbuckmccready]

@mschneider https://github.com/jbuckmccready/solana_zeroize_dep_example

[klefevre]

hey @mschneider, thank you for tackling this issue. Do you have any idea when this is going to be fixed? Currently we can't support Solana because of this issue. In our case, the conflict occurs with the last version of sqlx (0.7.2 at this time).

You can find a demo of the issue here (which is basically the same as the two above): https://github.com/klefevre/solana_zeroize_dep_example/

[dbcfd]

hey @mschneider, thank you for tackling this issue. Do you have any idea when this is going to be fixed? Currently we can't support Solana because of this issue. In our case, the conflict occurs with the last version of sqlx (0.7.2 at this time).

You can find a demo of the issue here (which is basically the same as the two above): https://github.com/klefevre/solana_zeroize_dep_example/

Here's a branch with a fix for just solana-program https://github.com/dbcfd/solana/tree/fix/deps. Still have to work through the other parts of solana.

[ronanyeah]

I am also unable to use several sqlx features because of the zeroize version. Bumping aes-gcm-siv would be appreciated!

[mina86]

In the meanwhile, here’s a workaround:

[patch.crates-io]
# aes-gcm-siv 0.10.3 and curve25519-dalek 3.x pin zeroize to <1.4
# which conflicts with other dependencies requiring zeroize ^1.5.
# We’re patching both crates to unpin zeroize.
#
# For aes-gcm-siv we’re using the same revision Solana uses in
# an (as of now) unreleased commit, see
# https://github.com/solana-labs/solana/commit/01f1bf27994d9813fadfcd134befd3a449aaa0bd
#
# For curve25519-dalek we’re using commit from a PR, see
# https://github.com/dalek-cryptography/curve25519-dalek/pull/606
aes-gcm-siv = { git = "https://github.com/RustCrypto/AEADs", rev = "6105d7a5591aefa646a95d12b5e8d3f55a9214ef" }
curve25519-dalek = { git = "https://github.com/dalek-cryptography/curve25519-dalek", rev = "8274d5cbb6fc3f38cdc742b4798173895cd2a290" }

The second entry points at a commit in a PR. I’m not sure how GitHub treats those and if it’s going to remain available. If not, change repository to "https://github.com/mina86/curve25519-dalek/".

The aes-gcm-siv patch should be unnecessary as soon as new solana crates release happens.

[ronanyeah]

@mina86 Works for me, thank you.

[juchiast]

If anyone got subtle dep conflict:

aes-gcm-siv = { git = "https://github.com/RustCrypto/AEADs", rev = "555ae1d82d000f01899498f969e6dc1d0a4fe467" }

[serejke]

It is a blocker to import ethers-rs and solana-sdk in the same binary.

All workarounds above are fragile.

error: failed to select a version for `zeroize`.
    ... required by package `curve25519-dalek v3.2.1`
    ... which satisfies dependency `curve25519-dalek = "^3.2.1"` of package `solana-program v1.17.11`
    ... which satisfies dependency `solana-program = "=1.17.11"` of package `solana-sdk v1.17.11`
    ... which satisfies dependency `solana-sdk = "^1.17.11"` of package `solana v0.1.0 (~/myproject/solana)`
versions that meet the requirements `>=1, <1.4` are: 1.3.0, 1.2.0, 1.1.1, 1.1.0, 1.0.0
all possible versions conflict with previously selected packages.
  previously selected package `zeroize v1.5.3`
    ... which satisfies dependency `zeroize = "^1.5"` of package `elliptic-curve v0.13.5`
    ... which satisfies dependency `elliptic-curve = "^0.13.5"` of package `ethers-core v2.0.11`
    ... which satisfies dependency `ethers-core = "^2.0.11"` of package `ethers v2.0.11`
    ... which satisfies dependency `ethers = "^2.0.11"` of package `ethereum v0.1.0 (~/myproject/ethereum)`
    ... which satisfies path dependency `ethereum` (locked to 0.1.0) of package `main v0.1.0 (~/myproject/main)`
failed to select a version for `zeroize` which could resolve this conflict

[dougEfresh]

@mschneider May I assist with solving this issue ?

[mschneider]

from my pov resolution of this issue is still blocked by a zeroize upgrade. if you have a patch that resolves the issues feel free to send a PR, but I can’t guarantee it’ll get merged quickly because there’s a couple of downstream projects that need to be upgraded as well. not gonna lie the way rust crypto 4 has been rolled out is very painful. the current workaround is to prevent conflicts by pinning the old version and only using packages compatible with it.

[appetrosyan]

@mschneider I have done a similar upgrade before for hyperledger ursa.

@dougEfresh if you have any intermediate work on this, let me know, otherwise I'm opening a draft PR with the patches.

[appetrosyan]

OK. So at present the patch uis using a Solana vendored curve25519-dalek that removed the explicit pin on zeroize. However, the upgrade to the curve25519-dalek of version 4.0.0 wasn't made, and I think I should just do it.

[35359595]

Not quite sure why my PR with this patch to this repo was autoclosed. I've raised another one into agave. LMK.

[iamnotstatic]

I'm still experiencing the same issue while using solana-client,solana-sdk and ethers-rs @mschneider

Cargo.toml

Error

[mattxyzeth]

solana crates are incompatible with libp2p due to this issue.

[IntelC-H]

How can I fix it?

[iamnotstatic]

In the meanwhile, here’s a workaround:

[patch.crates-io]
# aes-gcm-siv 0.10.3 and curve25519-dalek 3.x pin zeroize to <1.4
# which conflicts with other dependencies requiring zeroize ^1.5.
# We’re patching both crates to unpin zeroize.
#
# For aes-gcm-siv we’re using the same revision Solana uses in
# an (as of now) unreleased commit, see
# https://github.com/solana-labs/solana/commit/01f1bf27994d9813fadfcd134befd3a449aaa0bd
#
# For curve25519-dalek we’re using commit from a PR, see
# https://github.com/dalek-cryptography/curve25519-dalek/pull/606
aes-gcm-siv = { git = "https://github.com/RustCrypto/AEADs", rev = "6105d7a5591aefa646a95d12b5e8d3f55a9214ef" }
curve25519-dalek = { git = "https://github.com/dalek-cryptography/curve25519-dalek", rev = "8274d5cbb6fc3f38cdc742b4798173895cd2a290" }

The second entry points at a commit in a PR. I’m not sure how GitHub treats those and if it’s going to remain available. If not, change repository to "https://github.com/mina86/curve25519-dalek/".

The aes-gcm-siv patch should be unnecessary as soon as new solana crates release happens.

@IntelCore0607 Here this worked for me

[tarcieri]

This PR should fix it: https://github.com/anza-xyz/agave/pull/513

[IntelC-H]

Thank you, @iamnotstatic What are these? What's it for?

Thank you in advance

[iamnotstatic]

Thank you, @iamnotstatic What are these? What's it for?

Thank you in advance

These are patches that are compatible with the crates

[IntelC-H]

How can I use it? Could you send me the sample code?

[iamnotstatic]

How can I use it? Could you send me the sample code?

[patch.crates-io]
# aes-gcm-siv 0.10.3 and curve25519-dalek 3.x pin zeroize to <1.4
# which conflicts with other dependencies requiring zeroize ^1.5.
# We’re patching both crates to unpin zeroize.
#
# For aes-gcm-siv we’re using the same revision Solana uses in
# an (as of now) unreleased commit, see
# https://github.com/solana-labs/solana/commit/01f1bf27994d9813fadfcd134befd3a449aaa0bd
#
# For curve25519-dalek we’re using commit from a PR, see
# https://github.com/dalek-cryptography/curve25519-dalek/pull/606
aes-gcm-siv = { git = "https://github.com/RustCrypto/AEADs", rev = "6105d7a5591aefa646a95d12b5e8d3f55a9214ef" }
curve25519-dalek = { git = "https://github.com/dalek-cryptography/curve25519-dalek", rev = "8274d5cbb6fc3f38cdc742b4798173895cd2a290" }

[dependencies]
ethers = { version = "2.0", features = ["ws"] }
tokio = { version = "1.37.0", features = ["full"] }
eyre = "0.6.12"
serde = "1.0.197"
serde_json = "1.0"
futures = "0.3"
dotenv = "0.15.0"
teloxide = "0.12.2"
solana-client = "1.18.9"
solana-sdk = "1.18.9"
solana-program = "1.18.9"
solana-transaction-status = "1.18.9"
mpl-token-metadata = "4.1.2"
regex = "1.10.4"
num-format = "0.4.4" 

[ItsFunny]

I'm still experiencing the same issue while using solana-client,solana-sdk and ethers-rs @mschneider

Cargo.toml

Error

Have you solved this?

[IntelC-H]

No

[uandere]

Still not working for combination rustls-acme and solana-sdk:

this .toml:

[dependencies]
axum = { version = "0.7.5", features = ["default", "macros"] }
clap = { version = "4.5.4", features = ["derive"] }
moka = { version = "0.12.5", features = ["future"] }
reqwest = { version = "0.12.3", features = ["json"] }
serde = { version = "1.0.197", features = ["derive"] }
serde_json = "1.0.115"
tokio = { version = "1.37.0", features = ["full", "rt-multi-thread"] }
regex = "1.10.4"
lazy_static = "1.4.0"
http = "1.1.0"
anyhow = { version = "1.0.82", features = ["backtrace"] }
solana-sdk = { version = "1.18.16" }
solana-client = { version = "1.18.16" }
bincode = { version = "1.3.3" }
thiserror = { version = "1.0.61" }
# TODO: This shall probably be moved directly to the project files for safety reasons
jupiter-swap-api-client = { git = "https://github.com/jup-ag/jupiter-api-rust-example.git", branch = "main" }
hyper = { version = "1.3.1", features = ["full"] }
base64 = "0.22.1"
chatgpt_rs = { version = "1.2.3" }
tower-http = { version = "0.5.2", features = ["cors"] }
serde_with = { version = "3.8.1", features = ["base64"] }
uuid = { version = "1.8.0", features = ["v4", "fast-rng", "macro-diagnostics", "serde"] }
axum-server = { version = "0.6.0" }

## ATTENTION: RUSTLS & SOLANA ARE IN A CONFLICT: `zeroize` doesn't have compatible versions
rustls-acme = { version = "0.10.1", features = ["axum", "tokio"] }

...produces this error:

error: failed to select a version for `zeroize`.
    ... required by package `curve25519-dalek v3.2.1`
    ... which satisfies dependency `curve25519-dalek = "^3.2.1"` of package `solana-program v1.18.16`
    ... which satisfies dependency `solana-program = "=1.18.16"` (locked to 1.18.16) of package `solana-sdk v1.18.16`
    ... which satisfies dependency `solana-sdk = "^1.18.16"` (locked to 1.18.16) of package `jupiter_swap_bot v0.1.0 (/Users/nazar/Projects/Work/jupiter_swap_bot)`
versions that meet the requirements `>=1, <1.4` are: 1.3.0, 1.2.0, 1.1.1, 1.1.0, 1.0.0

all possible versions conflict with previously selected packages.

  previously selected package `zeroize v1.8.1`
    ... which satisfies dependency `zeroize = "^1.7"` (locked to 1.8.1) of package `aws-lc-rs v1.7.3`
    ... which satisfies dependency `aws-lc-rs = "^1.5.2"` (locked to 1.7.3) of package `rustls-acme v0.10.1`
    ... which satisfies dependency `rustls-acme = "^0.10.1"` (locked to 0.10.1) of package `jupiter_swap_bot v0.1.0 (/Users/nazar/Projects/Work/jupiter_swap_bot)`

failed to select a version for `zeroize` which could resolve this conflict

Any suggestions on that, please?

@IntelCore0607 @iamnotstatic @mina86

P.S Patching aes-gcm-siv and curve25519-dalek doesn't work. The error is:

error: failed to select a version for `subtle`.
    ... required by package `aes-gcm-siv v0.10.3 (https://github.com/RustCrypto/AEADs?rev=6105d7a5591aefa646a95d12b5e8d3f55a9214ef#6105d7a5)`
    ... which satisfies dependency `aes-gcm-siv = "^0.10.3"` of package `solana-zk-token-sdk v1.18.16`
    ... which satisfies dependency `solana-zk-token-sdk = "^1.17.6"` (locked to 1.18.16) of package `spl-token-2022 v1.0.0`
    ... which satisfies dependency `spl-token-2022 = "=1.0.0"` (locked to 1.0.0) of package `solana-rpc-client-api v1.18.16`
    ... which satisfies dependency `solana-rpc-client-api = "=1.18.16"` (locked to 1.18.16) of package `solana-client v1.18.16`
    ... which satisfies dependency `solana-client = "^1.18.16"` (locked to 1.18.16) of package `jupiter_swap_bot v0.1.0 (/Users/nazar/Projects/Work/jupiter_swap_bot)`
versions that meet the requirements `>=2, <2.5` are: 2.4.1, 2.4.0, 2.3.0, 2.2.3, 2.2.2, 2.2.1, 2.1.1, 2.1.0, 2.0.0

all possible versions conflict with previously selected packages.

  previously selected package `subtle v2.5.0`
    ... which satisfies dependency `subtle = "^2.4"` (locked to 2.5.0) of package `digest v0.10.7`
    ... which satisfies dependency `digest = "^0.10.7"` (locked to 0.10.7) of package `solana-sdk v1.18.16`
    ... which satisfies dependency `solana-sdk = "^1.18.16"` (locked to 1.18.16) of package `jupiter_swap_bot v0.1.0 (/Users/nazar/Projects/Work/jupiter_swap_bot)`

failed to select a version for `subtle` which could resolve this conflict

[ara-selini]

What's the recommended solution now? This is a pretty big pain point for us.

[samuelvanderwaal]

What's the recommended solution now? This is a pretty big pain point for us.

It depends on what exactly the conflict is for you. You might be able to replace the external crypto lib with something that doesn't use curve25519-dalek such as they do here: https://github.com/solana-labs/solana/issues/26688#issuecomment-1273133391, or you might be able to patch in a new version of curve25519-dalek to fix it. I haven't seen a standard suggestion from Solana yet.

[AraMambreyan]

What's the recommended solution now? This is a pretty big pain point for us.

It depends on what exactly the conflict is for you. You might be able to replace the external crypto lib with something that doesn't use curve25519-dalek such as they do here: #26688 (comment), or you might be able to patch in a new version of curve25519-dalek to fix it. I haven't seen a standard suggestion from Solana yet.

Our dependency conflict is with ethers-rs which will be very difficult to fix as it is deep in the stack.

How do we patch it? The suggestion above doesn’t work.

[35359595]

@AraMambreyan I presume https://github.com/alloy-rs/alloy is not working either?

[AraMambreyan]

@AraMambreyan I presume https://github.com/alloy-rs/alloy is not working either?

Migrating to Alloy will be a huge effort for a repository as large as ours.

I doubt it will work anyway as I’d expect it to use newer versions of the dependencies than ethers-rs.

[appetrosyan]

I would suggest vendoring your own versions of the dependencies instead. I was supposed to fix this, but had to move to other projects. We could pool efforts on that. You should have my contact details, so we can discuss

[appetrosyan]

Meanwhile, here's a stack which we use in our internal projects:

ethers = { version = "=0.13.0" } # zeroize dep issue again
ethers-abi = { version = "18.0.0"}

[patch.crates-io]
der = { git = "https://github.com/tavakyan/formats.git", branch = "fix/downgrade-zeroize-version" }
curve25519-dalek = { git = "https://github.com/solana-labs/curve25519-dalek.git", rev = "b500cdc2a920cd5bff9e2dd974d7b97349d61464" }

Which should work with

resolver = "2"

[AraMambreyan]

I would suggest vendoring your own versions of the dependencies instead. I was supposed to fix this, but had to move to other projects. We could pool efforts on that. You should have my contact details, so we can discuss

Oh, hey Alex, good to see you here, small world!

Will ping you.

[tarcieri]

Note: there's an upstream fix being worked on here: https://github.com/anza-xyz/agave/pull/1693

[appetrosyan]

That's a good start.

[danielementary]

I used the patches from solana Cargo.toml and it worked for compiling my project with both Solana and libp2p