search

zingolabs / zaino

Zaino is an indexer for the Zcash blockchain implemented in Rust.
Other
5 stars 6 forks source link

update zingo-dep and zaino-testutils #73

Closed idky137 closed 1 month ago

idky137 commented 1 month ago

Zaino-testutils needs to be updated for ZIP317 and ZIP320 before integration is possible. The biggest issue is dealing with dependency conflicts between sub-dependencies in zingolib, librustzcash and the nym-sdk. One option would be to remove all nym code but this should be avoided is possible as it will require work now and add considerably more work later.

idky137 commented 1 month ago

This is an outline of an attempt at upgrading the zingolib and librustzcash deps in zaino, and errors encountered along the way.

  1. Update zingolib and zingo-netutils to "tag = zaino_dep_001", update librustzcash dep to "tag = zcash_client_sqlite-0.11.2_plus_zingolabs_changes-1-g7ad60b5d5-2-g121371a08".
    • cargo build: error: failed to select a version for subtle. ... required by package zcash_client_backend v0.13.0 (https://github.com/zingolabs/librustzcash.git?tag=zcash_client_sqlite-0.11.2_plus_zingolabs_changes-1-g7ad60b5d5-2-g121371a08#121371a0) ... which satisfies git dependency zcash_client_backend of package zingolib v0.2.0 (https://github.com/zingolabs/zingolib.git?tag=zaino_dep_001#b34f0b53) ... which satisfies git dependency zingolib of package zaino-testutils v0.1.0 (/home/idky137/src/zingo-indexer/zaino-testutils) ... which satisfies path dependency zaino-testutils (locked to 0.1.0) of package integration-tests v0.1.0 (/home/idky137/src/zingo-indexer/integration-tests) versions that meet the requirements ^2.2.3 (locked to 2.4.1) are: 2.4.1

all possible versions conflict with previously selected packages.

previously selected package subtle v2.5.0 ... which satisfies dependency subtle = "~2.5.0" of package zingolib v0.2.0 (https://github.com/zingolabs/zingolib.git?tag=zaino_dep_001#b34f0b53) ... which satisfies git dependency zingolib of package zaino-testutils v0.1.0 (/home/idky137/src/zingo-indexer/zaino-testutils) ... which satisfies path dependency zaino-testutils (locked to 0.1.0) of package integration-tests v0.1.0 (/home/idky137/src/zingo-indexer/integration-tests)

failed to select a version for subtle which could resolve this conflict

all possible versions conflict with previously selected packages.

previously selected package subtle v2.5.0 ... which satisfies dependency subtle = "~2.5.0" of package zingolib v0.2.0 (https://github.com/zingolabs/zingolib.git?tag=zaino_dep_001#b34f0b53) ... which satisfies git dependency zingolib of package zaino-testutils v0.1.0 (/home/idky137/src/zingo-indexer/zaino-testutils) ... which satisfies path dependency zaino-testutils (locked to 0.1.0) of package integration-tests v0.1.0 (/home/idky137/src/zingo-indexer/integration-tests)

failed to select a version for subtle which could resolve this conflict

all possible versions conflict with previously selected packages.

previously selected package subtle v2.5.0 ... which satisfies dependency subtle = "~2.5.0" of package zingolib v0.2.0 (https://github.com/zingolabs/zingolib.git?tag=zaino_dep_001#b34f0b53) ... which satisfies git dependency zingolib of package zaino-testutils v0.1.0 (/home/idky137/src/zingo-indexer/zaino-testutils) ... which satisfies path dependency zaino-testutils (locked to 0.1.0) of package integration-tests v0.1.0 (/home/idky137/src/zingo-indexer/integration-tests)

failed to select a version for subtle which could resolve this conflict

all possible versions conflict with previously selected packages.

previously selected package zeroize v1.6.0 ... which satisfies dependency zeroize = "^1.2" of package bip0039 v0.11.0 ... which satisfies dependency bip0039 = "^0.11" of package zingolib v0.2.0 (https://github.com/zingolabs/zingolib.git?tag=zaino_dep_001#b34f0b53) ... which satisfies git dependency zingolib of package zaino-testutils v0.1.0 (/home/idky137/src/zingo-indexer/zaino-testutils) ... which satisfies path dependency zaino-testutils of package integration-tests v0.1.0 (/home/idky137/src/zingo-indexer/integration-tests)

failed to select a version for zeroize which could resolve this conflicti

  1. Made new Zingolib branch "git = https://github.com/idky137/zingolib.git, branch = update_dep_for_zaino", updated zaino's dep to this. 2.1 Moved rustls dep from workspace to zingocli:
    • rustls = { version = "0.23.13", features = ["ring"] }

2.2 Restricted bip0039 dep:

2.3 Downgraded subtle dep:

2.4 Downgrade rustls dep:

2.5 Downgrade tonic dep:

error[E0433]: failed to resolve: could not find transport in tonic --> /home/idky137/.cargo/git/checkouts/librustzcash-2a71d81ac8756eb7/121371a/zcash_client_backend/src/proto/service.rs:270:41 270 impl CompactTxStreamerClient { ^^^^^^^^^ could not find transport in tonic
note: found an item that was configured out --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:110:9 110 pub mod transport; ^^^^^^^^^ note: the item is gated here --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:109:1
109 #[cfg(any(feature = "server", feature = "channel"))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error[E0433]: failed to resolve: could not find transport in tonic --> /home/idky137/.cargo/git/checkouts/librustzcash-2a71d81ac8756eb7/121371a/zcash_client_backend/src/proto/service.rs:274:31 274 D: TryInto, ^^^^^^^^^ could not find transport in tonic
note: found an item that was configured out --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:110:9 110 pub mod transport; ^^^^^^^^^ note: the item is gated here --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:109:1
109 #[cfg(any(feature = "server", feature = "channel"))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error[E0433]: failed to resolve: could not find transport in tonic --> /home/idky137/.cargo/git/checkouts/librustzcash-2a71d81ac8756eb7/121371a/zcash_client_backend/src/proto/service.rs:272:64 272 pub async fn connect(dst: D) -> Result<Self, tonic::transport::Error> ^^^^^^^^^ could not find transport in tonic
note: found an item that was configured out --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:110:9 110 pub mod transport; ^^^^^^^^^ note: the item is gated here --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:109:1
109 #[cfg(any(feature = "server", feature = "channel"))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error[E0433]: failed to resolve: could not find transport in tonic --> /home/idky137/.cargo/git/checkouts/librustzcash-2a71d81ac8756eb7/121371a/zcash_client_backend/src/proto/service.rs:277:31 277 let conn = tonic::transport::Endpoint::new(dst)?.connect().await?; ^^^^^^^^^ could not find transport in tonic
note: found an item that was configured out --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:110:9 110 pub mod transport; ^^^^^^^^^ note: the item is gated here --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/lib.rs:109:1
109 #[cfg(any(feature = "server", feature = "channel"))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For more information about this error, try rustc --explain E0433. error: could not compile zcash_client_backend (lib) due to 4 previous errors

2.6 downgrade tonic-build:

error[E0599]: no method named extend found for struct RootCertStore in the current scope --> zingo-netutils/src/lib.rs:98:28 98 root_store.extend(webpki_roots::TLS_SERVER_ROOTS.0.iter().map( anchor_ref { -----------^^^^^^ method not found in RootCertStore

help: one of the expressions' fields has a method of the same name | 98 | root_store.roots.extend(webpki_roots::TLS_SERVER_ROOTS.0.iter().map(|anchor_ref| { | ++++++

error[E0599]: no method named with_root_certificates found for struct ConfigBuilder<ClientConfig, WantsCipherSuites> in the current scope --> zingo-netutils/src/lib.rs:110:22 109 let config = ClientConfig::builder() __- 110 .with_root_certificates(root_store) -^^^^^^^^^^^^^^^^^^^^^^ method not found in ConfigBuilder<ClientConfig, WantsCipherSuites> _____ 
= note: the method was found for
        - `ConfigBuilder<ClientConfig, WantsVerifier>`
error[E0277]: the trait bound HttpsConnector<HttpConnector>: hyper_util::client::legacy::connect::Connect is not satisfied --> zingo-netutils/src/lib.rs:124:52 124 let client = client_from_connector(connector, false); --------------------- ^^^^^^^^^ the trait tower::Service<http::Uri> is not implemented for HttpsConnector<HttpConnector>, which is required by `HttpsConnector: hyper_
required by a bound introduced by this call 
= help: the trait `tower::Service<tonic::transport::Uri>` is implemented for `HttpsConnector<T>`
= note: required for `HttpsConnector<HttpConnector>` to implement `hyper_util::client::legacy::connect::Connect`

note: required by a bound in client_from_connector --> zingo-netutils/src/lib.rs:46:12 | 44 | pub fn client_from_connector<C, B>(connector: C, http2_only: bool) -> Box<Client<C, B>> | --------------------- required by a bound in this function 45 | where 46 | C: Connect + Clone, | ^^^^^^^ required by this bound in client_from_connector

error[E0599]: the method boxed_clone exists for struct MapRequest<Box<Client<HttpsConnector<HttpConnector>, _>>, {closure@lib.rs:127:34}>, but its trait bounds were not satisfied --> zingo-netutils/src/lib.rs:147:53 147 Ok(CompactTxStreamerClient::new(svc.boxed_clone())) ^^^^^^^^^^^ method cannot be called due to unsatisfied trait bounds
::: /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/map_request.rs:10:1 10 pub struct MapRequest<S, F> { --------------------------- doesn't satisfy _: Service<_> or _: ServiceExt<_> 
= note: the following trait bounds were not satisfied:
        `MapRequest<Box<hyper_util::client::legacy::Client<HttpsConnector<HttpConnector>, _>>, {closure@zingo-netutils/src/lib.rs:127:34: 127:89}>: tower::Service<_>`
        which is required by `MapRequest<Box<hyper_util::client::legacy::Client<HttpsConnector<HttpConnector>, _>>, {closure@zingo-netutils/src/lib.rs:127:34: 127:89}>: ServiceExt<_>`
        `&MapRequest<Box<hyper_util::client::legacy::Client<HttpsConnector<HttpConnector>, _>>, {closure@zingo-netutils/src/lib.rs:127:34: 127:89}>: tower::Service<_>`
        which is required by `&MapRequest<Box<hyper_util::client::legacy::Client<HttpsConnector<HttpConnector>, _>>, {closure@zingo-netutils/src/lib.rs:127:34: 127:89}>: ServiceExt<_>`
        `&mut MapRequest<Box<hyper_util::client::legacy::Client<HttpsConnector<HttpConnector>, _>>, {closure@zingo-netutils/src/lib.rs:127:34: 127:89}>: tower::Service<_>`
        which is required by `&mut MapRequest<Box<hyper_util::client::legacy::Client<HttpsConnector<HttpConnector>, _>>, {closure@zingo-netutils/src/lib.rs:127:34: 127:89}>: ServiceExt<_>`
error[E0599]: the method boxed_clone exists for struct MapRequest<Box<Client<HttpConnector, _>>, {closure@lib.rs:153:34}>, but its trait bounds were not satisfied --> zingo-netutils/src/lib.rs:173:53 173 Ok(CompactTxStreamerClient::new(svc.boxed_clone())) ^^^^^^^^^^^ method cannot be called due to unsatisfied trait bounds
::: /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/map_request.rs:10:1 10 pub struct MapRequest<S, F> { --------------------------- doesn't satisfy _: Service<_> or _: ServiceExt<_> 
= note: the following trait bounds were not satisfied:
        `MapRequest<Box<hyper_util::client::legacy::Client<HttpConnector, _>>, {closure@zingo-netutils/src/lib.rs:153:34: 153:89}>: tower::Service<_>`
        which is required by `MapRequest<Box<hyper_util::client::legacy::Client<HttpConnector, _>>, {closure@zingo-netutils/src/lib.rs:153:34: 153:89}>: ServiceExt<_>`
        `&MapRequest<Box<hyper_util::client::legacy::Client<HttpConnector, _>>, {closure@zingo-netutils/src/lib.rs:153:34: 153:89}>: tower::Service<_>`
        which is required by `&MapRequest<Box<hyper_util::client::legacy::Client<HttpConnector, _>>, {closure@zingo-netutils/src/lib.rs:153:34: 153:89}>: ServiceExt<_>`
        `&mut MapRequest<Box<hyper_util::client::legacy::Client<HttpConnector, _>>, {closure@zingo-netutils/src/lib.rs:153:34: 153:89}>: tower::Service<_>`
        which is required by `&mut MapRequest<Box<hyper_util::client::legacy::Client<HttpConnector, _>>, {closure@zingo-netutils/src/lib.rs:153:34: 153:89}>: ServiceExt<_>`
warning: unused import: tower::ServiceExt --> zingo-netutils/src/lib.rs:18:5 18 use tower::ServiceExt; ^^^^^^^^^^^^^^^^^

= note: #[warn(unused_imports)] on by default

Some errors have detailed explanations: E0277, E0432, E0599. For more information about an error, try rustc --explain E0277. warning: zingo-netutils (lib) generated 1 warning error: could not compile zingo-netutils (lib) due to 6 previous errors; 1 warning emitted warning: build failed, waiting for other jobs to finish...

  1. Made new librustzcash branch "git = https://github.com/idky137/librustzcash.git, branch = zaino_dep_fix", updated zingolib and zaino's deps to this. 3.1 Downgraded tonic and tonic build deps in librustzcash:
    • tonic = { version = "0.12", default-features = false }
    • tonic = { version = "0.10", default-features = false }

error: could not compile zcash_client_backend (lib) due to 300 previous errors...

  1. Update deps in zaino to new branches (even though errors..), upgrade rust version in zaino from 1.76 to 1.77, cargo update tokio:

...

error[E0599]: no associated item named GRPC_STATUS found for struct Status in the current scope --> /home/idky137/.cargo/git/checkouts/librustzcash-2a71d81ac8756eb7/121371a/zcash_client_backend/src/proto/service.rs:2075:48 2075 ... tonic::Status::GRPC_STATUS, ^^^^^^^^^^^ associated item not found in Status

note: if you're trying to build a new Status consider using one of the following associated functions: Status::new Status::ok Status::cancelled Status::unknown and 22 others --> /home/idky137/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tonic-0.12.2/src/status.rs:162:5 | 162 | pub fn new(code: Code, message: impl Into) -> Status { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ... 173 | pub fn ok(message: impl Into) -> Status { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ... 178 | pub fn cancelled(message: impl Into) -> Status { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ... 186 | pub fn unknown(message: impl Into) -> Status { | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For more information about this error, try rustc --explain E0599. error: could not compile zcash_client_backend (lib) due to 1 previous error warning: build failed, waiting for other jobs to finish...

all possible versions conflict with previously selected packages.

previously selected package zaino-testutils v0.1.0 (/home/idky137/src/zingo-indexer/zaino-testutils) ... which satisfies path dependency zaino-testutils of package integration-tests v0.1.0 (/home/idky137/src/zingo-indexer/integration-tests)

previously selected package reqwest v0.12.4 ... which satisfies dependency reqwest = "^0.12.4" of package nym-socks5-client-core v0.1.0 (https://github.com/nymtech/nym?branch=master#c3ec970a) ... which satisfies git dependency nym-socks5-client-core of package nym-sdk v0.1.0 (https://github.com/nymtech/nym?branch=master#c3ec970a) ... which satisfies git dependency nym-sdk of package zaino-serve v0.1.0 (/home/idky137/src/zingo-indexer/zaino-serve) ... which satisfies path dependency zaino-serve of package zainod v0.1.0 (/home/idky137/src/zingo-indexer/zainod) ... which satisfies path dependency zainod of package zaino-testutils v0.1.0 (/home/idky137/src/zingo-indexer/zaino-testutils) ... which satisfies path dependency zaino-testutils of package integration-tests v0.1.0 (/home/idky137/src/zingo-indexer/integration-tests)

failed to select a version for reqwest which could resolve this conflict

NOTE: Im fairly certain (through other investigations) that this goes back to the coswasm-std dep in nym's code.

idky137 commented 1 month ago

This is currently not possible as nym has a pinned library. to stop the Zaino project being held up we are temporarily removing the nym code and will re-introduce it once this conflict has been resolved.