Cyclic dependency between solana and solana-program-library makes it hard to compile project with personal solana repo

[godmodegalactus]

Problem

Compiling a repository with a local or solana git repo is not possible because there is cyclic dependency between solana and solana-program-library.

Here is an example:

I have a project which uses some solana libraries like solana-geyser-plugin-interface, solana-transaction-status and few more, and i want to compile it against my personal repository.

# cargo file
solana-geyser-plugin-interface = { path = "../solana/geyser-plugin-interface" }
solana-logger = { path = "../solana/solana/logger" }
solana-sdk = { path = "../solana/mangolana/solana/sdk" }
solana-transaction-status = { path = "../solana/transaction-status" }

When i do cargo build I get following error :

error[E0308]: mismatched types
    --> /home/galactus/mangolana/solana/account-decoder/src/parse_account_data.rs:37:18
     |
27   |         m.insert(
     |         - ... which causes `m` to have type `HashMap<solana_sdk::pubkey::Pubkey, ParsableAccount>`
28   |             *ADDRESS_LOOKUP_PROGRAM_ID,
     |             -------------------------- this argument has type `solana_sdk::pubkey::Pubkey`...
...
37   |         m.insert(spl_token::id(), ParsableAccount::SplToken);
     |           ------ ^^^^^^^^^^^^^^^ expected `solana_sdk::pubkey::Pubkey`, found `spl_token_2022::solana_program::pubkey::Pubkey`
     |           |
     |           arguments to this method are incorrect
     |
     = note: `spl_token_2022::solana_program::pubkey::Pubkey` and `solana_sdk::pubkey::Pubkey` have similar names, but are actually distinct types
note: `spl_token_2022::solana_program::pubkey::Pubkey` is defined in crate `solana_program`
    --> /home/galactus/.cargo/registry/src/index.crates.io-6f17d22bba15001f/solana-program-1.16.13/src/pubkey.rs:87:1
     |
87   | pub struct Pubkey(pub(crate) [u8; 32]);
     | ^^^^^^^^^^^^^^^^^
note: `solana_sdk::pubkey::Pubkey` is defined in crate `solana_program`
    --> /home/galactus/mangolana/solana/sdk/program/src/pubkey.rs:87:1
     |
87   | pub struct Pubkey(pub(crate) [u8; 32]);
     | ^^^^^^^^^^^^^^^^^
     = note: perhaps two different versions of crate `solana_program` are being used?
note: method defined here
    --> /home/galactus/.rustup/toolchains/1.70.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/collections/hash/map.rs:1103:12
     |
1103 |     pub fn insert(&mut self, k: K, v: V) -> Option<V> {
     |            ^^^^^^

error[E0308]: mismatched types
    --> /home/galactus/mangolana/solana/account-decoder/src/parse_account_data.rs:38:18
     |
27   |         m.insert(
     |         - ... which causes `m` to have type `HashMap<solana_sdk::pubkey::Pubkey, ParsableAccount>`
28   |             *ADDRESS_LOOKUP_PROGRAM_ID,
     |             -------------------------- this argument has type `solana_sdk::pubkey::Pubkey`...
...
38   |         m.insert(spl_token_2022::id(), ParsableAccount::SplToken2022);
     |           ------ ^^^^^^^^^^^^^^^^^^^^ expected `solana_sdk::pubkey::Pubkey`, found `spl_token_2022::solana_program::pubkey::Pubkey`
     |           |
     |           arguments to this method are incorrect
     |
     = note: `spl_token_2022::solana_program::pubkey::Pubkey` and `solana_sdk::pubkey::Pubkey` have similar names, but are actually distinct types
note: `spl_token_2022::solana_program::pubkey::Pubkey` is defined in crate `solana_program`
    --> /home/galactus/.cargo/registry/src/index.crates.io-6f17d22bba15001f/solana-program-1.16.13/src/pubkey.rs:87:1
     |
87   | pub struct Pubkey(pub(crate) [u8; 32]);
     | ^^^^^^^^^^^^^^^^^
note: `solana_sdk::pubkey::Pubkey` is defined in crate `solana_program`
    --> /home/galactus/mangolana/solana/sdk/program/src/pubkey.rs:87:1
     |
87   | pub struct Pubkey(pub(crate) [u8; 32]);
     | ^^^^^^^^^^^^^^^^^
     = note: perhaps two different versions of crate `solana_program` are being used?
note: method defined here
    --> /home/galactus/.rustup/toolchains/1.70.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/collections/hash/map.rs:1103:12
     |
1103 |     pub fn insert(&mut self, k: K, v: V) -> Option<V> {
     |            ^^^^^^

...

Here I have no dependency with spl-token or spl-token-2022.

This makes it hard to compile and test new solana features.

Proposed Solution

It am not against this cyclic dependency between both the projects but I believe that definitions that are in solana should stay in solana like Pubkey, Instruction etc.

[joncinque]

If you're using local solana libraries, I think you'll need to do the same thing as the monorepo, by patching the spl dependencies to use your local sdk, ie:

https://github.com/solana-labs/solana/blob/3008cd8ac156acd1d9752494abc3e7991417d9f0/Cargo.toml#L423-L447

[buffalu]

thank u @joncinque saved us some copy-paste 😆

[stephenwilliams1990]

@joncinque I'm having the same issue but from running a client where I am importing solana-program and solana-sdk (they aren't stored locally). Is there a way to achieve the above considering I can't specify the path to "sdk/program" for the solana-program import?

[CriesofCarrots]

@joncinque I'm having the same issue but from running a client where I am importing solana-program and solana-sdk (they aren't stored locally). Is there a way to achieve the above considering I can't specify the path to "sdk/program" for the solana-program import?

@stephenwilliams1990 , the patch approach should still work with solana dependencies with other non-crates-io sources, like github. Is that what you're using? If you have the patch statements and are still hitting errors, I recommend deleting your Cargo.lock (or if you don't want to do that, remove any entries for crates.io solana crates in the lockfile).

[stephenwilliams1990]

Thanks for responding @CriesofCarrots , the patch method ended up working fine here.

[formysister]

After anchor version changed, I got similar error.

Anyone experienced?

[hammad-ali18]

After anchor version changed, I got similar error.

Anyone experienced? may be it the code it should be taking pubkey from anchor_lang cargo clean then cargo update cargo add anchor-spl@0.29.0 too