metaplex-foundation / js

A JavaScript SDK for interacting with Metaplex's programs
357 stars 182 forks source link

Issue using metaplex.nfts().create() on Quicknode #381

Closed indynov closed 1 year ago

indynov commented 1 year ago

Using Quicknode either Mainnet/Devnet I get this error when I attempt to use metaplex.nfts().create()

D:\#files\programming\solana\node_modules\@metaplex-foundation\js\dist\cjs\types\Account.cjs:32
    throw new SdkError.AccountNotFoundError(account.publicKey, name, {
          ^

AccountNotFoundError [MetaplexError]: Account Not Found
>> Source: SDK
>> Problem: The account of type [MintAccount] was not found at the provided address [4um9a36fPUrGiqr5sudHYgeuHEkH41TyjMkuxMwp2vmK].
>> Solution: Ensure the provided address is correct and that an account exists at this address.

    at assertAccountExists (D:\#files\programming\solana\node_modules\@metaplex-foundation\js\dist\cjs\types\Account.cjs:32:11)
    at Object.toMintAccount (D:\#files\programming\solana\node_modules\@metaplex-foundation\js\dist\cjs\types\Account.cjs:24:7)
    at Object.handle (D:\#files\programming\solana\node_modules\@metaplex-foundation\js\dist\cjs\plugins\nftModule\operations\findNftByMint.cjs:69:39)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Disposable.run (D:\#files\programming\solana\node_modules\@metaplex-foundation\js\dist\cjs\utils\Disposable.cjs:30:14)
    at async Object.handle (D:\#files\programming\solana\node_modules\@metaplex-foundation\js\dist\cjs\plugins\nftModule\operations\createNft.cjs:69:17)
    at async Disposable.run (D:\#files\programming\solana\node_modules\@metaplex-foundation\js\dist\cjs\utils\Disposable.cjs:30:14)
    at async D:\#files\programming\solana\compiled.js.file:388:15 {
  key: 'metaplex.errors.sdk.account_not_found',
  title: 'Account Not Found',
  problem: 'The account of type [MintAccount] was not found at the provided address [4um9a36fPUrGiqr5sudHYgeuHEkH41TyjMkuxMwp2vmK].',
  solution: 'Ensure the provided address is correct and that an account exists at this address.',
  source: 'sdk',
  sourceDetails: undefined,
  cause: undefined,
  logs: undefined
}

When I use https://api.devnet.solana.com RPC I do not get the error.

I would like to use Quicknode though since I cannot use the solana.com RPCs for production application and to make as many API requests.

It says "The account of type [MintAccount] was not found at the provided address [4um9a36fPUrGiqr5sudHYgeuHEkH41TyjMkuxMwp2vmK]." would you be able to loop until the mint account is found before proceeding? That may fix the issue.

Here is the full code that will run. Only need to edit the top 2 variables. It gives an error message 50%+ of the time. pastebin.com/raw/XCWGfRGJ

dn-l commented 1 year ago

Getting same errors on Quicknode, didn't try other nodes

matlirman commented 1 year ago

Getting same error - any update on this?

SaladAuthority commented 1 year ago

From my understanding of this issue, Quicknode indexes data from Solana blockchain, so its data from READ operations may be stale sometimes. I could be wrong in this point though as its an assumption on my end

The CreateNft operation has a race condition where it builder.sendAndConfirms and then calls nfts().findByMint right after, expecting nfts().findByMint to have fresh data.

I worked around this issue by adding retries to nfts().findByMint.

In the createNft method, I replaced this:

const nft = await metaplex.nfts().findByMint(
  {
    mintAddress: output.mintAddress,
    tokenAddress: output.tokenAddress,
  },
  scope
);

with this:

    await new Promise(f => setTimeout(f, 2000));
    let nft;
    for (let i=0;i<60;i++) {
      try {
        nft = await metaplex.nfts().findByMint({
          mintAddress: output.mintAddress,
          tokenAddress: output.tokenAddress
        }, scope);
        break;
      } catch (error) {
        await new Promise(f => setTimeout(f, 2000));
      }
    }
SaladAuthority commented 1 year ago

I opened a pull request for this issue:

https://github.com/metaplex-foundation/js/pull/406/files

Tested by building locally and confirming the "Account Not Found" error doesnt show up anymore

lorisleiva commented 1 year ago

Hi there 👋

Thanks for raising this, see my comment here for an explanation of what's happening and how I'm planning to tackle this whenever I can. 🙂

https://github.com/metaplex-foundation/js/pull/406#issuecomment-1313539571

indynov commented 1 year ago

I think we should use retries mechanism so that it doesn't display the error and verify everything is working correctly. It shouldn't display an error to people as that makes people not want to use it until the code is verified and error fixed by someone familiar with the project code and the error no longer displays. It will only use retries if necessary like on Quicknode until you find a better solution it would be better to add this to the code.

SaladAuthority commented 1 year ago

Thank you Lorisleiva for your work!

Our hunch is correct that Quicknode takes some time to get the fresh data, from my recent correspondence with Quicknode, pasted below:

Hi SaladAuthority, 

When you make a call through the QuickNode API, you are querying clusters of nodes. So, one call may be routed to node A, while a second call will be routed to node B. For your use case, you 'try to read x immediately after' inserting x into the Solana blockchain. In this case, it is likely that the call you are making is not hitting the same node and is hitting a node that may be briefly behind the tip of the chain; hence the information is not yet available on that node. 

We would suggest a couple of workarounds: 

Creating logic in your code to timeout momentarily before calling getMultipleAccounts
Running the getMultipleAccounts call a few blocks behind the tip of the chain so that your success rate on the requests is much higher.
Creating stickiness to one node with our custom header x-session-hash. This way, when you make multiple calls, you will hit the same node on each request.

Here is a curl example for x-session-hash

curl -X POST \
   <YOUR QUICKNODE URL> \
  --header 'Content-Type: application/json' \
  --header 'x-session-hash: <CUSTOM SESSION STRING>' \
  --data-raw '{
  "method": "getBloc,",
  "params": [],
  "id": 1,
  "jsonrpc": "2.0"
}'

Injecting this header with a custom session string (e.g., a GUID) into your requests will improve the "stickiness" of your sessions and improve the chances that you will hit the same node over and over.
indynov commented 1 year ago

Should use the retry technique until it is found before proceeding as that fixes the issue of "Mint Account Not Found" and also use the x-session-hash: header for Quicknode but cannot rely on it always since they say it will only improve and not guarantee the stickiness. In future someone can optimize using findByMint but we can use what is available for now until someone actually did the work to optimize it. It will only generally use the retry once on Quicknode to find it and all others it will not affect them since they will not need to retry to find the mint. It will very rarely need to use 1 retry since using the header will help also.

KartikSoneji commented 1 year ago

@SaladAuthority

In this case, it is likely that the call you are making is not hitting the same node and is hitting a node that may be briefly behind the tip of the chain; hence the information is not yet available on that node.

The whole point of a consensus mechanism is to avoid this exact problem. When a transaction is finalized, it should be propagated to most of the nodes in the cluster. Quicknode might be doing something different for read vs write transactions here.

@indynov The x-session-hash hash is a good idea, you can add that directly to the Connection object.

(docs)

const SESSION_STRING = new Array(32)
    .fill(0)
    .map(() => Math.ceil(Math.random() * 16).toString(16))
    .join("");

let connection = new Connection("https://api.devnet.solana.com", {
    httpHeaders: {
        "x-session-hash": SESSION_STRING
    }
});

@lorisleiva Maybe the Metaplex.make() function can automatically detect a quicknode rpc and add a session hash?

lorisleiva commented 1 year ago

Thanks for your help @KartikSoneji! I think the x-session-hash is too application-specific to be part of the JS SDK's source code but hopefully, this will help others having the same issue.