search

hashlips-lab / safe-nft-metadata-provider

A safe metadata provider for your NFT collection.
MIT License
74 stars 77 forks source link

v2 beta Issues when copying data from private to public #50

Closed gabmacinas closed 2 years ago

gabmacinas commented 2 years ago

I've check on my environment variable to see if all of the values we're in the same format but throws an error when copying the data from private to public folder

[metadata-provider] [2022-06-10 10:45:37] $ node dist/index.js
[metadata-provider] [2022-06-10 10:45:38] Revealing "Asset" for token 1...
[metadata-provider] [2022-06-10 10:45:38] Error copying "Asset" for token 1.
[metadata-provider] [2022-06-10 10:45:38] NoSuchKey: null
[metadata-provider] [2022-06-10 10:45:38]     at Request.extractError (/workspace/node_modules/aws-sdk/lib/services/s3.js:711:35)
[metadata-provider] [2022-06-10 10:45:38]     at Request.callListeners (/workspace/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
[metadata-provider] [2022-06-10 10:45:38]     at Request.emit (/workspace/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
[metadata-provider] [2022-06-10 10:45:38]     at Request.emit (/workspace/node_modules/aws-sdk/lib/request.js:686:14)
[metadata-provider] [2022-06-10 10:45:38]     at Request.transition (/workspace/node_modules/aws-sdk/lib/request.js:22:10)
[metadata-provider] [2022-06-10 10:45:38]     at AcceptorStateMachine.runTo (/workspace/node_modules/aws-sdk/lib/state_machine.js:14:12)
[metadata-provider] [2022-06-10 10:45:38]     at /workspace/node_modules/aws-sdk/lib/state_machine.js:26:10
[metadata-provider] [2022-06-10 10:45:38]     at Request.<anonymous> (/workspace/node_modules/aws-sdk/lib/request.js:38:9)
[metadata-provider] [2022-06-10 10:45:38]     at Request.<anonymous> (/workspace/node_modules/aws-sdk/lib/request.js:688:12)
[metadata-provider] [2022-06-10 10:45:38]     at Request.callListeners (/workspace/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
[metadata-provider] [2022-06-10 10:45:38]   code: 'NoSuchKey',
[metadata-provider] [2022-06-10 10:45:38]   region: null,
[metadata-provider] [2022-06-10 10:45:38]   time: 2022-06-10T10:45:38.984Z,
[metadata-provider] [2022-06-10 10:45:38]   requestId: 'tx00000000000002b0fca2c-0062a320d2-319989f1-nyc3c',
[metadata-provider] [2022-06-10 10:45:38]   extendedRequestId: undefined,
[metadata-provider] [2022-06-10 10:45:38]   cfId: undefined,
[metadata-provider] [2022-06-10 10:45:38]   statusCode: 404,
[metadata-provider] [2022-06-10 10:45:38]   retryable: false,
[metadata-provider] [2022-06-10 10:45:38]   retryDelay: 74.05141726685349
[metadata-provider] [2022-06-10 10:45:38] }
[metadata-provider] [2022-06-10 10:45:39] Revealing "Metadata" for token 1...
[metadata-provider] [2022-06-10 10:45:39] Error copying "Metadata" for token 1.
[metadata-provider] [2022-06-10 10:45:39] NoSuchKey: null
[metadata-provider] [2022-06-10 10:45:39]     at Request.extractError (/workspace/node_modules/aws-sdk/lib/services/s3.js:711:35)
[metadata-provider] [2022-06-10 10:45:39]     at Request.callListeners (/workspace/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
[metadata-provider] [2022-06-10 10:45:39]     at Request.emit (/workspace/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
[metadata-provider] [2022-06-10 10:45:39]     at Request.emit (/workspace/node_modules/aws-sdk/lib/request.js:686:14)
[metadata-provider] [2022-06-10 10:45:39]     at Request.transition (/workspace/node_modules/aws-sdk/lib/request.js:22:10)
[metadata-provider] [2022-06-10 10:45:39]     at AcceptorStateMachine.runTo (/workspace/node_modules/aws-sdk/lib/state_machine.js:14:12)
[metadata-provider] [2022-06-10 10:45:39]     at /workspace/node_modules/aws-sdk/lib/state_machine.js:26:10
[metadata-provider] [2022-06-10 10:45:39]     at Request.<anonymous> (/workspace/node_modules/aws-sdk/lib/request.js:38:9)
[metadata-provider] [2022-06-10 10:45:39]     at Request.<anonymous> (/workspace/node_modules/aws-sdk/lib/request.js:688:12)
[metadata-provider] [2022-06-10 10:45:39]     at Request.callListeners (/workspace/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
[metadata-provider] [2022-06-10 10:45:39]   code: 'NoSuchKey',
[metadata-provider] [2022-06-10 10:45:39]   region: null,
[metadata-provider] [2022-06-10 10:45:39]   time: 2022-06-10T10:45:39.040Z,
[metadata-provider] [2022-06-10 10:45:39]   requestId: 'tx0000000000000a3e87e0b-0062a320d3-3199e787-nyc3c',
[metadata-provider] [2022-06-10 10:45:39]   extendedRequestId: undefined,
[metadata-provider] [2022-06-10 10:45:39]   cfId: undefined,
[metadata-provider] [2022-06-10 10:45:39]   statusCode: 404,
[metadata-provider] [2022-06-10 10:45:39]   retryable: false,
[metadata-provider] [2022-06-10 10:45:39]   retryDelay: 59.67148001329863
[metadata-provider] [2022-06-10 10:45:39] }

image folder structure image

liarco commented 2 years ago

Hi @gabmacinas, thank you very much for taking time to test v2 and provide me with some feedback. I really appreciate it!

I can't reproduce this issue, but the error is basically saying the the source file cannot be found (on the private side) when trying to copy it. It's talking about both metadata and assets, may you please show me a couple of filenames in your private folders in order to check they are actually correct?

Also, can you make sure the S3_PATH_PREFIX is empty as well? (I see your folders seem to be in the root folder of your bucket)

Thank you.

gabmacinas commented 2 years ago

Hello @liarco here's my file structure inside the private assets and metadata folder image image

also S3_PATH_PREFIX is by default to be null in my environment variables as well image

also does the repository affects the buckets region? I certainly doubt that this would be the case since my bucket was deployed in NYC3 instead of FRA1 and changed the parameters to the environment variables.

liarco commented 2 years ago

@gabmacinas everything looks fine there. The region is absolutely important (as well as the bucket name and all the other configuration data related to DigitalOcean's spaces), but it should be included in the S3_ENDPOINT_URL (each region has its own URL, https://nyc3.digitaloceanspaces.com in your case).

If the endpoint URL is also correct I will try a deployment on the NYC3 region ASAP in order to see if I can replicate the issue on my side... I have everything set up for FRA1 and it works as expected there, it looks like the only difference is the region (I can't think of other configuration parameters that might be causing the 404 error).

liarco commented 2 years ago

@gabmacinas a little update: I just saw that the error is [metadata-provider] [2022-06-10 10:45:39] NoSuchKey: null, that null there sounds a little strange to me... I'm gonna force a 404 error on my side to check that, but I think it should be telling you the exact key that couldn't be found. The fact that it says null makes me thing there might be something wrong with the data from the contract.

Is it possible for you to share the contract address in order to run my tests on that as well?

gabmacinas commented 2 years ago

heres the contract address that I've used rinkeby : 0xc52E65A3868642200442c38A855DDd737743cde3

gabmacinas commented 2 years ago

@gabmacinas everything looks fine there. The region is absolutely important (as well as the bucket name and all the other configuration data related to DigitalOcean's spaces), but it should be included in the S3_ENDPOINT_URL (each region has its own URL, https://nyc3.digitaloceanspaces.com in your case).

If the endpoint URL is also correct I will try a deployment on the NYC3 region ASAP in order to see if I can replicate the issue on my side... I have everything set up for FRA1 and it works as expected there, it looks like the only difference is the region (I can't think of other configuration parameters that might be causing the 404 error).

For this both the s3 and the App are in the same region and I've also noticed that the key is null. As for the contract I'm using the ERC721A collection.

liarco commented 2 years ago

@gabmacinas Ok, but I guess it's our own version of the contract (I mean, the one by the HashLips Lab, based on ERC721A), because the default code here is made to work with that version, so the original ERC721 (or other customizations) might require some changes in the source code.

liarco commented 2 years ago

@gabmacinas I'm sorry but this was closed automatically by mistake.

I'm sorry if this took me a lot of time to get back to you, but finally I hope I found out what's happening. I had a situation recently where I got the same error as yours and it seems to be related to leading / in object keys which are not working as expected.

The error was occurring depending on the way a user was configuring the paths (e.g. public/assets or /public/assets). Now I'm sanitizing all the keys in order to avoid this kind issues no matter what the configuration is.

May you please let me know if this fixes the error? Thank you.

gabmacinas commented 2 years ago

@gabmacinas I'm sorry but this was closed automatically by mistake.

I'm sorry if this took me a lot of time to get back to you, but finally I hope I found out what's happening. I had a situation recently where I got the same error as yours and it seems to be related to leading / in object keys which are not working as expected.

The error was occurring depending on the way a user was configuring the paths (e.g. public/assets or /public/assets). Now I'm sanitizing all the keys in order to avoid this kind issues no matter what the configuration is.

May you please let me know if this fixes the error? Thank you.

Hi @liarco, I'm closing this issue as I've tested your recommendation above by adding / in S3_PATH_PREFIX variable. Thanks a lot! It Works!.