search

metaplex-foundation / metaplex

A directory of what the Metaplex Foundation works on!
https://metaplex.com
Apache License 2.0
3.32k stars 6.26k forks source link

Using arweave storage option results in successful metadata upload and failed image upload resulting in metadata not containing valid uri and results in 100000+ rpc requests per hour #1823

Closed TranslatorDlique closed 2 years ago

TranslatorDlique commented 2 years ago

Which package is this bug report for?

candy machine cli

Issue description

  1. set config.json key "storage": "arweave" with image files 350-395kB in size each
  2. run metaplex candy-machine-v2 upload

Command

ts-node ./metaplex/js/packages/cli/src/candy-machine-v2-cli.ts verify_upload \
    -e mainnet-beta \
    -k ~/.config/solana/keypair.json \
    -cp config.json \
    -c test \
    --url <quicknodeprorpcaddress> \
    assets

Relevant log output

Error uploading file [object Object] FetchError: request to https://us-central1-metaplex-studios.cloudfunctions.net/uploadFile failed, reason: connect ETIMEDOUT 216.239.36.54:443
    at ClientRequest.<anonymous> (/home/bot/metaplex/js/node_modules/node-fetch/lib/index.js:1483:11)
    at ClientRequest.emit (node:events:402:35)
    at ClientRequest.emit (node:domain:475:12)
    at TLSSocket.socketErrorListener (node:_http_client:447:9)
    at TLSSocket.emit (node:events:390:28)
    at TLSSocket.emit (node:domain:475:12)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  type: 'system',
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT'
}
Error uploading file [object Object] FetchError: request to https://us-central1-metaplex-studios.cloudfunctions.net/uploadFile failed, reason: connect ETIMEDOUT 216.239.36.54:443
    at ClientRequest.<anonymous> (/home/bot/metaplex/js/node_modules/node-fetch/lib/index.js:1483:11)
    at ClientRequest.emit (node:events:402:35)
    at ClientRequest.emit (node:domain:475:12)
    at TLSSocket.socketErrorListener (node:_http_client:447:9)
    at TLSSocket.emit (node:events:390:28)
    at TLSSocket.emit (node:domain:475:12)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  type: 'system',
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT'
}
Error uploading file [object Object] Error: connect ETIMEDOUT 13.226.131.79:443
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1161:16) {
  errno: -110,
  code: 'ETIMEDOUT',
  syscall: 'connect',
  address: '13.226.131.79',
  port: 443,
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [Function: httpAdapter],
    transformRequest: [ [Function: transformRequest] ],
    transformResponse: [ [Function: transformResponse] ],
    timeout: 0,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    validateStatus: [Function: validateStatus],
    headers: {
      Accept: 'application/json, text/plain, */*',
      'User-Agent': 'axios/0.24.0'
    },
    url: 'https://arweave.net/price/366117',
    method: 'get',
    data: undefined
  },
  request: <ref *1> Writable {
    _writableState: WritableState {
      objectMode: false,
      highWaterMark: 16384,
      finalCalled: false,
      needDrain: false,
      ending: false,
      ended: false,
      finished: false,
      destroyed: false,
      decodeStrings: true,
      defaultEncoding: 'utf8',
      length: 0,
      writing: false,
      corked: 0,
      sync: true,
      bufferProcessing: false,
      onwrite: [Function: bound onwrite],
      writecb: null,
      writelen: 0,
      afterWriteTickInfo: null,
      buffered: [],
      bufferedIndex: 0,
      allBuffers: true,
      allNoop: true,
      pendingcb: 0,
      constructed: true,
      prefinished: false,
      errorEmitted: false,
      emitClose: true,
      autoDestroy: true,
      errored: null,
      closed: false,
      closeEmitted: false,
      [Symbol(kOnFinished)]: []
    },
    _events: [Object: null prototype] {
      response: [Function: handleResponse],
      error: [Function: handleRequestError]
    },
    _eventsCount: 2,
    _maxListeners: undefined,
    _options: {
      maxRedirects: 21,
      maxBodyLength: 10485760,
      protocol: 'https:',
      path: '/price/366117',
      method: 'GET',
      headers: [Object],
      agent: undefined,
      agents: [Object],
      auth: undefined,
      hostname: 'arweave.net',
      port: null,
      nativeProtocols: [Object],
      pathname: '/price/366117'
    },
    _ended: true,
    _ending: true,
    _redirectCount: 0,
    _redirects: [],
    _requestBodyLength: 0,
    _requestBodyBuffers: [],
    _onNativeResponse: [Function (anonymous)],
    _currentRequest: ClientRequest {
      _events: [Object: null prototype],
      _eventsCount: 7,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: false,
      _last: true,
      chunkedEncoding: false,
      shouldKeepAlive: false,
      maxRequestsOnConnectionReached: false,
      _defaultKeepAlive: true,
      useChunkedEncodingByDefault: false,
      sendDate: false,
      _removedConnection: false,
      _removedContLen: false,
      _removedTE: false,
      _contentLength: 0,
      _hasBody: true,
      _trailer: '',
      finished: true,
      _headerSent: true,
      _closed: false,
      socket: [TLSSocket],
      _header: 'GET /price/366117 HTTP/1.1\r\n' +
        'Accept: application/json, text/plain, */*\r\n' +
        'User-Agent: axios/0.24.0\r\n' +
        'Host: arweave.net\r\n' +
        'Connection: close\r\n' +
        '\r\n',
      _keepAliveTimeout: 0,
      _onPendingData: [Function: nop],
      agent: [Agent],
      socketPath: undefined,
      method: 'GET',
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      path: '/price/366117',
      _ended: false,
      res: null,
      aborted: false,
      timeoutCb: null,
      upgradeOrConnect: false,
      parser: null,
      maxHeadersCount: null,
      reusedSocket: false,
      host: 'arweave.net',
      protocol: 'https:',
      _redirectable: [Circular *1],
      [Symbol(kCapture)]: false,
      [Symbol(kNeedDrain)]: false,
      [Symbol(corked)]: 0,
      [Symbol(kOutHeaders)]: [Object: null prototype]
    },
    _currentUrl: 'https://arweave.net/price/366117',
    [Symbol(kCapture)]: false
  },
  response: undefined,
  isAxiosError: true,
  toJSON: [Function: toJSON]
}
Error uploading file [object Object] FetchError: request to https://us-central1-metaplex-studios.cloudfunctions.net/uploadFile failed, reason: Client network socket disconnected before secure TLS connection was established
    at ClientRequest.<anonymous> (/home/bot/metaplex/js/node_modules/node-fetch/lib/index.js:1483:11)
    at ClientRequest.emit (node:events:402:35)
    at ClientRequest.emit (node:domain:475:12)
    at TLSSocket.socketErrorListener (node:_http_client:447:9)
    at TLSSocket.emit (node:events:390:28)
    at TLSSocket.emit (node:domain:475:12)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  type: 'system',
  errno: 'ECONNRESET',
  code: 'ECONNRESET'
}
Error uploading file [object Object] FetchError: request to https://us-central1-metaplex-studios.cloudfunctions.net/uploadFile failed, reason: socket hang up
    at ClientRequest.<anonymous> (/home/bot/metaplex/js/node_modules/node-fetch/lib/index.js:1483:11)
    at ClientRequest.emit (node:events:402:35)
    at ClientRequest.emit (node:domain:475:12)
    at TLSSocket.socketOnEnd (node:_http_client:471:9)
    at TLSSocket.emit (node:events:402:35)
    at TLSSocket.emit (node:domain:475:12)
    at endReadableNT (node:internal/streams/readable:1343:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  type: 'system',
  errno: 'ECONNRESET',
  code: 'ECONNRESET'
}

Operating system

Linux

Priority this issue should have

Low (slightly annoying)

Check the Docs First

Khushhal23 commented 2 years ago

Hello! Please verify if your png files are less than 10 MB:) If they're more than 10 MB, then please use arweave-sol instead of arweave

stegaBOB commented 2 years ago

We have revamped the upload command and this shouldnt be possible anymore. Please pull from master and retry if you're still having issues.