search

ipfs / helia-verified-fetch

A fetch-like API for obtaining verified & trustless IPFS CIDs on the web
https://npmjs.com/package/@helia/verified-fetch
Other
11 stars 1 forks source link

Fix infinite querying bug (reproducible on some gateway conformance tests that were disabled in https://github.com/ipfs/helia-verified-fetch/pull/81) #84

Closed SgtPooki closed 1 month ago

SgtPooki commented 1 month ago

To reproduce:

  1. pull down the package
  2. npm run reset && npm i && npm run build
  3. Remove the commented out test for TestNativeDag in packages/gateway-conformance/src/conformance.spec.ts
  4. cd packages/gateway-conformance && DEBUG="-mocha*,-babel*,-pw*,*,*:trace,-conformance-tests*,-kubo-rpc-client*" npm run test -- -g 'TestNativeDag'
SgtPooki commented 1 month ago

Note that I tested a npm pack version of https://github.com/ipfs/helia/pull/538 against gateway-conformance tests that are timing out, and it doesn't seem to infinitely loop anymore but it still times out

SgtPooki commented 1 month ago

seems like isEvicted fix from Alex resolved part of the problem here:

  helia:verified-fetch fetch http://localhost:3442/ipfs/bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe +10ms
  helia:verified-fetch:get-resolved-accept-header no explicit IPLD content-type requested, returning incoming accept header undefined +0ms
  helia:verified-fetch output type undefined +0ms
  helia:verified-fetch:trace finding handler for cid code "85" and output type "undefined" +10ms
  helia:verified-fetch:trace calling handler "handleRaw" +0ms
  helia:verified-fetch:byte-range-context:trace no range request detected +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe finding 1-5 new provider(s) for bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe found 0/5 new providers +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe session is ready +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe found initial session peers for bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe fetching BLOCK for bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe from http://127.0.0.1:3440/ +0ms
  helia:trustless-gateway-block-broker:127.0.0.1 GET http://127.0.0.1:3440/ipfs/bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe?format=raw 500 +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe:error error querying provider TrustlessGateway { url: URL { href: 'http://127.0.0.1:3440/', origin: 'http://127.0.0.1:3440', protocol: 'http:', username: '', password: '', host: '127.0.0.1:3440', hostname: '127.0.0.1', port: '3440', pathname: '/', search: '', searchParams: URLSearchParams {}, hash: '' }, log: [Function: debug] { namespace: 'helia:trustless-gateway-block-broker:127.0.0.1', useColors: true, color: 32, extend: [Function: extend], destroy: [Function: deprecated], enabled: [Getter/Setter], inspectOpts: {}, error: [Function: debug] { namespace: 'helia:trustless-gateway-block-broker:127.0.0.1:error', useColors: true, color: 166, extend: [Function: extend], destroy: [Function: deprecated], enabled: [Getter/Setter], inspectOpts: {} }, trace: [Function: debug] { namespace: 'helia:trustless-gateway-block-broker:127.0.0.1:trace', useColors: true, color: 129, extend: [Function: extend], destroy: [Function: deprecated], enabled: [Getter/Setter], inspectOpts: {} }, diff: 0, prev: undefined, curr: 1716236294767 } }, evicting from session Error: unable to fetch raw block for CID bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe
    at TrustlessGateway.getRawBlock (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/packages/verified-fetch/node_modules/@helia/block-brokers/dist/src/trustless-gateway/trustless-gateway.js:108:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async TrustlessGatewaySession.queryProvider (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/packages/verified-fetch/node_modules/@helia/block-brokers/dist/src/trustless-gateway/session.js:19:23)
    at async raceSignal (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/node_modules/race-signal/dist/src/index.js:28:16)
    at async Job.run (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/node_modules/@libp2p/utils/dist/src/queue/job.js:55:28) +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe no session peers had block for for bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe, finding new providers +73ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe finding 1-5 new provider(s) for bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe +0ms
  helia:trustless-gateway:session:bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe:error error retrieving session block for bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe Error: unable to fetch raw block for CID bafkreignx6d6kfdi6q3sqayf4bef7y54k7ztbhd7gci23wr6cpl2opqqoe
    at TrustlessGateway.getRawBlock (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/packages/verified-fetch/node_modules/@helia/block-brokers/dist/src/trustless-gateway/trustless-gateway.js:108:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async TrustlessGatewaySession.queryProvider (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/packages/verified-fetch/node_modules/@helia/block-brokers/dist/src/trustless-gateway/session.js:19:23)
    at async raceSignal (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/node_modules/race-signal/dist/src/index.js:28:16)
    at async Job.run (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-verified-fetch/node_modules/@libp2p/utils/dist/src/queue/job.js:55:28) +1ms
  basic-server timing out request +2s
  basic-server timing out request +87ms
  basic-server timing out request +4ms
  basic-server timing out request +2ms
  basic-server timing out request +1ms
  basic-server timing out request +11ms
  basic-server timing out request +15ms
  basic-server timing out request +10ms
  basic-server timing out request +11ms
  basic-server timing out request +11ms
  basic-server timing out request +9ms
  basic-server timing out request +11ms

I think the problem might be on code in this repo now.