search

storacha / freeway

🛣 Experimental IPFS HTTP gateway providing access to UnixFS data via CAR CIDs.
Other
14 stars 5 forks source link

Sharded directory file fetch returns empty CAR #86

Closed rvagg closed 10 months ago

rvagg commented 1 year ago

Not sure where this problem's going to come from, but this request returns an empty CAR:

bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54/78094.json

It should return:

bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54
bafybeiccsq2tmufaebxjxzyr5taaiilwjmea2vhky7xbpv5zzrmnjtwgxe
bafybeie5rojao66gqjtbbct24uaat43lxeqia77s6jcqr46up7u2nj74ua
bafkreiecc4sese6lnzsmykauzjoio6n67kjzmzyo7uklb6bh6p5dcoosgm

The final block being the requested file. Freeway will provide all of these blocks individually if requested with dag-scope=block.

Lots more for this same CID failing with the same error - root block missing, i.e. empty CAR:

 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /78094.json?dag-scope=entity&car-scope=file | 357
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /77115.json?dag-scope=entity&car-scope=file | 298
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /83.json?dag-scope=entity&car-scope=file    |  26
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /25.json?dag-scope=entity&car-scope=file    |  26
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /58.json?dag-scope=entity&car-scope=file    |  24
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /96.json?dag-scope=entity&car-scope=file    |  24
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /27.json?dag-scope=entity&car-scope=file    |  23
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /42.json?dag-scope=entity&car-scope=file    |  23
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /60.json?dag-scope=entity&car-scope=file    |  21
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /32.json?dag-scope=entity&car-scope=file    |  21
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /8.json?dag-scope=entity&car-scope=file     |  21
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /77.json?dag-scope=entity&car-scope=file    |  21
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /78.json?dag-scope=entity&car-scope=file    |  20
 bafybeigp4tbtictgbwgeqv4vni4yeqd6nqmibthikhnrhnw3smk3liqw54 | /38.json?dag-scope=entity&car-scope=file    |  20
alanshaw commented 1 year ago

Wow, there's 330,358 blocks in that DAG...I need to check but it might be that the worker is running out of CPU budget just reading the index.