search

filecoin-station / spark

💥 Storage Provider Retrieval Checker as a Filecoin Station Module 🛰️
https://filspark.com
Other
9 stars 2 forks source link

IPNI query failed, HTTP response: 404 #58

Closed zipiju closed 7 months ago

zipiju commented 7 months ago

Spark is unable to fetch retrieval providers due to HTTP 404 response. Restarting the Station fixes this immediatelly. DNS and connectivity are both okay while this is happening. This is on Linux machine running Station on node.js.

# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=109 time=28.4 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=109 time=26.6 ms
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 26.589/27.510/28.432/0.921 ms
# ping microsoft.com
PING microsoft.com (20.112.250.133) 56(84) bytes of data.
64 bytes from 20.112.250.133 (20.112.250.133): icmp_seq=1 ttl=102 time=144 ms
^C
--- microsoft.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 143.612/143.612/143.612/0.000 ms

HTTP 404 for a half a day or so:

[2024-04-04T09:42:28Z INFO  module:spark/main] Measurement submitted (id: 3649969074)
[2024-04-04T09:42:28Z INFO  module:spark/main] Sleeping for 79 seconds before starting the next task...
[2024-04-04T09:43:46Z INFO  module:spark/main] Getting current SPARK round details...
[2024-04-04T09:43:46Z INFO  module:spark/main] Current SPARK round: { roundId: "9678" }
[2024-04-04T09:43:46Z INFO  module:spark/main]   15 max tasks per node
[2024-04-04T09:43:46Z INFO  module:spark/main]   1000 retrieval tasks
[2024-04-04T09:43:46Z INFO  module:spark/main] {
[2024-04-04T09:43:46Z INFO  module:spark/main] Calling Filecoin JSON-RPC to get PeerId of miner f01853077
[2024-04-04T09:43:47Z INFO  module:spark/main] Found peer id: 12D3KooW9vvgYEhxLFx8Qf1LWdMB2JafT1n9dYcrncN1no3wJN2T
[2024-04-04T09:43:47Z INFO  module:spark/main] Querying IPNI to find retrieval providers for bafykbzacean2bgjljxf64wf7ih3ahgx6djm7cfhxhsgpkbvn2j33b5oyy7cza
[2024-04-04T09:43:48Z ERROR module:spark/main] IPNI query failed, HTTP response: 404
[2024-04-04T09:43:48Z INFO  module:spark/main] Submitting measurement...
[2024-04-04T09:43:48Z INFO  module:spark/main] {
      sparkVersion: "1.10.0",
[2024-04-04T09:43:48Z INFO  module:spark/main] Measurement submitted (id: 3649987167)
[2024-04-04T09:43:48Z INFO  module:spark/main] Sleeping for 79 seconds before starting the next task...
[2024-04-04T09:45:08Z INFO  module:spark/main] Getting current SPARK round details...
[2024-04-04T09:45:08Z INFO  module:spark/main] Current SPARK round: { roundId: "9678" }
[2024-04-04T09:45:08Z INFO  module:spark/main]   15 max tasks per node
[2024-04-04T09:45:08Z INFO  module:spark/main]   1000 retrieval tasks
[2024-04-04T09:45:08Z INFO  module:spark/main] {
[2024-04-04T09:45:08Z INFO  module:spark/main] Calling Filecoin JSON-RPC to get PeerId of miner f02636860
[2024-04-04T09:45:09Z INFO  module:spark/main] Found peer id: 12D3KooWHreGdoJYYexa3hEPRkq2EHMYa4rndm8f2a3QEhJXtGKH
[2024-04-04T09:45:09Z INFO  module:spark/main] Querying IPNI to find retrieval providers for bafykbzaceasowymr6cgoyqmg5dnyrz3xeiyq64ln4f33kbplfkox3ffcmxsyg
[2024-04-04T09:45:09Z ERROR module:spark/main] IPNI query failed, HTTP response: 404
[2024-04-04T09:45:09Z INFO  module:spark/main] Submitting measurement...
[2024-04-04T09:45:09Z INFO  module:spark/main] {
      sparkVersion: "1.10.0",
[2024-04-04T09:45:09Z INFO  module:spark/main] Measurement submitted (id: 3650004220)
[2024-04-04T09:45:09Z INFO  module:spark/main] Sleeping for 79 seconds before starting the next task...
[2024-04-04T09:46:28Z INFO  module:spark/main] Getting current SPARK round details...
[2024-04-04T09:46:28Z INFO  module:spark/main] Current SPARK round: { roundId: "9678" }
[2024-04-04T09:46:28Z INFO  module:spark/main]   15 max tasks per node
[2024-04-04T09:46:28Z INFO  module:spark/main]   1000 retrieval tasks
[2024-04-04T09:46:28Z INFO  module:spark/main] {
[2024-04-04T09:46:28Z INFO  module:spark/main] Calling Filecoin JSON-RPC to get PeerId of miner f02226869
[2024-04-04T09:46:29Z INFO  module:spark/main] Found peer id: 12D3KooWAXQ4F5L57v7xoaBFz6TGZsdqMPbJ46QPx17rJKp4CTtr
[2024-04-04T09:46:29Z INFO  module:spark/main] Querying IPNI to find retrieval providers for bafykbzacedrsocxs2dlw7fxrziozextdt56f3aj72oya6hqpldlfdn2hyodnw
[2024-04-04T09:46:29Z ERROR module:spark/main] IPNI query failed, HTTP response: 404
[2024-04-04T09:46:29Z INFO  module:spark/main] Submitting measurement...
[2024-04-04T09:46:29Z INFO  module:spark/main] {
      sparkVersion: "1.10.0",
[2024-04-04T09:46:29Z INFO  module:spark/main] Measurement submitted (id: 3650022128)
[2024-04-04T09:46:29Z INFO  module:spark/main] Sleeping for 78 seconds before starting the next task...
[2024-04-04T09:53:09Z INFO  module:spark/main] Measurement submitted (id: 3650107101)
[2024-04-04T09:53:09Z INFO  module:spark/main] Sleeping for 79 seconds before starting the next task...
[2024-04-04T09:54:28Z INFO  module:spark/main] Getting current SPARK round details...
[2024-04-04T09:54:28Z INFO  module:spark/main] Current SPARK round: { roundId: "9679" }
[2024-04-04T09:54:28Z INFO  module:spark/main]   15 max tasks per node
[2024-04-04T09:54:28Z INFO  module:spark/main]   1000 retrieval tasks
[2024-04-04T09:54:28Z INFO  module:spark/main] {
[2024-04-04T09:54:28Z INFO  module:spark/main] Calling Filecoin JSON-RPC to get PeerId of miner f01923787
[2024-04-04T09:54:29Z INFO  module:spark/main] Found peer id: 12D3KooWDZjxXYKproG6yeLVW7vis2jBVweXprTEhNvTFxBCAVNg
[2024-04-04T09:54:29Z INFO  module:spark/main] Querying IPNI to find retrieval providers for bafybeiface2e4laawnqcosy7ic2rjbsaybsl2vg2pm4bal3m6dhkyt4ygi
[2024-04-04T09:54:29Z ERROR module:spark/main] IPNI query failed, HTTP response: 404
[2024-04-04T09:54:29Z INFO  module:spark/main] Submitting measurement...
[2024-04-04T09:54:29Z INFO  module:spark/main] {
      sparkVersion: "1.10.0",
[2024-04-04T09:54:29Z INFO  module:spark/main] Measurement submitted (id: 3650124450)
[2024-04-04T09:54:29Z INFO  module:spark/main] Sleeping for 79 seconds before starting the next task...

Station restart here:

[spark]  ⇣ checking for updates
[spark]  ✓ no update available
[2024-04-04T09:54:36Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "f410fowa5fchvcyrjahvldu2axz3teqofhubfaawildq", state_root: "/home/filstation/.local/state/filecoin-station-core/modules/zinnia", cache_root: "/home/filstation/.cache/filecoin-station-core/modules/zinnia", files: ["spark/main.js"] }
[2024-04-04T09:54:36Z INFO  zinniad] Starting module file:///home/filstation/.cache/filecoin-station-core/sources/spark/main.js
[2024-04-04T09:54:36Z INFO  module:spark/main] Getting current SPARK round details...
[2024-04-04T09:54:40Z INFO  module:spark/main] Current SPARK round: { roundId: "9679" }
[2024-04-04T09:54:40Z INFO  module:spark/main]   15 max tasks per node
[2024-04-04T09:54:40Z INFO  module:spark/main]   1000 retrieval tasks
[2024-04-04T09:54:40Z INFO  module:spark/main] {
[2024-04-04T09:54:40Z INFO  module:spark/main] Calling Filecoin JSON-RPC to get PeerId of miner f01828146
[2024-04-04T09:54:41Z INFO  module:spark/main] Found peer id: 12D3KooWHJpgrX7LHiR2u5Y76YMYQnFpHEm1EQcQioDV8TfyKtVf
[2024-04-04T09:54:41Z INFO  module:spark/main] Querying IPNI to find retrieval providers for bafybeidken4tgfxmg3hsxhpev5pmmpft6i33jkr3wia4otfiwi6enfapai
[2024-04-04T09:54:42Z INFO  module:spark/main] IPNI returned 1 provider results
[2024-04-04T09:54:42Z INFO  module:spark/main] HTTP protocol is not advertised, falling back to Graphsync.
[2024-04-04T09:54:42Z INFO  module:spark/main] Fetching: ipfs://bafybeidken4tgfxmg3hsxhpev5pmmpft6i33jkr3wia4otfiwi6enfapai?dag-scope=block&protocols=graphsync&providers=%2Fip4%2F162.219.34.221%2Ftcp%2F41604%2Fp2p%2F12D3KooWHJpgrX7LHiR2u5Y76YMYQnFpHEm1EQcQioDV8TfyKtVf
[2024-04-04T09:54:44Z ERROR module:spark/main] Retrieval failed with status code 502: no candidates found
[2024-04-04T09:54:44Z INFO  module:spark/main] {
[2024-04-04T09:54:44Z INFO  module:spark/main] Submitting measurement...
[2024-04-04T09:54:44Z INFO  module:spark/main] {
      sparkVersion: "1.10.0",
[2024-04-04T09:54:44Z INFO  module:spark/main] Measurement submitted (id: 3650127595)
[2024-04-04T09:54:44Z INFO  module:spark/main] Sleeping for 73 seconds before starting the next task...
bajtos commented 7 months ago

IPNI's response 404 means that no advertisement was found for the requested CID. This is expected.

You can see in your logs that each IPNI request was asking for a different CID. Only ~10% of the CIDs sampled have an IPNI advertisement, that's why most retrieval checks end up with the result "IPNI 404 ERROR". Again, this is expected given the current state of the Filecoin retrievals.