Node Setup Problem - Githubissues

What is the bug? Node is not working anymore. Hubble application is attempting to download the latest database snapshot but receives a 404 error (not found). Specifically, it’s trying to access "https://download.farcaster.xyz/snapshots/MAINNET/DB_SCHEMA_2/latest.json," but the file cannot be located at that URL

Please see the error below

hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"Stopped watching contract events\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"Stopped watching block numbers\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"Stopped fname events provider\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"Sync engine stopped\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"L2EventsProvider stopped\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"Stopped libp2p...\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"stopping engine\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"engine stopped\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":30,\"time\":1730109958922,\"pid\":28,\"hostname\":\"71018ed088c2\",\"msg\":\"Hubble stopped, exiting normally\"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | 2024-10-28T10:05:58: PM2 log: App name:hubble id:0 disconnected hubble-1 | 2024-10-28T10:05:59: PM2 log: App [hubble:0] exited with code [1] via signal [SIGINT] hubble-1 | 2024-10-28T10:06:00: PM2 log: 0 application online, retry = 3 hubble-1 | 2024-10-28T10:06:02: PM2 log: 0 application online, retry = 2 hubble-1 | 2024-10-28T10:06:04: PM2 log: 0 application online, retry = 1 hubble-1 | 2024-10-28T10:06:06: PM2 log: 0 application online, retry = 0 hubble-1 | 2024-10-28T10:06:06: PM2 log: Stopping app:hubble id:0 hubble-1 | 2024-10-28T10:06:06: PM2 log: pid=28 msg=process killed hubble-1 | 2024-10-28T10:06:06: PM2 log: PM2 successfully stopped hubble-1 exited with code 0 hubble-1 | 2024-10-28T10:06:09: PM2 log: Launching in no daemon mode hubble-1 | 2024-10-28T10:06:09: PM2 log: App [hubble:0] starting in -cluster mode- hubble-1 | 2024-10-28T10:06:09: PM2 log: App [hubble:0] online hubble-1 | {"message":"\n Hubble Startup Checks\n","timestamp":"2024-10-28T10:06:12.573Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"------------------------\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Farcaster: 2024.10.16 Hubble: 1.16.1\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Detected 15GB of RAM\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Directory /home/node/app/apps/hubble/.rocks is writable\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Directory /home/node/app/apps/hubble/.hub is writable\n","timestamp":"2024-10-28T10:06:12.575Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Found PeerId 12D3KooWSWPyAHGRJu616Sdz3FQJYwJeaVxSyJa5PnTdsb8ar38g\n","timestamp":"2024-10-28T10:06:12.585Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Hubble Monitoring enabled\n","timestamp":"2024-10-28T10:06:12.588Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Network is MAINNET(1)\n","timestamp":"2024-10-28T10:06:12.588Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Bootstrapping from 4 peers\n","timestamp":"2024-10-28T10:06:12.593Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Hub Operator FID is 522239(cemvural)\n","timestamp":"2024-10-28T10:06:12.798Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Connected to L1 Ethereum node\n","timestamp":"2024-10-28T10:06:13.043Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"✅ | Connected to L2 OP Mainnet node\n","timestamp":"2024-10-28T10:06:13.230Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"\nMonitor Your Node\n","timestamp":"2024-10-28T10:06:13.292Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"----------------\n","timestamp":"2024-10-28T10:06:13.294Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"🔗 | Grafana at http://localhost:3000\n","timestamp":"2024-10-28T10:06:13.295Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"\n Starting Hubble\n","timestamp":"2024-10-28T10:06:13.295Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"------------------\n","timestamp":"2024-10-28T10:06:13.296Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"Please wait... This may take several minutes\n","timestamp":"2024-10-28T10:06:13.296Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":60,\"time\":1730109973908,\"pid\":29,\"hostname\":\"71018ed088c2\",\"err\":{\"type\":\"Error\",\"message\":\"Failed to start hub: failed to fetch snapshot metadata - Failed to get latest snapshot from https://download.farcaster.xyz/snapshots/MAINNET/DB_SCHEMA_11/latest.json [AxiosError: Request failed with status code 404]\",\"stack\":\"Error: Failed to start hub\n at file:///home/node/app/apps/hubble/src/cli.ts:682:14\n at /home/node/app/node_modules/neverthrow/dist/index.cjs.js:113:102\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at Command. (file:///home/node/app/apps/hubble/src/cli.ts:680:25)\ncaused by: HubError: failed to fetch snapshot metadata - Failed to get latest snapshot from https://download.farcaster.xyz/snapshots/MAINNET/DB_SCHEMA_11/latest.json [AxiosError: Request failed with status code 404]\n at Hub.attemptCatchupSyncWithSnapshot (file:///home/node/app/apps/hubble/src/hubble.ts:935:18)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at Hub.start (file:///home/node/app/apps/hubble/src/hubble.ts:632:27)\n at Command. (file:///home/node/app/apps/hubble/src/cli.ts:680:25)\"},\"msg\":\"Failed to start hub\"}\n","timestamp":"2024-10-28T10:06:13.983Z","type":"out","process_id":0,"app_name":"hubble"} hubble-1 | {"message":"{\"level\":60,\"time\":1730109973985,\"pid\":29,\"hostname\":\"71018ed088c2\",\"reason\":\"Hub Startup failed\",\"msg\":\"shutting down hub\"}\n","timestamp":"2024-10-28T10:06:13.985Z","type":"out","process_id":0,"app_name":"hubble"}

How can it be reproduced? (optional) Tried in a new server and reproduced the error. You can reproduce the the error when you want to set up the node

Additional context (optional) Node worked for weeks perfectly. This problem occured a few days ago

farcasterxyz / hub-monorepo

Node Setup Problem #2386