search

filecoin-project / lassie

A minimal universal retrieval client library for IPFS and Filecoin
Other
111 stars 17 forks source link

Will get the Error during Fetch #490

Open Terryhung opened 4 days ago

Terryhung commented 4 days ago

Hi team,

Once I try to run lassie fetch --provider-timeout=60s -o test.car -p bafybeihpoof6hr4fpzpnecgl2clxiz5kuvj4ramgpptzadipctlzwfkapm

and I will get the error like:

Found 3 storage provider candidate(s) in the indexer:
    12D3KooWMfYopqfm9HKfVnjHSnsZGQyyRYRG6TyNzgbJnqhxfjDv, Protocols: [transport-graphsync-filecoinv1]
    12D3KooWCDJ1bbfgKe77ynRs9iRKrG14RWQkE2Y1QYAk7icYhsa8, Protocols: [transport-graphsync-filecoinv1]
    12D3KooWKny8aBXqm6y9FhG84T2xbUQut1CHGLJWXehzvdfH8EEP, Protocols: [transport-graphsync-filecoinv1]
Retrieving from [12D3KooWMfYopqfm9HKfVnjHSnsZGQyyRYRG6TyNzgbJnqhxfjDv] (started-retrieval)...
Retrieving from [12D3KooWKny8aBXqm6y9FhG84T2xbUQut1CHGLJWXehzvdfH8EEP] (started-retrieval)...
Retrieving from [12D3KooWCDJ1bbfgKe77ynRs9iRKrG14RWQkE2Y1QYAk7icYhsa8] (started-retrieval)...
Retrieving from [12D3KooWCDJ1bbfgKe77ynRs9iRKrG14RWQkE2Y1QYAk7icYhsa8] (connected-to-provider)...
Retrieving from [12D3KooWCDJ1bbfgKe77ynRs9iRKrG14RWQkE2Y1QYAk7icYhsa8] (proposed)...
Retrieving from [12D3KooWMfYopqfm9HKfVnjHSnsZGQyyRYRG6TyNzgbJnqhxfjDv] (connected-to-provider)...
Retrieving from [12D3KooWCDJ1bbfgKe77ynRs9iRKrG14RWQkE2Y1QYAk7icYhsa8] (accepted)...
Retrieving from [12D3KooWKny8aBXqm6y9FhG84T2xbUQut1CHGLJWXehzvdfH8EEP] (connected-to-provider)...
Retrieving from [12D3KooWCDJ1bbfgKe77ynRs9iRKrG14RWQkE2Y1QYAk7icYhsa8] (first-byte-received)...
2024-11-27T21:00:25.078+0800    ERROR   dt_graphsync    graphsync/graphsync.go:347  attempting to configure data store: persistence option already registered
Retrieval failure for [12D3KooWCDJ1bbfgKe77ynRs9iRKrG14RWQkE2Y1QYAk7icYhsa8]: retrieval timed out after 1m0s
Retrieving from [12D3KooWKny8aBXqm6y9FhG84T2xbUQut1CHGLJWXehzvdfH8EEP] (proposed)...
Retrieving from [12D3KooWKny8aBXqm6y9FhG84T2xbUQut1CHGLJWXehzvdfH8EEP] (accepted)...
2024-11-27T21:01:25.886+0800    ERROR   dt_graphsync    graphsync/graphsync.go:347  attempting to configure data store: persistence option already registered

How to fix it? thanks!

rvagg commented 4 days ago

I'm not sure @Terryhung, I haven't seen that before and I'm unsure if that error message will be the ultimate cause for your retrievals to fail. I wouldn't at all be surprised if it's just the nature of the graphsync transport layer, we've historically had a lot of trouble getting it configured properly on the SP side to send and it's been slowly phased out in favour of trustless-http. The fact that you only have graphsync candidates for your CID is a bit unfortunate.

Try again and see if it works after waiting. I know that's not a great answer but it does seem to help, perhaps on the SP side things get reset. I've just tried your exact command I have a CAR downloading as I type this. Although one has timed out. A lot of this depends on the quality of the SP's stack. Fetching larger graphs can be a tricky business when the other side isn't set up to spew the data to you fast.

You could also try excluding some of the providers that your initial discovery finds using --exclude-providers (comma separated). Perhaps exclude 2 of the 3 and see how it goes.