Closed AskAlexSharov closed 1 month ago
I'd like to suggest an alternative to using r2 to distributing chain contents.
I think we should do the following:
the rational for this is that it removes the R2 dependency and has a torrent based fail-over as other peers with the chain file can server it.
I think we can use our snapshotters as the primary provider for the torrents, with a back-up on R2. This makes the model consistent with all other files.
An example of how this may look using info hash ant txt records, (we may also be able to use URL records - but this would need more investigation)
latest._chain.mainnet.erigon.network. CNAME 1.20240128._chain.mainnet.erigon.network.
0.20240127._chain.mainnet.erigon.network. IN TXT "5623d8d6aa7f3aca0d034fd2345ba1d64202145e"
0.20240128._chain.mainnet.erigon.network. IN TXT "8972b8d6aa7f3aca0d034fd2345ba1d6445da4a4"
1.20240128._chain.mainnet.erigon.network. IN TXT "7319f8d6aa7f3aca0d034fd2345ba1d6429dc37f"
or
latest._chain.mainnet.erigon.network. IN TXT "v=202401281"
_chain.mainnet.erigon.network. IN TXT "v=202401270;l=5623d8d6aa7f3aca0d034fd2345ba1d64202145e"
_chain.mainnet.erigon.network. IN TXT "v=202401280;l=8972b8d6aa7f3aca0d034fd2345ba1d6445da4a4"
_chain.mainnet.erigon.network. IN TXT "v=202401281;l=7319f8d6aa7f3aca0d034fd2345ba1d6429dc37f"
For security and integrity we would likely need to enable DNSSEC on our domains - then the client can check that the record is valid as it is signed by us.
If you assume the above the downloader start up would be as follows:
Checklist:
.chain.toml
).chain.toml
on R2 or onraw.github
(attempt to use raw.github was here: https://github.com/ledgerwatch/erigon-snapshot/pull/155 ). pros ofraw.github
- support of git branches, easier release/rollback.chain.toml
must be updated before or after merge PR tohttps://github.com/ledgerwatch/erigon-snapshot
.chain.toml
in case of problem.chain.toml
and then.chain.toml
updated (before downloading finish). will node stuck?.chain.toml
from erigon side.chain.toml
? wait? something else?)Sub-tasks: