feat: add Bedrock support

[dandavid3000]

Hi ! I have the issue below from bedrock-init_1. It seems something is wrong with the connection to download bedrock file

bedrock-init_1    | curl: (56) OpenSSL SSL_read: Connection reset by peer, errno 104
simple-optimism-node_bedrock-init_1 exited with code 56

bedrock-init_1    | Downloading bedrock.tar...
healthcheck_1     | {"level":30,"time":1686366721760,"msg":"starting app server"}
healthcheck_1     | {"level":30,"time":1686366721773,"port":7300,"hostname":"0.0.0.0","msg":"app server started"}
healthcheck_1     | {"level":30,"time":1686366721773,"msg":"starting main loop"}
healthcheck_1     | {"level":50,"time":1686366721788,"msg":"target client not connected"}
fault-detector_1  | {"level":30,"time":1686366722574,"msg":"starting app server"}
fault-detector_1  | {"level":30,"time":1686366722593,"port":7300,"hostname":"0.0.0.0","msg":"app server started"}
fault-detector_1  | {"level":30,"time":1686366722593,"msg":"initializing service"}
fault-detector_1  | {"level":30,"time":1686366722593,"msg":"waiting for L1 provider..."}
fault-detector_1  | {"level":30,"time":1686366723470,"msg":"L1 provider connected"}
fault-detector_1  | {"level":30,"time":1686366723470,"msg":"waiting for L2 provider..."}
fault-detector_1  | {"level":30,"time":1686366723473,"msg":"L2 provider not connected, retrying..."}
bedrock-init_1    | curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
simple-optimism-node_bedrock-init_1 exited with code 33

When this happens, I cannot shutdown docker correctly by using docker-compose down. I had to force close

Stopping simple-optimism-node_op-geth_1 ... Stopping simple-optimism-node_op-node_1 ...

[smartcontracts]

Hmmm weird sounds like Cloudflare R2 isn't handling the byte range request that would be needed to resume the download. Gonna see if that's something I can fix on the Cloudflare side or if I need some other mechanism to resume downloads.

Edit: I ran the curl command locally and resuming the download seems to work fine. Gonna dig in more... Edit: I wasn't able to reproduce this. Can you try changing set -eou to set -exu at the top of init-bedrock.sh to get more detailed output about what's being executed?

[dandavid3000]

Hmmm weird sounds like Cloudflare R2 isn't handling the byte range request that would be needed to resume the download. Gonna see if that's something I can fix on the Cloudflare side or if I need some other mechanism to resume downloads.

Edit: I ran the curl command locally and resuming the download seems to work fine. Gonna dig in more... Edit: I wasn't able to reproduce this. Can you try changing set -eou to set -exu at the top of init-bedrock.sh to get more detailed output about what's being executed?

Is this the file mainnet_bedrock.rar that we're gonna download ?

It's good if we can have an option to in .env to extract a local file. That could save time.

Please find below the log with the command above.

Attaching to simple-optimism-node_bedrock-init_1
bedrock-init_1    | + source ./scripts/utils.sh
bedrock-init_1    | + INITIALIZED_FLAG=/shared/initialized.txt
bedrock-init_1    | + BEDROCK_JWT_PATH=/shared/jwt.txt
bedrock-init_1    | + GETH_DATA_DIR=/geth
bedrock-init_1    | + TORRENTS_DIR=/torrents/mainnet
bedrock-init_1    | + BEDROCK_TAR_PATH=/downloads/bedrock.tar
bedrock-init_1    | + BEDROCK_TMP_PATH=/bedrock-tmp
bedrock-init_1    | + '[' -e /shared/initialized.txt ']'
bedrock-init_1    | + echo 'Bedrock node needs to be initialized...'
bedrock-init_1    | Bedrock node needs to be initialized...
bedrock-init_1    | Initializing via download...
bedrock-init_1    | Fetching download link...
bedrock-init_1    | + echo 'Initializing via download...'
bedrock-init_1    | + echo 'Fetching download link...'
bedrock-init_1    | ++ config bedrock/mainnet/bedrock-download
bedrock-init_1    | +++ cast call 0xcbebc5ba53ff12165239cbb3d310fda2236d6ad2 'config(address,string)(string)' 0x68108902De3A5031197a6eB3b74b3b033e8E8e4d bedrock/mainnet/bedrock-download --rpc-url https://goerli.infura.io/v3/84842078b09946638c03157f83405213
bedrock-init_1    | ++ echo https://op.datadirs.xyz/mainnet-bedrock.tar
bedrock-init_1    | + BEDROCK_TAR_DOWNLOAD=https://op.datadirs.xyz/mainnet-bedrock.tar
bedrock-init_1    | + echo https://op.datadirs.xyz/mainnet-bedrock.tar
bedrock-init_1    | https://op.datadirs.xyz/mainnet-bedrock.tar
bedrock-init_1    | Downloading bedrock.tar...
bedrock-init_1    | + echo 'Downloading bedrock.tar...'
bedrock-init_1    | + download https://op.datadirs.xyz/mainnet-bedrock.tar /downloads/bedrock.tar
bedrock-init_1    | ++ curl -sI https://op.datadirs.xyz/mainnet-bedrock.tar
bedrock-init_1    | ++ awk '{print $2}'
bedrock-init_1    | + SIZE='200
bedrock-init_1    | Sat,
bedrock-init_1    | application/x-tar
bedrock-init_1    | 440818831360
bedrock-init_1    | "9c52a341994f4fd4a72891f49248f47e-9777"
bedrock-init_1    | Wed,
bedrock-init_1    | Accept-Encoding
bedrock-init_1    | max-age=14400
bedrock-init_1    | MISS
bedrock-init_1    | {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IxMvEzP2GRNh3Kr7WD73y066J2PhYV2YU62HiPNOIiRQzlmG7ni%2BjL9inaBSKcuzJKr3wRy%2FYgrD7Y809JOWNmI8idw2p8HJQ4JO3xUpkbzSugk48x7WpDDuAOHZBwi3kPA%3D"}],"group":"cf-nel","max_age":604800}
bedrock-init_1    | {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
bedrock-init_1    | cloudflare
bedrock-init_1    | 7d53ec398b49f138-CDG
bedrock-init_1    | h3=":443";'
bedrock-init_1    | + monitor_pid=40
bedrock-init_1    | + curl --no-progress-meter -o /downloads/bedrock.tar -C - https://op.datadirs.xyz/mainnet-bedrock.tar
bedrock-init_1    | + true
bedrock-init_1    | + sleep 60
bedrock-init_1    | ++ ls -l /downloads/bedrock.tar
' Wed, $'Accept-Encoding\r' $'max-age=14400\r' $'MISS\r' '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IxMvEzP2GRNh3Kr7WD73y066J2PhYV2YU62HiPNOIiRQzlmG7ni%2BjL9inaBSKcuzJKr3wRy%2FYgrD7Y809J' $'cloudflare\r' $'7d53ec398b49f138-CDG\r' 'h3=":443";' '{printf "Download Progress: %.2f%%\n", $5/size*100}'
bedrock-init_1    | awk: line 2: syntax error at or near end of file
bedrock-init_1    | + echo ''
bedrock-init_1    | + true
bedrock-init_1    | + sleep 60
bedrock-init_1    | 
bedrock-init_1    | ++ ls -l /downloads/bedrock.tar
' Wed, $'Accept-Encoding\r' $'max-age=14400\r' $'MISS\r' '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IxMvEzP2GRNh3Kr7WD73y066J2PhYV2YU62HiPNOIiRQzlmG7ni%2BjL9inaBSKcuzJKr3wRy%2FYgrD7Y809J' $'cloudflare\r' $'7d53ec398b49f138-CDG\r' 'h3=":443";' '{printf "Download Progress: %.2f%%\n", $5/size*100}'
bedrock-init_1    | awk: line 2: syntax error at or near end of file
bedrock-init_1    | 
bedrock-init_1    | + echo ''
bedrock-init_1    | + true
bedrock-init_1    | + sleep 60
bedrock-init_1    | ++ ls -l /downloads/bedrock.tar
' Wed, $'Accept-Encoding\r' $'max-age=14400\r' $'MISS\r' '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IxMvEzP2GRNh3Kr7WD73y066J2PhYV2YU62HiPNOIiRQzlmG7ni%2BjL9inaBSKcuzJKr3wRy%2FYgrD7Y809J' $'cloudflare\r' $'7d53ec398b49f138-CDG\r' 'h3=":443";' '{printf "Download Progress: %.2f%%\n", $5/size*100}'
bedrock-init_1    | awk: line 2: syntax error at or near end of file
bedrock-init_1    | + echo ''
bedrock-init_1    | + true
bedrock-init_1    | 
bedrock-init_1    | + sleep 60
bedrock-init_1    | ++ ls -l /downloads/bedrock.tar
' Wed, $'Accept-Encoding\r' $'max-age=14400\r' $'MISS\r' '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IxMvEzP2GRNh3Kr7WD73y066J2PhYV2YU62HiPNOIiRQzlmG7ni%2BjL9inaBSKcuzJKr3wRy%2FYgrD7Y809J' $'cloudflare\r' $'7d53ec398b49f138-CDG\r' 'h3=":443";' '{printf "Download Progress: %.2f%%\n", $5/size*100}'
bedrock-init_1    | awk: line 2: syntax error at or near end of file
bedrock-init_1    | + echo ''
bedrock-init_1    | 
bedrock-init_1    | + true
bedrock-init_1    | + sleep 60
bedrock-init_1    | ++ ls -l /downloads/bedrock.tar

I cannot see the human readable version anymore. I checked the file from simple-optimism-node_torrent_downloads/_data/bedrock.tar . Its size is increasing so I will leave it for a while to see

[dandavid3000]

After downloading the file for 285gb. bedrock init is dead. We need a mechanism to resume the download process

bedrock-init_1    | ++ ls -l /downloads/bedrock.tar
' Wed, $'Accept-Encoding\r' $'max-age=14400\r' $'MISS\r' '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IxMvEzP2GRNh3Kr7WD73y066J2PhYV2YU62HiPNOIiRQzlmG7ni%2BjL9inaBSKcuzJKr3wRy%2FYgrD7Y809J' $'cloudflare\r' $'7d53ec398b49f138-CDG\r' 'h3=":443";' '{printf "Download Progress: %.2f%%\n", $5/size*100}'
bedrock-init_1    | awk: line 2: syntax error at or near end of file
bedrock-init_1    | 
bedrock-init_1    | + echo ''
bedrock-init_1    | + true
bedrock-init_1    | + sleep 60
bedrock-init_1    | ++ ls -l /downloads/bedrock.tar
' Wed, $'Accept-Encoding\r' $'max-age=14400\r' $'MISS\r' '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=IxMvEzP2GRNh3Kr7WD73y066J2PhYV2YU62HiPNOIiRQzlmG7ni%2BjL9inaBSKcuzJKr3wRy%2FYgrD7Y809J' $'cloudflare\r' $'7d53ec398b49f138-CDG\r' 'h3=":443";' '{printf "Download Progress: %.2f%%\n", $5/size*100}'
bedrock-init_1    | awk: line 2: syntax error at or near end of file
bedrock-init_1    | + echo ''
bedrock-init_1    | + true
bedrock-init_1    | + sleep 60
bedrock-init_1    | 
bedrock-init_1    | curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)

[taempark]

@smartcontracts using wget instead of curl will do. We experienced the same problem for r2, too 😭

[dandavid3000]

@smartcontracts using wget instead of curl will do. We experienced the same problem for r2, too 😭

You can download the file with the link above and then use symbolic link to link it into the docker file.

ln -s path_to_source/mainnet_bedrock.rar path_to_docker/simple-optimism-node_torrent_downloads/_data/bedrock.tar

Simply disable the download part from bash script from file simple-optimism-node/scripts/init-bedrock.sh

echo "Fetching download link..."
#BEDROCK_TAR_DOWNLOAD=$(config "bedrock/$NETWORK_NAME/bedrock-download")
#echo $BEDROCK_TAR_DOWNLOAD

Re-run the docker and it should work :D However the speed is slow. You might need to add --snapshot=false for start-op-geth.sh to speed up the process.

[bimlas]

@dandavid3000 and @taempark are right, using wget to download snapshot works, then moving the archive to Docker volume (instead of linking) makes bedrock-init work.

$ sudo mv bedrock.tar /var/lib/docker/volumes/simple-optimism-node_torrent_downloads/_data/bedrock.tar

op-geth was slow for the first time, but modifying docker-compose.yml solved the problem:

  op-geth:
    entrypoint: ['/scripts/start-op-geth.sh', '--snapshot=false']

EDIT

BTW scripts/init-l2geth.sh is incomplete:

l2geth_1          | errexit         on
l2geth_1          | noglob          off
l2geth_1          | ignoreeof       off
l2geth_1          | monitor         off
l2geth_1          | noexec          off
l2geth_1          | xtrace          off
l2geth_1          | verbose         off
l2geth_1          | noclobber       off
l2geth_1          | allexport       off
l2geth_1          | notify          off
l2geth_1          | nounset         off
l2geth_1          | vi              off
l2geth_1          | pipefail        off
l2geth_1          | /scripts/init-l2geth.sh: line 26: syntax error: unexpected end of file

https://github.com/smartcontracts/simple-optimism-node/blob/41f9bac2b99366a549bd0f60e079340aa7169015/scripts/init-l2geth.sh#L20-L25

else block is missing.

[smartcontracts]

Hi hi, getting back to this. I just fixed the issue with the broken l2geth init script. Need to investigate the curl issue further.

[smartcontracts]

curl with -C - is supposed to resume the download if possible, I wonder why it's breaking

[smartcontracts]

Going to push an update that uses wget instead of curl

[smartcontracts]

Ok pushed an update to use wget instead of curl

[kchojn]

Gm, this branch works with Bedrock? Or still needs some fixes/features

[kchojn]

Gm, this branch works with Bedrock? Or still needs some fixes/features

I confirm, it works with Bedrock