Open melroy89 opened 2 years ago
@Cobra-Bitcoin I created a simple project that does exactly this.
This script will scrawl the bitcoin core web page (ONLY the bin
folder), and download all the files. Save the files into the correct folders, with the correct names. So, basically you are creating a mirror for the bin
folder.
See project: https://gitlab.melroy.org/bitcoin-dot-org/bitcoin-core-web-scraper 👍🏽
The source url that is used: https://bitcoincore.org/bin/
. I'm using a simple Python framework called Scrapy to achieve my goals. Spider source code file.
EDIT: I also containerized this web crawler app: https://hub.docker.com/r/danger89/bitcoinscraper! 🥳
The Docker image will check the website once a week for changes automatically.
Or trigger manually from a running container (for example the first time): docker exec -it bitcoincore_scraper ./start_spider.py
Result: https://bitcoin.melroy.org/bin/
Hi,
I'm running a full mirror, but I'm missing the
bin
folder (https://bitcoin.org/bin/) in this GitHub repo..This folder contains the bitcoin core. Please help me to sync and mirror the whole site!
Which should include the bin folder as well.
What do you use? Are you using a script to sync the
bin
directory? The source files are currently stored: https://bitcoincore.org/bin/Please help me.
Regards, Melroy van den Berg