Closed wileyj closed 3 years ago
ping @MarvinJanssen
Looks good to me apart from the one point regarding mounting files.
Why do we create separate branches per mode: mocknet/testnet/mainnet + persist / non-persist? We could also split it over multiple compose files.
docker-compose -f mainnet.yaml up
docker-compose -f testnet.yaml up
Or even layer them:
docker-compose -f mainnet.yaml -f persist.yaml up
Etc. Let me know what you think. I can make those changes. We can define separate envs or another context switch to keep the same level of operability without having to manage a bunch of branches.
Definitely could do that - but that would also require specific changes to the .env files. You're kind of getting into the whole reason this repo exists - it wasn't originally supposed to do what it's used for today. it was designed literally for local mocknet dev work. it had to be repurposed quickly to provide mainnet features.
the next update, perhaps we can work together on consolidating. the idea of relying on the end-user to use the correct files on startup i think wouldn't be the best, but pehaps a wrapper could make this easier, i.e. ./launch testnet
Definitely could do that - but that would also require specific changes to the .env files. You're kind of getting into the whole reason this repo exists - it wasn't originally supposed to do what it's used for today. it was designed literally for local mocknet dev work. it had to be repurposed quickly to provide mainnet features.
the next update, perhaps we can work together on consolidating. the idea of relying on the end-user to use the correct files on startup i think wouldn't be the best, but pehaps a wrapper could make this easier, i.e.
./launch testnet
Totally! It kind of evolved beyond that, maybe it would be worth renaming the repo at some point.
I like the idea of a wrapper script that will invoke the right docker-compose files. The composability makes it very versatile. (Not to mention docker-compose.override.yaml
.) You could also separate the env files via compose and give each mode their own.
Contrived example for mocknet.yaml
:
follower:
image: ${STACKS_IMAGE}
container_name: ${STACKS_FOLLOWER_NAME}
env_file:
- common.env
- mocknet.env
Regarding networks/DNS, those will be inherited by compose files down the chain, where the ones on the right override any parameters (if present). We can also explicitly set a network name.
It can be worth it to have a call and discuss the different ways we could go about it before we start consolidating for the next update.
Definitely could do that - but that would also require specific changes to the .env files. You're kind of getting into the whole reason this repo exists - it wasn't originally supposed to do what it's used for today. it was designed literally for local mocknet dev work. it had to be repurposed quickly to provide mainnet features. the next update, perhaps we can work together on consolidating. the idea of relying on the end-user to use the correct files on startup i think wouldn't be the best, but pehaps a wrapper could make this easier, i.e.
./launch testnet
Totally! It kind of evolved beyond that, maybe it would be worth renaming the repo at some point.
I like the idea of a wrapper script that will invoke the right docker-compose files. The composability makes it very versatile. (Not to mention
docker-compose.override.yaml
.) You could also separate the env files via compose and give each mode their own.Contrived example for
mocknet.yaml
:follower: image: ${STACKS_IMAGE} container_name: ${STACKS_FOLLOWER_NAME} env_file: - common.env - mocknet.env
Regarding networks/DNS, those will be inherited by compose files down the chain, where the ones on the right override any parameters (if present). We can also explicitly set a network name.
It can be worth it to have a call and discuss the different ways we could go about it before we start consolidating for the next update.
Sure! you have my email, we can schedule there
Use the following template to create your pull request
Description
This PR adds the ability of stacks-blockchain to use a persistent data directory at the default location of
./persistent-data/stacks-blockchain/<mode>
, and./persistent-data/postgres
. Other changes are simplifying thesample.env
file, standardizing some env vars and removing unused vars. All changes are documented in the Readme Lastly - this PR adds the ability to import v1 bns-data (currently not enabled). This involves downloading historical v1 data from a static location and extracting the files that are used to store the data in postgres via the API. This required a change to the docker-compose run script, so there is now a shell script to download/verify the files (further work can potentially be done to verify the sha256 of each file) before starting the APItagging @MarvinJanssen for visibility from the foundation.
Type of Change
Does this introduce a breaking change?
Sort of - restarting postgres/stacks-node will cause a full resync of all data. However, afterwards all restarts should be much faster since the data will be persistent across reboots in the
./persistent-data
directory. Note that the./persistent-data/bns-data
will also need to be downloaded and extracted to import V1 data (not currently active in mainnet) - this step adds some extra time to the initial startup process (as long as these files are not deleted, they won't be re-downloaded).Are documentation updates required?
Changes documented here: https://github.com/blockstack/stacks-local-dev/blob/mainnet-persist/README.md