search

bitcoindevkit / bdk

A modern, lightweight, descriptor-based wallet library written in Rust!
Other
841 stars 303 forks source link

Test Compact Block Filters supported backend #102

Open afilini opened 3 years ago

afilini commented 3 years ago

Test all the backends with the #[bdk_blockchain_tests] proc macro like we do with electrum in the CI. This could be done while transitioning to GitHub Actions (see #59).

I would use nigiri to have an electrs/esplora local instance, then a btcd node for compact filters.

Currently the CI downloads and runs the full node but that's pretty slow, we could make docker containers with everything preinstalled for it.

A docker image for electrs was added here: https://github.com/bitcoindevkit/bdk/commit/98803b25734428c1a20c07c890ab90058e401f98

tiero commented 3 years ago

just FYI if you are interested in running nigiri in CI pipelines such as Travis or Github Action, you can do with a template like this that basically strip put Esplora front-end and the nigiri CLI, but you can access chopsticks REST endpoints like faucet or the enhanced /tx which automatically generate a block when a tx hex is broadcast-ed.

I can look it up if you need help.

or GH Action

shesek commented 3 years ago

Currently the CI downloads and runs the full node but that's pretty slow, we could make docker containers with everything preinstalled for it.

You could also consider using the cache action to persist the downloaded files across CI builds.

Or you could combine the approaches, build the docker environment image as part of the CI process and cache it. I did something similar for bwt [0] which may be useful (with travis, but some parts are still relevant).

Some advantages of this approach: (1) the CI script is more self-contained that way, it will include the full build process without depending on any prior setup. (2) the build environment is guaranteed to be in-sync with the git commit being tested (and the Dockerfile for the build environment checked into it), without having to separately update the docker environment image. (3) people that want to run the CI on their own won't be using an online docker image blob that's difficult to audit.

[0] Note that I linked to an older .travis.yml file that I used in the past, I eventually ended up changing it and not keeping the cache across different builds to make the reproducible builds more reliable, at the cost of slower builds.

afilini commented 3 years ago

I think this issue is actually a bit outdated, @notmandatory added a docker image for electrs a while back so that we don't have to download everything at every run, this is the commit https://github.com/bitcoindevkit/bdk/commit/98803b25734428c1a20c07c890ab90058e401f98

The issue remains valid though, because we are still not testing the Esplora and CompactFilters backends.. just the thing about downloading core every time is not true anymore.

shesek commented 3 years ago

@afilini I see. But it is still currently the case that the CI script is not self-contained and requires prior setup, so this could benefit from my suggestion. (In fact, I can't seem to find the Dockerfile for bitcoindevkit/electrs anywhere -- how does that get built?)

notmandatory commented 3 years ago

@shesek the bitcoindevkit/electrs image is built from the bitcoindevkit/bitcoin-regtest-box project. For now I've just built and published the image (and one called bitcoindevkit/bitcoind) manually. In the future we could automate building and publishing these docker images with a GitHub actions workflow.

Also our current bdk ci workflow is caching cargo artifacts so the builds are pretty fast.