search

cryptoadvance / specter-desktop

A desktop GUI for Bitcoin Core optimised to work with hardware wallets
MIT License
794 stars 237 forks source link

ContinuousIntegration POC #43

Closed k9ert closed 4 years ago

k9ert commented 4 years ago

As every professional Software Project, we'll need a continuousIntegration Setup. For Reasonable alternatives are Travis-CI and Gitlab. I'm tending towards gitlab for the following reasons:

There is one disadvantage: Third-party-runtime-dependencies are better supported in Travis-CI. For that reason the POC for junction (which i did some time ago) was quite simple as it was possible to simply run the bitcoind within the travis-CI-agent: https://github.com/k9ert/junction/blob/cicd/.travis.yml There is not even anything specific needed to be done in order to tell Travis-CI that bitcoind will run on some ports on the agents.

But for the above reasons (and more below) i think it's worth doing the extra-effort to run on gitlab. On gitlab, though, it's not possible to start a service opening a port on the agent (at least not on the publicly available agents). For that you need to run a docker-executor and specify a service-image ( https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#what-is-a-service ). At the same time, that means the the tests are no longer in control when/how a bitcoind get spawned. That is imho a pitty as i really like the idea to spin-up the bitcoind as a python fixture ( https://github.com/cryptoadvance/specter-desktop/blob/master/tests/conftest.py#L15 ).

So in order to keep the ability to spinup bitcoinds on demand as a fixture, we could do that via docker. That might also be a major advantage because we might potentially integrate a whole bitcoin-core-node as a docker-image WITHIN specter-desktop. I've created some code to show how that would work on my fork on the gitlab-branch: https://github.com/k9ert/specter-desktop/blob/gitlab/tests/conftest.py In that branch you can simply run the tests via docker by starting: pytest --docker

Unfortunately such an approach no longer works with the publicly available for-free-gitlab-agents. Priviliged mode is necessary to use "Docker in docker" and therefore we need to run our own agents. https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-workflow-with-docker-executor

I'm already running my own agents and would proceed in that direction.

k9ert commented 4 years ago

The docker-in-docker-approach did not run very well and i struggled with that for way too long and finally the share-docker-socket approach worked well and is also simpler. https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-socket-binding

I'll soon create a PR.

stepansnigirev commented 4 years ago

Thanks, Kim! I never worked with CI tools before so please choose what you are more comfortable with :)

k9ert commented 4 years ago

Unfortunately, i haven't investigated good enough. The workflow implemented by gitlab does not support PRs :-(. That's really a pitty. I can only reiterate what that guy is saying here: https://gitlab.com/gitlab-org/gitlab/issues/5667#note_71672784

This feature is a must have for any open source project, same as @greg.lee this is a show stopper for us. @jamedjo I understand PRs from untrusted forks might be dangerous but this is how most of the open source contribution models work these days (and it's been like that for quite a while now). Also your solution wouldn't solve our main problem, which is giving contributors feedback on their changes. Big fan of your product here, but I have to go back to Travis for now :(

As we now already have the setup, i'll continue to implement that as we might have other limitations on the travis-side but there is no reason to not use both solutions at the same time. Rather the opposite might be true, we use both:

k9ert commented 4 years ago

This is fixed, see the PRs #45 #46 #47