Set up Clariah CI/CD - Githubissues

CLARIAH / clariah-plus

This is the project planning repository for the CLARIAH-PLUS project. It groups all technical documents and discussions pertaining to CLARIAH-PLUS in a central place and should facilitate findability, transparency and project planning, for the project as a whole.

9 stars 6 forks source link

Set up Clariah CI/CD #48

Open mmisworking opened 2 years ago

mmisworking commented 2 years ago

Basic has been done on local server and a docker host

code --> gitlab --> runner --> build --> registry

Next step is to create a real life environment in dev

code -- gitlab (on k8s) -- runner (on k8s) --> build --> image --> Harbor --> replication --> scripts --> deploy to docker and or k8s

setup the environment will take place in week 7 in week 8 we hopefully have a fully operational proof-of concept.

jblom commented 2 years ago

@mmisworking sounds good. So around that time we could try to hook up a simple DANE environment? A minimal DANE environment consists of:

https://github.com/CLARIAH/DANE-server image
A RabbitMQ image
A running Elasticsearch cluster (we hook up k8s to one of our internal clusters)
One or more DANE workers (currently all private in B&G github origanisation, but will work towards opening them up)

Also once all our DANE workers are public, you will also be able to see the k8s yml files, which will speak more than a 1000 GH posts :-)

proycon commented 2 years ago

@vicding-mi @mmisworking Alongside the suggested DANE test from Jaap above, I also have a whole bunch of WP3 tools with test suites that I should be able to quickly hook up to the CI environment once it's available for testing. Let me know.

mmisworking commented 2 years ago

this week we going ffwd with building up the CI/CD env.
28 march we reserved a week to make in more production ready
production env should be ready and of april

proycon commented 2 years ago

@mmisworking @vicding-mi What's the status of this now? You expected the production env for CI/CD to be ready end of april?

mmisworking commented 2 years ago

True, but hardware was delivered later and the planning implementation/configuration of this new took some time. If the new environment is up and running we will deploy it in the new k8s clusters.

jblom commented 2 years ago

just as a sign of life and to (re)express our intensions. We can have a go with a DANE environment as soon as:

there is a CLARIAH docker registry
a persistent volume with enough data for a DANE ASR sandbox environment (e.g. 50 GB)
a namespace/project we have access to, so we can start rolling out such a DANE environment
once we have tested rolling this out manually, we can make sure to hook it up to a CLARIAH CI/CD