search

IQSS / dataverse-sample-data

Scripts and sample data for demo purposes
6 stars 11 forks source link

WIP - 13 dockerize #14

Open poikilotherm opened 4 years ago

poikilotherm commented 4 years ago
pdurbin commented 4 years ago

I talked about this briefly at standup. I think it's ok to merge but we'd want to follow up basically immediately with a change to the regex at https://github.com/IQSS/dataverse-ansible/blob/1449311f2bb5a260b08c001ab51f85c762b5a510/tasks/sampledata.yml#L52-L56

I'll coordinate with @donsizemore about this.

poikilotherm commented 4 years ago

Service reminder: please don't merge yet, I would like to include the Jenkins pipeline in this PR, having container image ready to go.

pdurbin commented 4 years ago

@poikilotherm oh, in that case, back to "community dev" it goes. 😄

I just created https://github.com/IQSS/dataverse-ansible/issues/138 to track how we'll have to make a change after we merge this pull request.

4tikhonov commented 4 years ago

@poikilotherm, @pdurbin, what is the status of this issue? We would like to have all data samples ready for Jenkins pipeline integrated with Selenium tests.

poikilotherm commented 4 years ago

The only thing missing is the webhook from this project to Jenkins to build and deploy to DockerHub. Everything else should be already working.

pdurbin commented 4 years ago

We also need to coordinate with @donsizemore as a mentioned at https://github.com/IQSS/dataverse-sample-data/pull/14#issuecomment-562221524 because we don't want the "sample data" stuff to break in dataverse-ansible.

poikilotherm commented 4 years ago

@4tikhonov I just added the necessary bit for Jenkins and now @dataversebot is happily pushing things to Docker Hub, see also Jenkins.

We should fix IQSS/dataverse-ansible now before merging this PR so no one gets :gun: . I would really appreciate feedback on using the image. See also IQSS/dataverse-kubernetes#66 (which does not provide a Job yet).

4tikhonov commented 4 years ago

Sounds great, @poikilotherm! We'll test it now.

poikilotherm commented 4 years ago

Alright before we merge this, I might add another script that automates setting the configuration to retrieve the API key, fetches the key, loads the data and deactivates the config setting again. That way it's even easier to use in a K8s job :wink:

We can reuse our scripts by using Python subprocess.

4tikhonov commented 4 years ago

@poikilotherm, good idea, can you make this script fully customizable with DATAVERSE_URL, DATAVERSE_TOKEN and GitHub to sample data repository as a parameters? We need something to wipe the database/SOLR quickly and run the same tests again and again.

poikilotherm commented 4 years ago

@4tikhonov I dunno if it makes sense to enable retrieval from a different GH repo, as I package sample data from this repo with the container image. The other vars are more or less already present.

To what extent or granularity do you want to be able to wipe? Simplest option is drop deployments... This is likely beyond this repository and script scope...

pdurbin commented 4 years ago

If it helps, I have "destroy" script that destroys all the datasets and then deletes all the dataverses: https://github.com/IQSS/dataverse-sample-data/blob/baf56de2a9b8b230168d6e6e1758a10201e0ed5e/destroy_all_dvobjects.py

donsizemore commented 2 years ago

@pdurbin I appreciate your not wanting to break dataverse-ansible but the sampledata bits aren't enabled by default anywhere I know of. Y'all do what you need to do, and I'll clean up on the Ansible side.

pdurbin commented 2 years ago

@donsizemore sounds good. You hitting "approve" put this in QA on the board but I noticed there are merge conflicts. @poikilotherm can you please resolve them? Also, are you still interested in this? It's been a while since we talked about it.

pdurbin commented 1 year ago

@poikilotherm do you still want this? As a reminder, there are merge conflicts. Thanks.

pdurbin commented 1 year ago

I just gave this old PR a shout out here: