search

VirtualFlyBrain / pipeline

A repository for amalgamating pipeline issue tickets and general VFB pipeline doc
https://hub.docker.com/repository/docker/virtualflybrain/pipeline
1 stars 0 forks source link

Make sure pipeline 2 data collection volume is cleared when pipeline is re-built #68

Closed matentzn closed 5 years ago

matentzn commented 5 years ago

Hey @Robbie1977 I realised there might be a point of failure of the pipeline unaccounted for. In the triplestore portion of the pipeline, the volume that is attached to vfb-collectdata should be cleared or in any case fresh each time the pipeline is run. Otherwise it is possible that the data collected from a previous run will be loaded into the triplestore, or even worse, data that has since been discarded from the pipeline but still lives in that directory. Can you clear it as part of the config (linked above)?

Robbie1977 commented 5 years ago

If we change to an upgrade rather than a restart it should work - I'll have a play

Robbie1977 commented 5 years ago

https://jenkins.virtualflybrain.org/view/pip_pipeline2/job/pip_vfb-triplestore/configure

Now upgrades rather than restarting which will add a new volume.