search

geneontology / noctua

Graph-based modeling environment for biology, including prototype editor and services
http://noctua.geneontology.org/
BSD 3-Clause "New" or "Revised" License
36 stars 13 forks source link

Data files need to be under control of github #34

Closed kltm closed 9 years ago

kltm commented 10 years ago

Need to have a better way of saving files permanently. Use GO SVN?

kltm commented 10 years ago

Related to #33.

kltm commented 10 years ago

Weeeell...currently, the CDN does allow versioning, so I'm going to change this issue a bit to reflect our needs.

What we're really looking for here is likely something along the lines of git or a permanent versioning document store (although, again, we might be able to do this directly on the CDN as it stands depending on the defined needs).

This item needs more input.

kltm commented 10 years ago

Revisit how the CDN versioning works, and then close out if it meets the base needs.

Worst case scenario would be a Jenkins job that scraped the CDN into version control somewhere.

kltm commented 10 years ago

Okay, checked S3 docs on versioning, and I no longer think this would work well, at least out of the box. S3 is dumb dumb dumb--really intended for building a platform on rather than direct use.

As it stands now, every PUT is going to increment version, regardless of actual changes. This doesn't sound so bad at first (besides possibly getting pricey) but given the ways that tools work, listing a bucket will get the versions as well, and tools limitations will quickly become an issue (the typical get is limited to 1000 entries for results). The intention seems to be that tools will take care of checksumming, etc. on their own.

It might just be easier to have a periodic download from S3 that commits to a VCS on Jenkins (which, I think, would avoid some of the irritating key issues we wanted to work around). Or just bite the bullet and go full VCS on toaster.

kltm commented 9 years ago

@cmungall brought up the idea of using git directly again. The easier versioning would be nice. Since it would be good to have this in place around the completion of #109, I'm adding this to the current push of high-priority items.

kltm commented 9 years ago

Another talk with @cmungall and @hdietze. The decided direction is to put the files under github in geneontology/lego-models.

To support this, minerva will need to be able to take a model directory as an argument. We'll also need to decide correct procedures for when to push/pull, how/when to clobber, and when/how to restart minerva to deal with upstream model changes.

This change cannot be rolled out until the current development server and clients are updated with recent development, like #109, and we are comfortable with rolling changes again.

kltm commented 9 years ago

Now at: https://github.com/geneontology/noctua-models This will not be the main until the conversion for #109 is completed by @hdietze , however, this is technically done so I'll let the rest happen over there.