Open maierthomas opened 6 years ago
Comment by alexbrdn
Thursday Oct 27, 2016 at 09:03 GMT
@heydenreich, @mcjaeger, @maximilianhuber, @bs-jokri, you are invited to give your feedback on this
Comment by bs-jokri
Monday Nov 07, 2016 at 09:48 GMT
Another idea might be to use tools like this one https://github.com/haduart/flycouchdb
However, I haven't tested it yet.
Comment by alexbrdn
Monday Nov 07, 2016 at 13:47 GMT
Granted, flycouchdb runs in the JVM, but it introduces a dependency on Clojure. Better than Python, I guess, but not by much.
I haven't found any CouchDB migration library written in Java, which would be ideal.
Comment by maxhbr
Monday Nov 07, 2016 at 14:03 GMT
I am not sure, but isn't Clojure a compile time dependency and python + python-couchdb a runtime dependency?
I think runtime dependency are worse.
Comment by alexbrdn
Thursday Nov 10, 2016 at 13:16 GMT
from discussion: we will try flycouchdb first. If that works, we won't try to invent the bicycle anew.
Comment by maxhbr
Wednesday May 10, 2017 at 08:32 GMT
@alex-evo Another alternative:
add to every thrift structure the filed Int dataTypeRevision
with a default value corresponding to the current revision (starting with 1
, old thrift structures without that value are treated as 0
). This would allow us to write migration scripts in the following way:
for each JSON-object check which type and revision it has. Then apply all known migrations and write the current revision into the objet.
Problem: how to handle global changes / merge of objects / ...
Changes in the Thrift data model sometimes produce incompatibilities between newer code and older data in existing couchdb instances. A library to facilitate automatic migration of the data to the new data model version should be developed.
So far, there were a few Python scripts developed for migrating data (see scripts/migrations), but there is no automation to that. It is necessary to manually start the correct scripts in the correct order. Also, these scripts introduce a dependency to Python, which is otherwise unnecessary.
The requirements for the new database migration library are: