search

torodb / stampede

The ToroDB solution to provide better analytics on top of MongoDB and make it easier to migrate from MongoDB to SQL
https://www.torodb.com/stampede/
GNU Affero General Public License v3.0
1.76k stars 118 forks source link

Several changes to support import mode on sharding #164

Closed gortiz closed 7 years ago

gortiz commented 7 years ago

Changed import mode feature to be able to change the mode by mongo database (instead of sql database). By doing that, each replication shard can enable/disable the import mode on their databases.

This feature exposed a bug on the snapshot merger that failed when an index is deleted and created on the same transaction. To be able to fix that, the merger has been split into several merged strategies, so it should be easier to understand what it does and extend it. Once it has been split, two new strategies have been added. They automatically accept a merge on database and collections if the original database/collection is the same as the last commited one.

codecov-io commented 7 years ago

Codecov Report

Merging #164 into devel will increase coverage by 2.07%. The diff coverage is 56.79%.

@@             Coverage Diff             @@
##              devel    #164      +/-   ##
===========================================
+ Coverage     18.33%   20.4%   +2.07%     
- Complexity     1513    1909     +396     
===========================================
  Files           780     866      +86     
  Lines         26823   27721     +898     
  Branches       2340    2406      +66     
===========================================
+ Hits           4917    5656     +739     
- Misses        21572   21647      +75     
- Partials        334     418      +84
Impacted Files Coverage Δ Complexity Δ
...in/java/com/torodb/backend/StructureInterface.java 0% <ø> (ø) 0 <0> (ø) :x:
...com/torodb/core/transaction/metainf/MetaField.java 0% <ø> (ø) 0 <0> (ø) :x:
...re/transaction/metainf/MetaDocPartIndexColumn.java 0% <ø> (ø) 0 <0> (ø) :x:
...om/torodb/core/transaction/metainf/MetaScalar.java 0% <ø> (ø) 0 <0> (ø) :x:
...orodb/core/transaction/metainf/MetaCollection.java 0% <ø> (ø) 0 <0> (ø) :x:
...com/torodb/core/transaction/metainf/MetaIndex.java 0% <ø> (ø) 0 <0> (ø) :x:
.../core/transaction/metainf/MutableMetaSnapshot.java 0% <ø> (ø) 0 <0> (ø) :x:
.../torodb/core/transaction/metainf/MetaDatabase.java 0% <ø> (ø) 0 <0> (ø) :x:
...ransaction/metainf/MetaIdentifiedDocPartIndex.java 0% <ø> (ø) 0 <0> (ø) :x:
...orodb/core/transaction/metainf/MetaFieldIndex.java 0% <ø> (ø) 0 <0> (ø) :x:
... and 221 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b08a4b9...761e879. Read the comment docs.