Smarter dedup function when merging db

Unfortunately, current "Remove duplication" function doesn’t help, because 

a. my db has “well-defined” question duplicates with different answers.

b. If I change, add or delete something in the new db which should be merged to 
the old one,

the added stuff is “the better” one with less errors, because I try to 
improve the database continuously.

It’s fatal to used the lowest index instead of the higher ones…

My db is completely “question-answer-duplicate-free” and is a->z sorted, 
this means that I never have adjacent duplicates!

If I define a question twice or three times, then the answer is always 
different. If this is not the case the two entries are identical.

However: It’s not possible to use the current db to handle this feature in a 
decent way.

I you want to realize the “db modify” function in the future, I give you 
just an suggestion how you could solve this:

#1 You should add a db feature “with predefined key” (e.g. column k as the 
very last field).

In this case you e.g. expect the following db entries q (question), a (answer), 
l (level), k (key).

In case of a db with predefined key, you must handle the entries due to the key 
entries.

#2 The function “db update” should work like this:

a. loop at old db: For each entry of the old db check if the old key is in the 
new db

if not => DELETE the old entry

b. loop at new db: For each entry of the new db check if the new key is in the 
old db

if not => INSERT new entry with new key into old db.

If yes => if q(new) <> q(old) or a(new) <> a(old) or l(new) <> l(old): UPDATE 
old set with new, else do nothing

c. if k(new) is empty => INSERT the new entry without key. For future updates 
these entries will be kicked at the new “db update”.

If you want to, you can handle these entries like the current merging 
functionality.

After this, the update is done and you can kick the new db and keep the old one.

With such a mechanism you do not need to take care about the key filling 
itself. If the user wants to provide the keys within the db than he can do that.

However: This could be a future approach that may solve the mentioned issue.
Original issue reported on code.google.com by mrlhwlib...@gmail.com on 30 Apr 2013 at 4:26
Logan676 / anymemo

Smarter dedup function when merging db #209
