textbrowser / biblioteq

Archive and catalog the world for today's and tomorrow's generations! Awesome and everyware.
https://textbrowser.github.io/biblioteq/
Other
217 stars 46 forks source link

SQL database merge functionality : duplicate myoid #262

Closed meteos77 closed 1 year ago

meteos77 commented 1 year ago

I have encountered a problem with merging several databases because I can't edit some documents after the merge

it seems that the documents with the problem have the same "myoid" field

error message : "Unable to create or update the entry. Please verify that the entry does not already exist."

I join you the error file erreur-messages-fusion.txt.zip

precision : I create databases from scratch then I merge so the numbering always starts with 1 in the myoid field.

meteos77 commented 1 year ago

herewith the bases concerned : in this example 2 x myoid = 1 base1.sqlite.zip base2.sqlite.zip

base_1_and_2.sqlite.zip

textbrowser commented 1 year ago

If the myoid field is the problem, I cannot do anything about that.

textbrowser commented 1 year ago

I do not see myoid problems.

Error 1: unique constraint failed: book_binding_types.binding_type unable to fetch row. Statement: INSERT INTO book_binding_types (binding_type) VALUES (?).

Error 2: unique constraint failed: book_binding_types.binding_type unable to fetch row. Statement: INSERT INTO book_binding_types (binding_type) VALUES (?).

Error 3: unique constraint failed: book_binding_types.binding_type unable to fetch row. Statement: INSERT INTO book_binding_types (binding_type) VALUES (?).

Error 4: unique constraint failed: book_binding_types.binding_type unable to fetch row. Statement: INSERT INTO book_binding_types (binding_type) VALUES (?).

Error 5: unique constraint failed: book_sequence.value unable to fetch row. Statement: INSERT INTO book_sequence (value) VALUES (?).

Error 6: unique constraint failed: book_sequence.value unable to fetch row. Statement: INSERT INTO book_sequence (value) VALUES (?).

Error 7: unique constraint failed: book_sequence.value unable to fetch row. Statement: INSERT INTO book_sequence (value) VALUES (?).

Error 8: unique constraint failed: book_sequence.value unable to fetch row. Statement: INSERT INTO book_sequence (value) VALUES (?).

Error 9: unique constraint failed: book_sequence.value unable to fetch row. Statement: INSERT INTO book_sequence (value) VALUES (?).

Error 10: unique constraint failed: languages.language unable to fetch row. Statement: INSERT INTO languages (language) VALUES (?).

Error 11: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 12: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 13: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 14: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 15: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 16: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 17: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 18: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 19: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 20: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 21: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 22: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 23: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 24: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 25: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 26: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 27: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 28: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 29: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 30: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 31: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 32: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 33: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 34: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?).

Error 35: unique constraint failed: monetary_units.monetary_unit unable to fetch row. Statement: INSERT INTO monetary_units (monetary_unit) VALUES (?).

Error 36: unique constraint failed: monetary_units.monetary_unit unable to fetch row. Statement: INSERT INTO monetary_units (monetary_unit) VALUES (?).

meteos77 commented 1 year ago

I know, I read the log which apparently makes mistakes without consequences.

meteos77 commented 1 year ago

on the other hand I do not know if the functionality must change the myoid because with 2 merged bases there is a risk to find the same myoid?

textbrowser commented 1 year ago

Sure and change the foreign keys of all dependent database entries.

textbrowser commented 1 year ago

There is nothing in SQLite which allows you to merge multiple databases.

textbrowser commented 1 year ago

https://sqlitestudio.pl/

textbrowser commented 1 year ago

The merge tool is complete. If there are problems, those will be addressed.

I will not be adding logic to invent values for unique fields.

textbrowser commented 1 year ago

I know, I read the log which apparently makes mistakes without consequences.

The log denotes duplicates.

BQ cannot manufacture data or correct entries because of your particular data. Unique identifiers identify child-parent relationships. Detecting a duplicate OID, remembering it, and then applying it to future children is beyond the scope of the merge process.

textbrowser commented 1 year ago

I do not see any mention of a merge process.

https://github.com/pawelsalawa/sqlitestudio/blob/master/ChangeLog.md

textbrowser commented 1 year ago

So, whatever BQ is doing, it's unique and complete.

meteos77 commented 1 year ago

What I don't understand is that the myoids are assigned by BQ? the process does not work because there are errors reported that prevent the proper functioning?

textbrowser commented 1 year ago

If the final database has more items than your original databases, the process works.

textbrowser commented 1 year ago

If your databases cause conflict, BQ cannot resolve them.

meteos77 commented 1 year ago

I agree but I'm not clear on what is causing the conflict, hence my problem.

This is not the first time I have merged 2 bases and usually it worked.

I need to find out why, now that it doesn't work. I have the documents created in the reception base but I can't edit some of them?

textbrowser commented 1 year ago

Again, we're going to argue semantics over a utility that does not exist elsewhere. If you have a better implementation, implement it. :)

I'm not going to implement a mechanism which detects duplicates with relationship fields and modifies them. If myoid 1 exists in database A and database B for book X, I will not invent an myoid because the invented myoid may exist in a subsequent query. In addition, the invented myoid needs to be assigned to child entries of the original myoid.

textbrowser commented 1 year ago

"Error 24: unique constraint failed: locations.location, locations.type unable to fetch row. Statement: INSERT INTO locations (location,type) VALUES (?,?)."

This clearly indicates that a duplicate location cannot be inserted into the locations table.

meteos77 commented 1 year ago

I am not asking for further development, I am looking for an explanation for the non functioningent supplémentaire, je cherche une explication au non fonctionnement

textbrowser commented 1 year ago

I myself don't know what causes a failure and the log helps me with it. There are many, many, many possible errors that SQLite may report and you have to review them and think about them. So I cannot tell you what to do.

meteos77 commented 1 year ago

thank you with your translation in clear, I begin to understand :-)

textbrowser commented 1 year ago

Consider a complex process that may result in many failures. And suppose I design the process. You're asking me: list all 3 trillion error possibilities and how I may correct them. My response: no.

meteos77 commented 1 year ago

For this afternoon, I go back to the library for the news is the beautiful function BQ/Request

I will keep you informed of my progress have a nice day

textbrowser commented 1 year ago

I am not upset nor perplexed. I am confused about the request. Have you read the warnings about Tylenol, for example? It's like that. Many, many possibilities and many, many decisions.

textbrowser commented 1 year ago

I'm asking you to think for yourself about what you see in the report. Make decisions and learn how to correct them. If you believe a problem exists, then report it. Reporting something because you see red does not mean it's a problem with BQ.

textbrowser commented 1 year ago

"I am not programmed to invent data."

textbrowser commented 1 year ago

For this afternoon, I go back to the library for the news is the beautiful function BQ/Request

I will keep you informed of my progress have a nice day

Good luck and a nice day.

meteos77 commented 1 year ago

I am making progress but I have not yet found the solution to my PB. I know that if database enumerations are present in both databases it will cause errors, so I have to delete them first in one but there are still some left. to follow ...

textbrowser commented 1 year ago

If an enumeration exists, that's fine. What's the problem?