Trying to understand how to get partial updates working

[PabloSalinas27]

Relevant links of the documentation that I read in relation to this (I read more, but I think these are the relevant ones): https://github.com/capacitor-community/sqlite/blob/master/docs/ImportExportJson.md https://github.com/capacitor-community/sqlite/blob/master/docs/Ionic-Angular-Usage.md Issue: Problem 1 is very basic: The format required is something like: const partialImport1: any = { database : "db-from-json", version: 1, encrypted : false, mode : "partial", tables :[ //here comes the tables + data]//]} So, having a database on the server with a field in each column with the last date of modification. The original plan was: I will send the date , the server will take all the fields that changed from the sent date and will return them in the format i desire, be that json, or just raw sql querys. So the "version" field that should be returned could be the day that the request is sent?: -GET ?date=123456 (this is the last actualization date the app had) -SERVER RESPONSE {database:"dbname", version:123457(this day) -(on client) db.importFromJson(SERVER RESPONSE)

Problem 2: So, I'm developing an app with ionic/angular/capacitor and a backend with flask. I can modify the backend as I wish. I have a database on the app that needs to be updated aprox every day. The plan was to ship the app with a database already populated, but it comes with a catch. The first download of the app would need to download all the changes since the last version updated on the playstore/appstore.

[jepiqueau]

@PabloSalinas27 First thanks for looking at using the plugin. For me, the version number should be changed only when you change the schema of your database., not each time you upload new data. So i do not see the purpose of having the version number being related to the date. Wen ones open your app (ionic/angular/capacitor), the first thing that your app should do is to look at the syncdate with the getSyncDate method and request the server to send all modifications since this date. On the server, you can create a response following the json format requested by the @capacitor-community/sqlite plugin and if there are some schema update you could increase the version number in the json format from the old + 1 or equal to the version number of your server database. When the response is received by the app, the app call the isJsonValid method and then the importToJson method. After this in the app you should set the new synchronization date to the date of the day by using setSyncDate

Hope this clarify

[jepiqueau]

@PabloSalinas27 Any progress on this issue?

[PabloSalinas27]

I finally got arround it by just sending a gynormus string (i know its horrible so i'll definetly revisit the issue on chrismas when i got more time bc of exams) and parsing it client side. I still don't get how the pluging asks for the database (what type of http request arrives at the server so i could configure it to listen to), because now i just send something of the form: api.page.newdata/the_date_of_last_actualization and on serverside I take with flask the endpoint and parse it to stract the new information form the database and send it as a massive string. Sorry for doing it this way, I will definetly use this feature of the pluging in a couple of weeks so it is not horrible code as it is now.

[jepiqueau]

@PabloSalinas27 the plusieurs do not deal with http request it il your app that should do it . Your app should exportToJson and send thé json object to thé server and on a server side you should collect the json object and convert it in SQL statements and vice versa for the importFromJson

[PabloSalinas27]

Got it working finaly, It was very helpful to get it working this:

the plusieurs do not deal with http request it il your app that should do it

I finaly understood that was the job of my http module Now is working as it should, but im encountering this problem: We send to the app too much data, and for some reason the pluging stops digesting it after arround 100ms. I suspect is a feature of the JVM to try to free the app to do more things but i don't know. This is the line of the log of AndroidStudio:

I/s.medfinder.ap: Background concurrent copying GC freed 240536(8746KB) AllocSpace objects, 5(28MB) LOS objects, 49% free, 15MB/31MB, paused 45us total 101.237ms

The json we are sending is something of this type: https://gist.github.com/PabloSalinas27/083c4c57ecb2f5405597918a3a2be7b8 And it is massive. Is there a way of permitting the amount of time required to do this transaction or are we better off splitting the request into pages of, lets say, 1000 objects and then parsing those each time? Like some sort of page to loop until the end: {"page":2, "totalpage":6, databasetable: { "database": "medfinder.db", "encrypted": false, "mode": "partial", "tables": [...] }

[jepiqueau]

@PabloSalinas27 Sorry to come back so late, i did not see your last comment. I do not know what is the maximum size but as it is partial import, you can split it in several imports

[PabloSalinas27]

Sorry to come back so late, i did not see your last comment.

Don't worry, I fully understand you are doing it on your free time, so thanks for even responding in the first place, you helped me to understand this plugging not only with documentation and your responses directly to me, but also the responses to others were very helpful.

About the issue, that is what we ended up doing, a loop that splits the data in chunks and sends it to the pluging. It is a bit on the slow side, but it works like a charm.

Thanks for everything.