Improve the web upload API

[rkdarst]

This is a somewhat major issue that needs discussion. @arashbm and I have already discussed and sort of have a plan. @arashbm, would you care to update here? I will do it tomorrow if not.

This was somewhat discussed in denzilferreira/aware-client#29 (misnamed issue, but has beginnings of a plan in it including configuration variables) and denzilferreira/aware-client#25 (just background info).

[denzilferreira]

For the /sdcard/ storage, I just though of something else: ethics/security people may not like the idea of one app collecting data, and it possibly being available for other apps who haven't requested permissions to collect that data. I hadn't even thought about this before... for now I'll ignore it.

I've fixed a few of the issues:

• We now do: create once, latest once, insert/delete after that.
• We only upload the tables where there is new data.
• There is a permission that is required to read/write to the databases on the storage. An app without the com.aware.READ_CONTEXT_DATA or com.aware.WRITE_CONTEXT_DATA can't use/query the database on the phone. Of course, you can always move the .sql file away from the phone, but nothing we can do there.

[rkdarst]

Hi,

I see these changes in 834506, I didn't realize that there were more upload API changes there. It looks nice, but I think there is still more we would like to do. I aim to write this out tomorrow, then you can comment. We have a backwards-compatible plan so it shouldn't matter if it is only in our version or both.

Also, I didn't know about those {READ,WRITE}_CONTEXT_DATA permissions. That is very good and I am adding a note of that to my materials. Do apps need either READ_CONTEXT_DATA or the READ_EXTERNAL_STORAGE (if that's the right one) permission to read the context data, or both?

Thanks,

• Richard

[denzilferreira]

They need both.

[rkdarst]

Here is what I have written about the API changes: https://github.com/CxAalto/koota-server/wiki/AwareDev#new-api

Everything should be backwards compatible. This gives the option to have long-term studies which clean up data from the devices, and I think makes the Android behavior like the iOS behavior. It should be implementable with two new variables and some conditionals.

[rkdarst]

@denzilferreira, by the way, I am still getting uploads that are like 57 requests / 5 inserts. I meant to go look at the patch that was supposed to improve things, but haven't gotten around to it yet. Do you think it should currently be making so many requests? If you can't think of anything, I will get into the code and see what I can find.

Thanks,

• Richard

[rkdarst]

I have a commit for this here: https://github.com/CxAalto/aware-client/commit/ed30c9ed52a96cbe45417fa77c545d0e5cb6f5fe and it serves our needs. Note: commit may be rebased, title is "Add support for more API: webservice_stateless and webservice_only" in "darst" branch.

@denzilferreira, it does not change default behavior and does not present any extra user interface. As you say it will affect some sensors or plugins, but the researcher/administrator will have to beware then. If you want this patch, I will first update it to have proper indention, and notes that it does not work for some plugins.