Adding read json and write json function to JS API

[krmanik]

Is your feature request related to a problem? Please describe.

• Anki desktop uses localhost to show cards so .json file can be accessed but in AnkiDroid reading .json leads to cors related errors
• I am creating addons like this one for AnkiDroid. The addons gamify the learning process but it needs to save the state in .json or .txt files for viewing later.

Describe the solution you'd like Option to read and write json file from reviewer using JS API. I have created PR #8113, but it leads to security related issues like arbitrary read/write of files. So, I created this issue for discussions and ideas.

Describe alternatives you've considered The alternative is to have separate app running localhost like this one with cors options but still write json data from reviewer not possible to save info.

[david-allison]

• Do we need JSON? Can we be persistence-agnostic? (int/string/float/byte[] works for SQLite). Do we want to be?
• Do we want a syncable subset of addon settings?
• How do we allow users to back them up/restore them

I'd consider using MetaDB, but there's quite a few options that we have, and we should consider requirements before making a decision

[krmanik]

Currently MetaDB store following.

/**
 * Used to store additional information besides what is stored in the deck itself.
 * <p>
 * Currently it used to store:
 * <ul>
 * <li>The languages associated with questions and answers.</li>
 * <li>The state of the whiteboard.</li>
 * <li>The cached state of the widget.</li>
 * </ul>
 */

• Do we need JSON? Can we be persistence-agnostic? (int/string/float/byte[] works for SQLite). Do we want to be?

If there options implemented to access dB and modify according to add-ons requirements from reviewer then JSON not need here. Key value pair can be stored in SQLite.

• Do we want a syncable subset of addon settings?

The sync of settings can be done when the file is in collection.media. If settings is in plain text then it will easily modified by user and may be add-ons developer don't want it.

• How do we allow users to back them up/restore them

The backups and restore is same as other file that sync between AnkiDroid and AnkiDesktop. The file should be in collection.media which only modified by the add-ons and protect from external modification.

I'd consider using MetaDB, but there's quite a few options that we have, and we should consider requirements before making a decision

MetaDB is good options but there is sync problem. Also if number of add-ons increases then the dB size also grow. So, here separate options for separate add-ons should be implemented with sync. And .dB file in collection.media will be helpful.

JSON read will be helpful in reading local JSON file. For e.g. I have created deck for Chinese characters learning with help of hanziwriter in AnkiDroid. It uses JSON data for characters stroke order. For online use there is no problems. But loading local JSON file failed to load on AnkiDroid. On AnkiDeskop JSON file is loaded from local file.

[mikehardy]

If I understand correctly, this is a problem of different styles of general access to files in the collection, from within the card?

This seems like it would take it farther from the Anki Desktop experience, and I'm not sure how AnkiMobile does it.

I'd like to go closer to that experience, not farther, and have facilities available from within the template be the same on all platforms if at all possible - storing things in our (AnkiDroid's) MetaDB is not something I like then, that will not be compatible.

I agree with David that at the core of it, it would be best if it was format agnostic but really in order to say anything more / more helpful I need to know exactly how you would do this on AnkiDesktop and same on AnkiMobile

[krmanik]

This seems like it would take it farther from the Anki Desktop experience, and I'm not sure how AnkiMobile does it.

I have tried on AnkiMobile, the json file failed to load. I think loading json files only supported on AnkiDesktop.

I agree with David that at the core of it, it would be best if it was format agnostic but really in order to say anything more / more helpful I need to know exactly how you would do this on AnkiDesktop and same on AnkiMobile

If JS Addons support to be cross platform then JS API should be there. In AnkiMobile the features will be implemented when there are demands from more users. For AnkiDesktop, I have implemented unofficial version of JS API.

As, storing state of addons in AnkiDesktop separate addons written in python can be used because desktop platform give power to developer to implement any features. But in AnkiMobile/AnkiDroid only selected features available. Using JS it is not possible to modify files for storing data.

So, I may consider separate json files for each addons to be stored in collection.media. The file will get sync with _ prefix named files. But if user delete manually then data will be lost. So, here storing inside app directory may prevent manual delete but syncing not possible.

Also for alternative .js file can be read on any platform. So data may be copied to js file and read when required.

The only problems here is writing of data for saving state. This features will increase the complexity. So, it will be better to give write permission to only selected file extension to prevent write of other files.

[mikehardy]

I think syncing is a vital part of the JS addons stuff, we want them to be cross platform (and backed up) which explains my focus on having them in the collection

My secondary focus is doing it the same as other platforms

If AnkiMobile doesn't allow file loading, but Anki Desktop does, then I really want whatever the JS interface is to be usable on both, and hopefully hide the implementation (loading from a localhost web server, or from an API here on AnkiDroid) from the actual JS Add-on authors.

[krmanik]

To hide implementation for reading file I have used this

webView.getSettings().setAllowUniversalAccessFromFileURLs(true);

It works but it is deprecated in API 30.

This method was deprecated in API level 30.
This setting is not secure, please use androidx.webkit.WebViewAssetLoader to load file content securely.

The jQuery code below on AnkiDroid works after above implementation similar to AnkiDesktop.

$.getJSON("data.json")

The asset loader already implemented in AbstractFlashcardViewer.java#L948-#L964.

[krmanik]

After closely viewing the add-ons for AnkiDesktop, I think sql db file in collection.media (for sync) will be helpful. The implementation will like this,

1. Create db
2. Read db
3. Update db
4. Delete db

Send sql commands from js to java for CRUD operation on db file and return boolean success/fail 
with info for respective commands.

But it needs to be separate file for separate add-ons to prevent from write from other add-ons.

[dae]

The media folder is not intended for frequently changing data, and that will increase the load on AnkiWeb. For small amounts of metadata, it is better stored as json objects in the collection config.

[krmanik]

Then here the implementation will be, storing data inside add-ons folder for specific addons (using JS API). This will reduce the load on AnkiWeb. And backup restore should be done manually. Like copy paste config file. Because sync of settings or other data for add-ons is not implemented by default on AnkiDesktop. So, we should follow same approach. It is up to add-ons developers how they guide users of theirs add-ons for backup and restore.

Also giving read and write permission to addons folder only when requested from JS API, may solve security related issues.

[mikehardy]

@infinyte7 why is not possible to put JSON configs for add-ons in the collection config? Then it will be synced across devices and available on desktop etc as well

[krmanik]

Thanks I will implement to put it in collection.media.

[dae]

No, please do not use the media folder for config. If the data is small, it can be stored in the collection config dictionary.

[krmanik]

The data will very small. For e.g. it may contain number of items/objects collected during review and put it in config.json. Also, it will not be used in each addon. It will be used when developer needs to store statistics and use later during review.

[github-actions[bot]]

Hello 👋, this issue has been opened for more than 2 months with no activity on it. If the issue is still here, please keep in mind that we need community support and help to fix it! Just comment something like still searching for solutions and if you found one, please open a pull request! You have 7 days until this gets closed automatically