Attachments Store/Retrieve as URL.

[NitzDKoder]

Hi @nolanlawson

As in CBL this plugin needs the attachments to be stored as blob in private memory absolute path and have a reference to it..this helps in store/retrieve attachments data as URL..Hence avoid content passing from native to Web level(JS) vice-versa.

https://github.com/couchbase/couchbase-lite-java-core/issues/683 https://github.com/couchbase/couchbase-lite-java-core/issues/895

_Retrieve: _

/** 
     * Get the URL of the file containing the contents. 
     * This property is somewhat deprecated and is made available only for use with platform APIs that 
     * require file paths/URLs, e.g. some media playback APIs. Whenever possible, use the `getContent()` 
     * method to get the input stream of the content instead. 
     * The file must be treated as read-only! DO NOT MODIFY OR DELETE IT. 
     * If the database is encrypted, attachment files are also encrypted and not directly readable, 
     * so this property will return null. 
     */ 
    @InterfaceAudience.Public 
    public URL getContentURL() {
        try { 
            return internalAttachment().getContentURL(); 
        } catch (MalformedURLException e) {
            Log.w(Database.TAG, e.toString());
        } catch (CouchbaseLiteException e) {
            Log.w(Database.TAG, e.toString());
        } 
        return null; 
    } 

https://github.com/couchbase/couchbase-lite-java-core/blob/release/1.2.0/src/main/java/com/couchbase/lite/Attachment.java

**Store:**

/** 
     * Sets the attachment with the given name. The Attachment data will be written 
     * to the Database when the Revision is saved. 
     * 
     * @param name             The name of the Attachment to set. 
     * @param contentType      The content-type of the Attachment. 
     * @param contentStreamURL The URL that contains the Attachment content. 
     */ 
    @InterfaceAudience.Public 
    public void setAttachment(String name, String contentType, URL contentStreamURL) {
        try { 
            InputStream inputStream = contentStreamURL.openStream();
            setAttachment(name, contentType, inputStream);
        } catch (IOException e) {
            Log.e(Database.TAG, "Error opening stream for url: %s", contentStreamURL);
            throw new RuntimeException(e);
        } 
    } 

https://github.com/couchbase/couchbase-lite-java-core/blob/release/1.2.0/src/main/java/com/couchbase/lite/UnsavedRevision.java

Thanks Nithin

[nolanlawson]

I'm sorry, could you clarify what this issue is about? I'm not really following your explanation.

Passing large binary data between the WebView and native context is indeed an expensive operation. It's one of the reasons I advise against using this plugin for Android, if you can avoid it. IndexedDB will be much more efficient on that platform, because it can directly store Blobs/ArrayBuffers/etc.

PouchDB supports attachments:false and attachments:true, so you can decide for yourself if you want large binary data sent back and forth. You can also just store references yourself.

Couchbase Lite is a pure Java library and doesn't have to contend with a WebView. The performance concerns are completely different from PouchDB's or any WebView-based database. Maybe I'm just not understanding what this issue is about, though.

[NitzDKoder]

@nolanlawson 1) want to use Sqlite as i want to store huge amount of data. 2)Lets say if i use this plugin and 10 mb file is picked at JS level / Java level is stored to SQLITE? 3)How is the content retrieved from sqlite?

Shouldn't be like how i described above?

Basically the plugin needs to enhanced to Store/Retrieve Attachments as URL..make sense?

[NitzDKoder]

@nolanlawson how does attachments:false and attachments:true works in PouchDb ?..pls brief..

[nolanlawson]

WebSQL only supports storing binary data as strings. This plugin adheres to that format in order to maintain compatibility with the spec.

This is not something I really intend to change, because the purpose of this module is to maintain compatibility with the spec. I support I could add support for the user supplying Blobs instead of strings and then handle it automagically, but that might end up creating compatibility issues with SQLite files created in other environments (Node, the browser, etc.).

Basically: don't use this module for storing large amounts of data. It's not going to be efficient. In fact, it will never be efficient, even if I implement the above method, because Cordova requires that you serialize all data sent between the WebView and the native layer, meaning it has to be converted to base64/strings anyway.

The best possible performance for storing huge amounts of data is IndexedDB, because it allows direct Blob storage. (Android supports this as of Lollipop, and iOS will support it in the next version. You can also fall back to base64/strings).

Hope that answers the question, also I realized PouchDB is not relevant to this discussion, so please ignore my comments on that.

[NitzDKoder]

@nolanlawson I get you..all you saying is syntax to store the doc is like websql..under the hood the actual storage differs(Websql/ SQLite).. can you come up with below in sqlite plugin ? and change the syntax of storage with a control flag?

hence solves the actual issue..

1)" Cordova requires that you serialize all data sent between the WebView and the native layer, meaning it has to be converted to base64/strings anyway."

Need not be right it can be just a URL?

like http://developer.couchbase.com/documentation/mobile/current/develop/guides/couchbase-lite/native-api/attachment/index.html

Attachments aren't stored in the database file itself. Instead they are individual files, contained in a directory right next to the database file. Each attachment file has a cryptic name that is actually a SHA-1 digest of its contents.

[nolanlawson]

It's the same problem - file contents still need to be serialized when sent across. Also I think it's out of the scope of this plugin to try to store files outside of the SQLite database.

Keep in mind that CBL is a pure-native plugin whereas this is a Cordova plugin that has to deal with a WebView. The two constraints are very different.

[NitzDKoder]

@nolanlawson No..I have cordova+cbl project with above said design..i use url scheme store and retrieve..everything works well..all you need is path of attachment to render or play or display..