lokiFSAdaptor

[annoyingmouse]

Not an issue as such so sorry but I was wondering if there was an example anywhere of using loki with window.requestFileSystem/window.webkitRequestFileSystem...? My aim is for my DB to persists in a Cordova app by saving it to the FileSystem... I'm developing on Chrome so I could save it to PERSISTENT storage on the browser as well. I've looked all over and it seems as though I'll need to write my own adapter but I was wondering if anyone had already done so so I could cast my eye over their solution?

[techfort]

Hi, sorry for not having something in place already (and I meant to for so long!). Funnily enough i created loki to obviate that problem (persistence in cordova), and then never codified the approach. I have a gist though from that first project, it may be outdated/broken so forgive me, but it may give you some inspiration? The gist is here: https://gist.github.com/techfort/a5f35450fb13c771a506 By the way - if you come up with something i'd love to see it because - as i said - it's been on my todo for so long (along with a blog post on how to finally kill off SQLite with LokiJS in a cordova environment)

[annoyingmouse]

Hi @techfort , thanks for getting back to me. I thought that that must be the case. My use case is both Cordova and Chrome so I'm trying to keep the code base the same across platforms so it looks like I need to override the loadDatabase and saveDatabase functions. Cheers, Dom

[annoyingmouse]

Hi again @techfort, I've been playing and wondered if you could cast your eye over my attempts https://github.com/annoyingmouse/lokiFileSystemAdapter/blob/master/lokiFileSystemAdapter.js.

It works a treat if I hard cose things and include it in the lokijs.js file but I figured it would be better to have it as an external adapter, but I'm running into issues... any idea what I'm doing wrong?

Cheers,

Dom

[techfort]

Hi @annoyingmouse , thanks for the feedback! It's going to be a little while before i can set up a cordova project to play with this, have you any log or error I can look at to try and see what is failing? The adapter itself looks good.

[annoyingmouse]

Hi @techfort, it's not a Cordova specific issue as I'm developing in Chrome as well ;-)

I've been playing in the console and, once I include the file, I run:

var fsAdapter = FileSystemAdapter({"base_dir":"testing","file_system":gFileSystem})

gFileSystem is a global reference to the DOMFileSystem Object

var db = new loki("test.db",{"adapter": fsAdapter});
var users = db.addCollection('users', { indices: ['email'] });
var odin = users.insert( { name : 'odin', email: 'odin.soap@lokijs.org', age: 38 } );
var thor = users.insert( { name : 'thor', email : 'thor.soap@lokijs.org', age: 25 } );
var stan = users.insert( { name : 'stan', email : 'stan.soap@lokijs.org', age: 29 } );
var oliver = users.insert( { name : 'oliver', email : 'oliver.soap@lokijs.org', age: 31 } );
var hector = users.insert( { name : 'hector', email : 'hector.soap@lokijs.org', age: 15} );
var achilles = users.insert( { name : 'achilles', email : 'achilles.soap@lokijs.org', age: 31 } );

This all works a treat and I can see the 'users' collection fine... but when I execute this:

db.saveDatabase();

I get this error:

Uncaught TypeError: this.persistenceAdapter.saveDatabase is not a functionmessage: "this.persistenceAdapter.saveDatabase is not a function"stack: (...)get stack: function () { [native code] }set stack: function () { [native code] }arguments: nullcaller: nulllength: 1name: ""prototype: StackTraceSetter__proto__: function Empty() {}<function scope>__proto__: ErrorLoki.saveDatabase @ lokijs.js:953(anonymous function) @ VM1910:2InjectedScript._evaluateOn @ VM1719:883InjectedScript._evaluateAndWrap @ VM1719:816InjectedScript.evaluate @ VM1719:682

I've tried all sorts of ways of getting it to work and re-engineered the adapter about a million times (possibly ;-) ) to no avail and now I'm feeling really silly :-(

[techfort]

hi @annoyingmouse - sorry I made the assumption about cordova based on your first post :) Would you be able to create a gist for the code of your app? it seems a reference to your persistence adapter is lost in the process, it could be a bug in the configureOptions method of Loki, or something trivial in your code.

[annoyingmouse]

Hey @techfort, not to worry. I'm using both you see.

The gist here (https://gist.github.com/annoyingmouse/d2429cd02e32c3ee27ec) though only includes the Chrome FileSystem API.

I'm displaying the data to ensure that everything is gravy and I think you'll see that it is. It's when I come to saveing the DB though that things go odd. I've also put it up on my Google Drive so you can see the console: https://7885122fe90f835fe114babc8ac9da45e70b9b9e.googledrive.com/host/0ByRgQIhodQXfaVdHOElsZTV5c3c/lokiFileSystemAdapter/

Thanks for you help with this by the way.

If I include the same function within lokijs.js like this:

function lokiFileSystemAdapter() {
}
lokiFileSystemAdapter.prototype.loadDatabase = function loadDatabase(dbname, callback) {
    gFileSystem.root.getFile(
        'test_directory/' + dbname,
        {
            create: false
        },
        function(fileEntry){
            fileEntry.file(function(file) {
                var reader = new FileReader();
                reader.onloadend = function(event) {
                    var contents = event.target.result;
                    callback(contents);
                };
                reader.readAsText(file);
            }, function(err){
                callback(new Error(err));
            });
        },
        function(err){
            callback(new Error(err));
        }
    );
};
lokiFileSystemAdapter.prototype.saveDatabase = function saveDatabase(dbname, dbstring, callback) {
    gFileSystem.root.getFile(
        'test_directory/' + dbname,
        {
            create: true
        },
        function(fileEntry) {
            fileEntry.createWriter(
                function(fileWriter) {
                    fileWriter.onwriteend = function() {
                        if (fileWriter.length === 0) {
                            var blob = new Blob(
                                [db.serialize()],
                                {
                                    type: 'text/plain'
                                }
                            );
                            fileWriter.write(blob);
                        }
                    };
                    fileWriter.truncate(0);
                },
                function(err){
                    if(window.cordova){
                        alert("Unable to write file: " + JSON.stringify(err))
                    }else{
                        console.log(err)
                    }
                }
            );
        },
        function(err){
            if(window.cordova){
                alert("Unable to get file: " + JSON.stringify(err))
            }else{
                console.log(err)
            }
        }
    );
};

And then override some stuff further up (line 387 or there abouts) with this:

var defaultPersistence = {
    'NODEJS': 'fs',
    //'BROWSER': 'localStorage',
    //'CORDOVA': 'localStorage'
    'BROWSER': 'lokiFileSystemAdapter',
    'CORDOVA': 'lokiFileSystemAdapter'
},
persistenceMethods = {
    'fs': LokiFsAdapter,
    'localStorage': LokiLocalStorageAdapter,
    'lokiFileSystemAdapter': lokiFileSystemAdapter
};

Then it works as expected. I'd prefer not to dirty your code though ;-)

[techfort]

@annoyingmouse ok i will need a little while to look into this, hopefully tomorrow, hope it's not too late. It does seem like there may be something weird going on in the setting of the adapter, maybe @obeliskos can chime in on this too. I've been stupidly busy with sudden deadlines at work, and the aftermath will go on till thursday...

[obeliskos]

Could it be your instancing of fsAdapter needs 'new' keyword? Maybe try :

var fsAdapter = new FileSystemAdapter({
    "base_dir": "test_directory",
    "file_system": fs
});

[techfort]

@obeliskos good catch. Even if that doesn't resolve the problem, it may resolve other (unexpected) ones :)

[annoyingmouse]

Morning @techfort and @obeliskos, I've updated the file on the Google drive link and I'm afraid it still has the same error :-(

But colour me daft for not clocking the lack of the "new" keyword, that was a good catch @obeliskos!

[annoyingmouse]

So insonmia has some plusses: https://gist.github.com/db0c4e53b71a5dfe18f2.git

It seems as though passing the DOMFileSystem (fs) to the adaptor wasn't making it too happy as it couldn't serialize it (it goes down-and-down forever)! So If I make the fs a global variable it's much happier... Also, taking away the nice AMD stuff seems to have helped as well - which is a shame :-(

So apart from not being able to pass in the fs and it not working as an AMD module I'm reasonably happy.

If you install HTML5 FileSystem Explorer Extended into Chrome:

• Open the page you'll see that it creates the file.
• Refresh and it loads the data from the file rather than creating it and saving it.
• Delete the file using HTML5 FileSystem Explorer Extended and then refresh again and it recreates it and saves it.

So I think I'll close this issue now, thank you for your help. I'm back to using the virgin lokijs.js file and while my adapter isn't perfect it does the job.

[techfort]

That's great news @annoyingmouse - well kind of. I will see if i get the time to put together a cordova/phonegap adapter which hopefully should not suffer from the same issues. I will reference your gist in the 1.3 docs for saving to FS in chrome if that's ok with you. And thanks a lot for the hard work on this!

[annoyingmouse]

Hi @techfort, sorry for the delay in replying. Of course you can reference my gist - and should you want to use my repo for your work then please feel free. And thank you very much for your hard work on LokiJS, it's brilliant!

[techfort]

@annoyingmouse that's great work and i'll definitely reference it! Glad you like Loki!

[cosmith]

Hey, I'm taking advantage of this issue since I'm using Loki in a Cordova app as well. I'm currently using the IndexedDB adapter, which works great, except for the fact that some Samsung devices apparently don't have access to IndexedDB...

I'm getting these errors : Uncaught Error: NOT_FOUND_ERR: DOM IDBDatabase Exception 3 and Uncaught ReferenceError: indexedDB is not defined.

So my question is, should I use this lokiFileSystemAdapter by @annoyingmouse to persist in cases when indexedDB is not an option? Or is there a new recommended way?

Thanks for your help ;)

[techfort]

@cosmith yeah the adapter by @annoyingmouse is the way to go. I still have to find time to develop an "official" one, but - regardless - a cordova-compatible fs adapter is what I would opt for.

[cosmith]

Thanks, I'm writing one based on the code by @annoyingmouse. I'll update here when it works well.

[techfort]

@cosmith if you do manage to "generalize" it i'd be happy to receive a PR for a cordova- fs adapter which is a really high priority at the moment.

[cosmith]

Hey, you can find my code here: https://github.com/cosmith/loki-cordova-fs-adapter .

It's written in ES6 though, not sure what you think about this.

[techfort]

@cosmith that's awesome! I also love that it is a separate project. How much testing have you done on this? I'd love to reference it in the docs and advertise it because it's a feature many have requested.As for ES6, i am slowly but surely migrating to ES6 and i only do ES6 for my new projects, but for as long as there's a transpiled version of the library i don't see why anybody would have a problem.

[cosmith]

I sent it in the Play Store yesterday, haven't seen any issues so far but maybe wait a few days before it's stabilized. I should probably write some tests too...

[techfort]

hey @cosmith ! really? out of sheer curiosity - can you tell the name of the app? i'd like to see LokiJS employed in the wild!

[cosmith]

The app is Truckfly - https://www.truckfly.com/ . Probably not very useful for you unless you're also a truck driver on the side ;) (the website is in french but the app should be in english)

[shivambarsaley]

hey @cosmith I m having a similar use case of using cordova for iPhone app. Can i start over using the cordova plugin ??

[cosmith]

I haven't tested it on iPhone, but you can try and tell me if it works ;)

[ashteya]

@shivambarsaley @cosmith I've tested it on iOS and it's working, thanks for making the adapter.

For anyone who might be interested, I wrote a tutorial on how to use LokiJS with this adapter in Ionic apps: http://gonehybrid.com/how-to-use-lokijs-for-local-storage-in-your-ionic-app/

[techfort]

@ashteya that's awesome, thank you! Just a detail: I'm going to link the adapter created by @cosmith in the official docs on lokijs.org so you can consider that an official component. This said, great article, thanks a lot! And yes - lots of thank you to @cosmith and @shivambarsaley

[cosmith]

Thanks @ashteya! I'm transitioning to React Native now so I made a very simple adapter for React Native's AsyncStorage too, I might open source it when I have a bit of time.

[paulhovey]

I've been wanting to use localforage for it's ability to store on any device and platform, so I created a simple adapter - https://github.com/paulhovey/loki-localforage-adapter . I've tested on Chrome desktop, Android device, and iOS simulator and seems to work everywhere.

[techfort]

hey @paulhovey that's awesome, thank you!