Enhancement: Add ability to pass in custom parser.

[austinfrey]

Curious to know if 'WebDB' would be open to a pull request allowing for the ability to pass in a custom file parser. Specifically, I'd like to use smarkt, but it looks like anything that implements a stringify method that can consume JSON and a parse method that spits out JSON, should work correctly. Happy to help if it's a useful idea, no worries if not :)

i.e.

var db = new WebDB('dat:// 494898...', { parser: require('smarkt') })

[RangerMauve]

This would be awesome! I'd want to have it to use with an "encrypted messaging" app I've been planning. If WebDB could parse out the messages automagically, it'd make life a lot easier.

[pfrazee]

I'd be 👍 for this. The function should probably take in metadata about the file in addition to content, so that you can examine the file path and etc.

[RangerMauve]

An ideal API for me would be:

async function serialize(fullURL, data) {

}

async function parse(fullURL, rawData) {

}

Getting data out of the full URL should be easy, and having it async will allow for doing fancy things like offloading to web workers.

[pfrazee]

@RangerMauve makes sense, except that serialize gets called by put() and similar, where the app passes an object to get written to storage

[RangerMauve]

My bad! End of the day so my mind is starting to fail. 😂 At least, that's my excuse

[pfrazee]

Hah all good, I def spent half a second going "oh yeah that's a good idea" and then I remembered

[austinfrey]

@pfrazee couple of Q's.

Initially, I was thinking something simpler in the webDB constructor

class WebDB extends EventEmitter {
    constructor(name, opt = {}) {
        this.parser = opts.parser || JSON
        ...
   }
...

What's the preferred way to pass in the metadata? and what would you be able to do with the metadata above and beyond what you can do already?

[pfrazee]

@austinfrey Passing it into the constructor works, but I don't think you can just replace JSON because you need to pass in metadata.

Here is where it's called, so I'd probably pass in the filepath and co as an archive, so something like:

var record = this.parseFile(await co.readFile(filepath), {filepath, archive: co})

[austinfrey]

@pfrazee I submitted a WIP PR just to highlight what I was thinking. I'm not 100% sure I understand what you mean or why it's necessary. I'm sure there's a good reason but I'm not understanding it :)

For instance, if the parser can take a .yaml file and return JSON, and take JSON and write it as yaml that's all that I would need correct? No other code should really need to change?

[pfrazee]

You need to know the filename so that you can detect the file type by the extension and use the correct parser. You can do that by examining the file content but it's slower.

[austinfrey]

I think I get it now, How I have it currently, you can pass in an alternative parser but then you could only use that. It's better to pass in n number of alternate options and have webDB detect the best choice.

[pfrazee]

TBH I think it'd be better to just do what I suggested. Then the app can handle the parser choice.