AlaSQL / alasql

AlaSQL.js - JavaScript SQL database for browser and Node.js. Handles both traditional relational tables and nested JSON data (NoSQL). Export, store, and import data from localStorage, IndexedDB, or Excel.
http://alasql.org
MIT License
6.96k stars 651 forks source link

Add jsonl support #1919

Closed kfigiela closed 1 month ago

kfigiela commented 2 months ago

JSON support is great, but JSONL is very common in streaming cases (e.g. logs).

$ cat test.jsonl
{"a": 1}
{"a": 2}
{"a": 42}

$ alasql "select * from jsonl('./test') where a > 1"
[
  {
    "a": 2
  },
  {
    "a": 42
  }
]
mathiasrw commented 2 months ago

Love it. Good and solid functionality to be added. Thank you.

Any chance you can add a test file that confirms its working (so we detect if we break that later)?

Is .jsonl the right name? What about .ndjson (new line delimited json) or I have even seen .jsons ?

mathiasrw commented 1 month ago

@kfigiela ?

kfigiela commented 1 month ago

@mathiasrw, thanks for reminder. I'll try to address those over the weekend and add tests.

As far as the extension/naming, according to a blog post these are "competing", but practically identical standards. We only parse, so as long as JSON.parse can handle it we should be good. I think, it makes sense to register them both, so either file extension works.

paulrutter commented 1 month ago

https://duckdb.org/2023/03/03/json.html uses ndjson indeed, would be good to rename.

kfigiela commented 1 month ago

On the other hand, Google BigQuery mentions JSONL everywhere. Nevertheless, we can have both aliases working and this is what I have implemented. Otherwise, I addressed the above comments and added some tests.

mathiasrw commented 1 month ago

Just provide all the most used names (alasql.jsonl = alasql.ndjson = alsql.....)

mathiasrw commented 1 month ago

Ahh - you already did :)