simonw / sqlite-utils

Python CLI utility and library for manipulating SQLite databases
https://sqlite-utils.datasette.io
Apache License 2.0
1.62k stars 109 forks source link

Special handling for SQLite column of type `JSON` #579

Open asg017 opened 1 year ago

asg017 commented 1 year ago

sqlite-utils should detect and have specially handling for column with a JSON column. For example:

CREATE TABLE "dogs" (
  id INTEGER PRIMARY KEY,
  name TEXT,
  friends JSON 
);

Automatic Nesting

According to "Nested JSON Values", sqlite-utils will only expand JSON if the --json-cols flag is passed. It looks like it'll try to json.load all text column to test if its JSON, which can get expensive on non-json columns.

Instead, sqlite-utils should be default (ie without the --json-cols flags) do the maybe_json() operation on columns with a declared JSON type. So the above table would expand the "friends" column as expected, withoutthe --json-cols flag:

sqlite-utils dogs.db "select * from dogs" | python -mjson.tool
[
    {
        "id": 1,
        "name": "Cleo",
        "friends": [
            {
                "name": "Pancakes"
            },
            {
                "name": "Bailey"
            }
        ]
    }
]

I'm sure there's other ways sqlite-utils can specially handle JSON columns, so keeping this open while I think of more

rsyring commented 7 months ago

I just created #612, which is very similar to this issue and recommends automatically handling JSON deserialization. I only kept it separate because I was mostly contemplating library usage and this issue seems to be focused on CLI usage.