Set default interview and document that feature

[simonw]

Following:

• 19

I think it should push every 10s by default if it hasn't had a full batch yet. Should be documented.

[simonw]

Documentation should go on this page: https://dclient.datasette.io/en/latest/inserting.html

[simonw]

Since JSON cannot be streamed but newline-delimited JSON can, I could document how to use jq to turn JSON into newline-delimited JSON. But I can't figure out how to do that, the closest I got was:

stream.py:

import json
import time

i = 0
print('[', flush=True)
while True:
    i += 1
    print(
        json.dumps({"id": i, "name": "abc Name of again again {}".format(i)}) + ',',
        flush=True,
    )
    time.sleep(0.5)

Then:

python stream.py | jq -c --stream 'fromstream(1|truncate_stream(inputs))'

But that output:

{"name":"abc Name of again again 1"}
{"id":2,"name":"abc Name of again again 2"}
{"id":3,"name":"abc Name of again again 3"}
{"id":4,"name":"abc Name of again again 4"}

The first id is missing and I can't figure out why.

[simonw]

I think it's missing because I'm outputting [ on a single line at the start and fromstream is processing a line at a time, perhaps?

But this suggests it's not:

python stream.py | jq --stream -c

Output:

[[0,"id"],1]
[[0,"name"],"abc Name of again again 1"]
[[0,"name"]]
[[1,"id"],2]
[[1,"name"],"abc Name of again again 2"]
[[1,"name"]]
[[2,"id"],3]
[[2,"name"],"abc Name of again again 3"]
[[2,"name"]]
[[3,"id"],4]