Closed shane-circuithub closed 9 months ago
Is this basically a variant on what we used to do when encoding to JSON?
I don't recall that we ever did actually use JSON for this but we did talk about it. As far as I know PostgreSQL doesn't have a way to reliably convert arbitrary things from JSON so I don't think JSON could really be used for this. There's also no guarantee that the FromJSON
instance on the Haskell side will line up with the JSON that Postgres would convert the array to. At least the text encoding is stable and we can convert things to and from it.
I don't recall that we ever did actually use JSON for this but we did talk about it.
Yea, I had it working but couldn't find anything in the old rel8 branches. Oh well. Would have been around Mar 2021.
As far as I know PostgreSQL doesn't have a way to reliably convert arbitrary things from JSON so I don't think JSON could really be used for this.
I think what I was doing previously was (for non-JSON things): casting to ::text
, then casting to ::jsonb
. This way you're basically always using the text encoding, but building structure in JSON.
There's also no guarantee that the FromJSON instance on the Haskell side will line up with the JSON that Postgres would convert the array to. At least the text encoding is stable and we can convert things to and from it.
FromJSON
isn't involved with this. The TypeInformation
would know that we have our own internal JSON structure that's used by various things, just like your Decoder
. That said, Decoder
is basically what I had before, and I didn't like it because we've now got to maintain two parsers - one for binary and one for text. This seems like a lot of complexity. We're also duplicating all of the parsing that postgresql-simple
does. This all makes me a little uneasy, and I think is what you're alluding to when you say "comes at a high cost"
This is a possible fix to #168. With this we can
catListTable
arbitrarily deep trees ofListTable
s.It comes at a relatively high cost, however.
Currently we represent nested arrays with anonymous records. This works reasonably well, except that we can't extract the field from the anonymous record when we need it (PostgreSQL theoretically suports
.f1
syntax since PG13 but it only works in very limited situations). But it does mean we can decode the results using Hasql's binary decoders, and ordering works how we expect to (array[row(array[9])] < array[row(array[10])]
).What this PR does is instead represent nested arrays as text. To be able to decode this, we need each
DBType
to supply a text parser in addition to a binary decoder. It also means that ordering is no longer intuitive, becausearray[array[9]::text] > array[array[10]::text]
. However, it does mean we can nestcatListTable
s to our heart's content and it will always just work.