canonical / sqlair

Friendly type mapping for SQL databases
Apache License 2.0
17 stars 8 forks source link

Add support for input slices #76

Closed Aflynn50 closed 7 months ago

Aflynn50 commented 1 year ago

This PR adds support for slices in SQLair inputs.

For example, for a named slice type S:

SELECT &P.* FROM p WHERE id IN ($S[:])

Slices of any length can be passed to the query as an argument. All the slice will be included in the query. There are plans to extend this syntax to allow taking slices of slices from within the SQL query itself.

The length of the slice arguments that the user passes are not know when the SQLair statement is prepared. The implementation therefore moves the generation of the SQL from the Prepare phase to the Query phase.

Normally, statements prepared on the database are cached and reused, however, because the SQLair generated SQL of the slice statements depend on the arguments to the query, statements containing slices are not currently cached. They are generated per query and not saved. There are plans cache statements containing slices however it is still a work in progress.

Aflynn50 commented 9 months ago

Update on PR state and notes from in person discussion with @letFunny and @niemeyer

Before the meeting on 03/11/23 this PR only allowed slices in IN clauses in the SQL statement. A SQLair IN clause could contain slices, structs (though not with an asterisk) and maps. The syntax for slice inputs was $S.* where S is a slice type. The slices had a maximum length (set at 8). If the user passed a slice with a shorter length then the inputs to the IN clause would be padded out to 8 with NULL values.

This approach had a number of drawbacks:

A new approach was proposed in the meeting:

As the PR stands at time of writing, the slices are no longer limited to being in the IN clause, the slices can be any length (though the statement prepared on the database is not cached) and the slice is no longer padded out to a fixed length with NULL values.

The new syntax S[:] is implemented (in a naive way) for inserting a full slice into a query.

Supporting taking slices of the slice within the query is a work in progress and may appear either in this PR or in a follow up PR.

Aflynn50 commented 9 months ago

This PR is being put on hold whilst we redesign the pipeline in the expr package. The current abstractions are not suited to the changes needed to add support for slices. This feature is still the priority, but may come in the form of multiple smaller PRs rather than this large one.

The problem with the current pipeline is that the SQL is geenrated at the Prepare stage meaning that it cannot depend on the input arguments. When a slice input is used in a query, the length of the input slice determines the number of argument placeholders in the generated SQL. The pipeline needs to be redesigned to allow for the SQL to depend on the input arguments.