[FEATURE]: Allow passing in an array into functions

[some1chan]

Describe what you want

Currently, the default behavior of Drizzle will turn arrays into either its singular element, or a record. This behavior is unintuitive, when you're trying to pass in an array in a PostgreSQL function.

For example, let's say you create a Postgres function called insert_vote...

CREATE OR REPLACE FUNCTION public.insert_vote(
    IN poll_id uuid,
    IN voting_user_id uuid,
    IN choice_ids uuid[]
) RETURNS TABLE (
    status text,
    existing_choices uuid[],
    votes_created uuid[],
    votes_deleted uuid[],
    max_votes smallint
) AS $BODY$
BEGIN
    -- Left empty
END;
$BODY$ LANGUAGE plpgsql;

...and you wish to interact with it with Drizzle.

import "dotenv/config";
import postgres from 'postgres';
import { drizzle, type PostgresJsDatabase } from 'drizzle-orm/postgres-js';

const client = postgres({
    hostname: process.env.POSTGRES_HOST,
    database: process.env.POSTGRES_DB,
    username: process.env.POSTGRES_USER,
    password: process.env.POSTGRES_PASSWORD,
    port: process.env.POSTGRES_PORT
        ? Number(process.env.POSTGRES_PORT)
        : 5432,
});
const db = drizzle(client, { schema, logger: new DefaultLogger() });

// IDs are the same, but this won't matter for our example. These are also ULIDs, and not UUIDs,
// but this also won't matter as Postgres doesn't do any validation on these.
const pollHexId = "018abb841f4ae4257adbba232b34f848";
const userId = "018abb841f4ae4257adbba232b34f848";
const choiceIds = ["018abb841f4ae4257adbba232b34f848"];
const sqlQuery = sql`SELECT public.insert_vote(${pollHexId}::uuid, ${userId}::uuid, ${choiceIds}::uuid[]);`;
console.log((await import('util')).inspect(sqlQuery, undefined, 5));

const insertVote = await db.execute(sqlQuery);
console.log("Finished!");

What you'll see in your logs is this:

// Note that the following sql`` structure is correct
SQL {
  queryChunks: [
    StringChunk { value: [ 'SELECT public.insert_vote(' ] },
    '018abb841f4ae4257adbba232b34f848',
    StringChunk { value: [ '::uuid, ' ] },
    '018abb841f4ae4257adbba232b34f848',
    StringChunk { value: [ '::uuid, ' ] },
    [ '018abb841f4ae4257adbba232b34f848' ],
    StringChunk { value: [ '::uuid[]);' ] }
  ],
  decoder: { mapFromDriverValue: [Function: mapFromDriverValue] },
  shouldInlineParams: false
}
// However, this isn't, as the 3rd parameter becomes a non-array.
Query: SELECT public.insert_vote($1, $2, ($3)); -- params: ["018abb841f4ae4257adbba232b34f848", "018abb841f4ae4257adbba232b34f848, "018abb841f4ae4257adbba232b34f848"]
PostgresError: malformed array literal: "018ac05623d3b7dec2b69bb78c8e0844"

For some reason, our array of our one item becomes our one item. We'll run into the record type if our array has more than one element. You can get the weird record behavior by just adding another uuid into the array.

// ...
const choiceIds = ["018abb841f4ae4257adbba232b34f848", "018abb841f4ae4257adbba232b34f848"];
// ...

// This is using inlineParams() for when I was testing. Error is the same for either-or.
Query: SELECT public.insert_vote('018abb841f4ae4257adbba232b34f848'::uuid, '018abb841f4ae4257adbba232b34f848'::uuid, ('018abb841f4ae4257adbba232b34f848', '018abb841f4ae4257adbba232b34f848')::uuid[]);
PostgresError: cannot cast type record to uuid[]
// Hope is that it would've turned into this:
// SELECT public.insert_vote(
//    '018abb841f4ae4257adbba232b34f848'::uuid,
//    '018abb841f4ae4257adbba232b34f848'::uuid,
//    '{018abb841f4ae4257adbba232b34f848, 018abb841f4ae4257adbba232b34f848}'::uuid[]);

This isn't very useful for us in our use case here.

Since Drizzle doesn't support the syntax around arrays passed to functions, we'll need to do this workaround:

import "dotenv/config";
import postgres from "postgres";
import { drizzle } from "drizzle-orm/postgres-js";
import { DefaultLogger, SQL, sql } from "drizzle-orm";

const client = postgres({
    hostname: process.env.POSTGRES_HOST,
    database: process.env.POSTGRES_DB,
    username: process.env.POSTGRES_USER,
    password: process.env.POSTGRES_PASSWORD,
    port: process.env.POSTGRES_PORT ? Number(process.env.POSTGRES_PORT) : 5432,
});
const db = drizzle(client, { logger: new DefaultLogger() });
const pollHexId = "018abb841f4ae4257adbba232b34f848";
const userId = "018abb841f4ae4257adbba232b34f848";
const choiceIds = [
    "018abb841f4ae4257adbba232b34f848",
    "018abb841f4ae4257adbba232b34f848",
];

const sqlQuery =
    sql`SELECT public.insert_vote(${pollHexId}::uuid, ${userId}::uuid, `.inlineParams();

// Main code starts here
const sanitizedChoiceIds: SQL<unknown>[] = [sql`'{`];
const rawChoiceIds: SQL<unknown>[] = [];
for (const choiceId of choiceIds) {
    // Sanitization happens with sql`${choiceId}`
    const sanitizedChoiceIdChunk = sql`${choiceId}`.queryChunks[1];
    // queryChunks = [
    //  StringChunk{ value: [ '' ] },
    //  '018abb841f4ae4257adbba232b34f848',
    //  'StringChunk { value: [ '' ] }
    // ]
    rawChoiceIds.push(sql.raw(sanitizedChoiceIdChunk!.toString()));
}
sanitizedChoiceIds.push(sql.join(rawChoiceIds, sql`, `));
sanitizedChoiceIds.push(sql`}'::uuid[]`);
const sanitizedChoiceIdsQuery = sql.join(sanitizedChoiceIds);
sqlQuery.append(sanitizedChoiceIdsQuery);
sqlQuery.append(sql`)`);
// ...and now we have our query!

console.log((await import("util")).inspect(sqlQuery, undefined, 8));
const insertVote = await db.execute(sqlQuery);
console.log((await import("util")).inspect(insertVote, undefined, 4));

console.log("Finished!");

This will finally output the right query:

// Ignoring the extremely long SQL query object inspect...
Query: SELECT public.insert_vote('018abb841f4ae4257adbba232b34f848'::uuid, '018abb841f4ae4257adbba232b34f848'::uuid, '{018abb841f4ae4257adbba232b34f848, 018abb841f4ae4257adbba232b34f848}'::uuid[])

Obviously, this isn't ideal. I'd personally love to see support for passing arrays into functions in Drizzle, as this will streamline the above code to simply be our original example.

[pluiedev]

Just ran into this today with a query that looks something like this:

select 
    sum(price * amount) 
from 
    products 
inner join 
    unnest(array[
        ('473907d1-0bbc-4cac-b40d-197cc467d06d', 2),
        ('ca0ad1f8-a038-448f-a322-d6985192a7ec', 1)
    ]::id_and_amount[]) selected 
on products.id = selected.id;

It's really quite inconvenient that we have to do this. Hope this could be more prioritized :+1:

EDIT: Turns out I can pass the array as a JSON array in my case and convert it back into an array of records in the query. That makes things nicer but still definitely not ideal.

[davidchalifoux]

Ran into this issue myself today.

To save time for anyone else, here's a reusable function for handling this. Extend/modify as needed!

Note that I'm casting with ::text[] here. You might need to cast to something else.

import { SQL, sql } from "drizzle-orm";

/**
 * Creates a raw SQL array.
 * Prevents Drizzle's default array behavior.
 * Input is still sanitized.
 *
 * Source: https://www.answeroverflow.com/m/1155016104721272925
 */
export function createRawSqlArray(itemList: string[]) {
  const sanitizedItems: SQL<unknown>[] = [sql`'{`];
  const rawItems: SQL<unknown>[] = [];

  for (const item of itemList) {
    rawItems.push(sql.raw(sql`${item}`.queryChunks[1]!.toString()));
  }

  sanitizedItems.push(sql.join(rawItems, sql`, `));
  sanitizedItems.push(sql`}'::text[]`);

  return sql.join(sanitizedItems);
}

[Duske]

Ran into this issue myself today.

To save time for anyone else, here's a reusable function for handling this. Extend/modify as needed!

Note that I'm casting with ::text[] here. You might need to cast to something else.

import { SQL, sql } from "drizzle-orm";

/**
 * Creates a raw SQL array.
 * Prevents Drizzle's default array behavior.
 * Input is still sanitized.
 *
 * Source: https://www.answeroverflow.com/m/1155016104721272925
 */
export function createRawSqlArray(itemList: string[]) {
  const sanitizedItems: SQL<unknown>[] = [sql`'{`];
  const rawItems: SQL<unknown>[] = [];

  for (const item of itemList) {
    rawItems.push(sql.raw(sql`${item}`.queryChunks[1]!.toString()));
  }

  sanitizedItems.push(sql.join(rawItems, sql`, `));
  sanitizedItems.push(sql`}'::text[]`);

  return sql.join(sanitizedItems);
}

Works like a charm 💯

[GorlikItsMe]

For me this function isn't ideal, I needed to be able to add different casts. In addition, the function did not work with all values. For example postgresql didnt liked me when I wanted to use this array: ["{foo}bar", "foo'bar", "foo\"bar"] After checking, it turned out that the query look like this:

const qs = () =>
  db.execute(
    sql`SELECT unnest(${createRawSqlArray(["{foo}bar", "foo'bar", 'foo"bar'])}) as name`
  );
const q = qs().getQuery();
console.log(q.sql); // SELECT unnest('{{foo}bar, foo'bar, foo"bar}'::text[]) as name

There is not quotes!

After my modification query look like this: SELECT unnest('{"{foo}bar", "foo''bar", "foo\"bar"}'::text[]) as name

My modified function:

import { SQL, sql } from "drizzle-orm";

export function createRawSqlArray(
  itemList: string[],
  castTo: "text" | "integer" | "timestamp"
) {
  const sanitizedItems: SQL<unknown>[] = [sql`'{`];
  const rawItems: SQL<unknown>[] = [];

  for (const item of itemList) {
    // This weird code is to sanitize the input
    const v = item
      .replaceAll(/'/g, "''")
      .replaceAll(/\\/g, "\\\\")
      .replaceAll(/"/g, '\\"');
    const withQuotes= `"${v}"`;
    rawItems.push(sql.raw(sql`${withQuotes}`.queryChunks[1]!.toString()));
  }

  sanitizedItems.push(sql.join(rawItems, sql`, `));
  sanitizedItems.push(sql`}'`);
  if (castTo === "integer") sanitizedItems.push(sql`::integer[]`);
  else if (castTo === "text") sanitizedItems.push(sql`::text[]`);
  else if (castTo === "timestamp") sanitizedItems.push(sql`::timestamp[]`);
  else throw new Error("Invalid castTo");

  return sql.join(sanitizedItems);
}

I am not sure if the sanitisation fully works. I would be grateful for feedback.

[ivan-kleshnin]

Works like a charm 💯

Really? With this approach we get harcoded array items which ruins prepared statements:

-- getting
AND skill = ANY('{PHP, HTML}'::text[]))
-- should be
AND skill = ANY($1))