Missing types for generated client

[barbern]

When using npx electric-sql generate to generate the client against the Postgres instance with an electrified table

$ \d t 
                 Table "public.t"
 Column |  Type   | Collation | Nullable | Default 
--------+---------+-----------+----------+---------
 id     | integer |           | not null | 
 s      | text    |           |          | 
Indexes:
    "t_pkey" PRIMARY KEY, btree (id)
Publications:
    "electric_publication"
Triggers:
    postgres_write__delete_generate_shadow_rows BEFORE DELETE ON t FOR EACH ROW EXECUTE FUNCTION electric.update_shadow_row_from_delete___public__t()
    postgres_write__upsert_generate_shadow_rows BEFORE INSERT OR UPDATE ON t FOR EACH ROW EXECUTE FUNCTION electric.create_shadow_row_from_upsert___public__t()
Triggers firing always:
    as_first__save_deleted_rows_to_tombstone_table AFTER DELETE ON t FOR EACH ROW EXECUTE FUNCTION electric.generate_tombstone_entry___public__t()
Triggers firing on replica only:
    satellite_write__save_operation_for_reordering BEFORE INSERT OR UPDATE ON t FOR EACH ROW WHEN (pg_trigger_depth() < 1) EXECUTE FUNCTION electric.reorder_main_op___public__t()

there are some sections with missing types

export const IntFilterSchema: z.ZodType<Prisma.IntFilter> = z.object({
  equals: z.union([ z.number(),z.lazy(() => IntFieldRefInputSchema) ]).optional(),
  in: z.union([ z.number().array(),z.lazy(() => ListIntFieldRefInputSchema) ]).optional(),
  notIn: z.union([ z.number().array(),z.lazy(() => ListIntFieldRefInputSchema) ]).optional(),
  lt: z.union([ z.number(),z.lazy(() => IntFieldRefInputSchema) ]).optional(),
  lte: z.union([ z.number(),z.lazy(() => IntFieldRefInputSchema) ]).optional(),
  gt: z.union([ z.number(),z.lazy(() => IntFieldRefInputSchema) ]).optional(),
  gte: z.union([ z.number(),z.lazy(() => IntFieldRefInputSchema) ]).optional(),
  not: z.union([ z.number(),z.lazy(() => NestedIntFilterSchema) ]).optional(),
}).strict();

(IntFieldRefInputSchema).

If I use zod-prisma-types to generate a Zod schema from the same Prisma schema file I end up with

export const IntFilterSchema: z.ZodType<Prisma.IntFilter> = z.object({
  equals: z.number().optional(),
  in: z.number().array().optional(),
  notIn: z.number().array().optional(),
  lt: z.number().optional(),
  lte: z.number().optional(),
  gt: z.number().optional(),
  gte: z.number().optional(),
  not: z.union([z.number(), z.lazy(() => NestedIntFilterSchema)]).optional(),
}).strict();

I understand a customized version of zod-prisma-types is used but I couldn't track down any changes for it (I'm assuming it's here) that could be responsible.

Where are these missing types supposed to come from?

[alco]

Hi @barbern. Could you share with us which version of the generator you have installed?

[barbern]

Hi @alco. I'm using 1.1.0 of @electric-sql/prisma-generator.

[barbern]

Also 0.6.4 of electric-sql if that's relevant.

[ccapndave]

I have the same issue - here are my errors:

https://gist.github.com/ccapndave/5ce9b142fa4dbd3b6b12ed796f77a576

[kevin-dp]

@barbern @ccapndave could you provide the complete DB schema of electrified tables please as well as the intermediate Prisma schema. Because without extra information i did not manage to reproduce these errors.

[ccapndave]

Sure. Here is our original Prisma migration:

model Vertex {
  // Common fields
  id   String @id
  type String

  createdAt DateTime @default(now()) @db.Timestamp()
  updatedAt DateTime @updatedAt @db.Timestamp()

  edgesIn  Edge[] @relation("toVertex")
  edgesOut Edge[] @relation("fromVertex")

  // Note
  noteHtml String?

  // Link
  linkValidatedEdgeId String?

  // Thread
  threadValidatedEdgeId String?
  threadTitle           String?

  // Person
  personContactId   String?
  personGivenName   String?
  personFamilyName  String?
  personDisplayName String?
}

model Edge {
  // Common fields
  id         String @id
  type       String
  confidence Float

  createdAt DateTime @default(now()) @db.Timestamp()
  updatedAt DateTime @updatedAt @db.Timestamp()

  fromVertex   Vertex @relation("fromVertex", fields: [fromVertexId], references: [id])
  fromVertexId String

  toVertex   Vertex @relation("toVertex", fields: [toVertexId], references: [id])
  toVertexId String

  // HasContact
  hasPersonMatchedProperty String?
}

We used that to generate SQL and then added the electrify statements (I think we also removed the DEFAULT columns because Electric told us they weren't supported):

-- CreateTable
CREATE TABLE "Vertex" (
    "id" TEXT NOT NULL,
    "type" TEXT NOT NULL,
    "createdAt" TIMESTAMP NOT NULL,
    "updatedAt" TIMESTAMP NOT NULL,
    "noteHtml" TEXT,
    "linkValidatedEdgeId" TEXT,
    "threadValidatedEdgeId" TEXT,
    "threadTitle" TEXT,
    "personContactId" TEXT,
    "personGivenName" TEXT,
    "personFamilyName" TEXT,
    "personDisplayName" TEXT,

    CONSTRAINT "Vertex_pkey" PRIMARY KEY ("id")
);

-- CreateTable
CREATE TABLE "Edge" (
    "id" TEXT NOT NULL,
    "type" TEXT NOT NULL,
    "confidence" DOUBLE PRECISION NOT NULL,
    "createdAt" TIMESTAMP NOT NULL,
    "updatedAt" TIMESTAMP NOT NULL,
    "fromVertexId" TEXT NOT NULL,
    "toVertexId" TEXT NOT NULL,
    "hasPersonMatchedProperty" TEXT,

    CONSTRAINT "Edge_pkey" PRIMARY KEY ("id")
);

-- AddForeignKey
ALTER TABLE "Edge" ADD CONSTRAINT "Edge_fromVertexId_fkey" FOREIGN KEY ("fromVertexId") REFERENCES "Vertex"("id") ON DELETE RESTRICT ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "Edge" ADD CONSTRAINT "Edge_toVertexId_fkey" FOREIGN KEY ("toVertexId") REFERENCES "Vertex"("id") ON DELETE RESTRICT ON UPDATE CASCADE;

/* ⚡ Electrify ⚡ */
ALTER TABLE "Vertex" ENABLE ELECTRIC;
ALTER TABLE "Edge" ENABLE ELECTRIC;

Here is the intermediate Prisma schema from node_modules/.prisma/client/schema.prisma:

generator client {
  provider = "prisma-client-js"
}

generator electric {
  provider      = "/Users/dave/Projects/skile/popt/node_modules/@electric-sql/prisma-generator/dist/bin.js"
  output        = "/Users/dave/Projects/skile/popt/src/generated/client"
  relationModel = "false"
}

datasource db {
  provider = "postgresql"
  url      = "postgresql://postgres:password@localhost:65432/electric"
}

model Edge {
  @@map("Edge")
  id                               String   @id
  type                             String
  confidence                       Float /// @zod.custom.use(z.number().or(z.nan()))
  createdAt                        DateTime @db.Timestamp(6)
  updatedAt                        DateTime @db.Timestamp(6)
  fromVertexId                     String
  toVertexId                       String
  hasPersonMatchedProperty         String?
  Vertex_Edge_fromVertexIdToVertex Vertex   @relation("Edge_fromVertexIdToVertex", fields: [fromVertexId], references: [id])
  Vertex_Edge_toVertexIdToVertex   Vertex   @relation("Edge_toVertexIdToVertex", fields: [toVertexId], references: [id])
}

model Vertex {
  @@map("Vertex")
  id                             String   @id
  type                           String
  createdAt                      DateTime @db.Timestamp(6)
  updatedAt                      DateTime @db.Timestamp(6)
  noteHtml                       String?
  linkValidatedEdgeId            String?
  threadValidatedEdgeId          String?
  threadTitle                    String?
  personContactId                String?
  personGivenName                String?
  personFamilyName               String?
  personDisplayName              String?
  Edge_Edge_fromVertexIdToVertex Edge[]   @relation("Edge_fromVertexIdToVertex")
  Edge_Edge_toVertexIdToVertex   Edge[]   @relation("Edge_toVertexIdToVertex")
}

[barbern]

pg_dump --username postgres --dbname electric --table t --schema-only

--
-- PostgreSQL database dump
--

-- Dumped from database version 14.9
-- Dumped by pg_dump version 14.9

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;

SET default_tablespace = '';

SET default_table_access_method = heap;

--
-- Name: t; Type: TABLE; Schema: public; Owner: postgres
--

CREATE TABLE public.t (
    id integer NOT NULL,
    s text
);

ALTER TABLE ONLY public.t REPLICA IDENTITY FULL;

ALTER TABLE public.t OWNER TO postgres;

--
-- Name: t t_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres
--

ALTER TABLE ONLY public.t
    ADD CONSTRAINT t_pkey PRIMARY KEY (id);

--
-- Name: t as_first__save_deleted_rows_to_tombstone_table; Type: TRIGGER; Schema: public; Owner: postgres
--

CREATE TRIGGER as_first__save_deleted_rows_to_tombstone_table AFTER DELETE ON public.t FOR EACH ROW EXECUTE FUNCTION electric.generate_tombstone_entry___public__t();

ALTER TABLE public.t ENABLE ALWAYS TRIGGER as_first__save_deleted_rows_to_tombstone_table;

--
-- Name: t postgres_write__delete_generate_shadow_rows; Type: TRIGGER; Schema: public; Owner: postgres
--

CREATE TRIGGER postgres_write__delete_generate_shadow_rows BEFORE DELETE ON public.t FOR EACH ROW EXECUTE FUNCTION electric.update_shadow_row_from_delete___public__t();

--
-- Name: t postgres_write__upsert_generate_shadow_rows; Type: TRIGGER; Schema: public; Owner: postgres
--

CREATE TRIGGER postgres_write__upsert_generate_shadow_rows BEFORE INSERT OR UPDATE ON public.t FOR EACH ROW EXECUTE FUNCTION electric.create_shadow_row_from_upsert___public__t();

--
-- Name: t satellite_write__save_operation_for_reordering; Type: TRIGGER; Schema: public; Owner: postgres
--

CREATE TRIGGER satellite_write__save_operation_for_reordering BEFORE INSERT OR UPDATE ON public.t FOR EACH ROW WHEN ((pg_trigger_depth() < 1)) EXECUTE FUNCTION electric.reorder_main_op___public__t();

ALTER TABLE public.t ENABLE REPLICA TRIGGER satellite_write__save_operation_for_reordering;

--
-- PostgreSQL database dump complete
--

generator client {
  provider = "prisma-client-js"
}

generator electric {
  provider      = "/private/tmp/client/node_modules/@electric-sql/prisma-generator/dist/bin.js"
  output        = "/private/tmp/client/src/generated/client"
  relationModel = "false"
}

datasource db {
  provider = "postgresql"
  url      = env("PRISMA_DB_URL")
}

model T {
  @@map("t")
  id Int     @id @default(autoincrement())
  s  String?
}

[kevin-dp]

@barbern I cannot reproduce your error on latest version of electric-sql (v0.7.0) and @electric-sql/prisma-generator (v1.1.0). These are the steps i took:

$ npx create-electric-app@latest issue-625
$ cd issue-625
$ yarn backend:up
$ yarn db:psql
issue_625=# CREATE TABLE public.t (
issue_625(#     id integer PRIMARY KEY NOT NULL,
issue_625(#     s text
issue_625(# );
CREATE TABLE
issue_625=# ALTER TABLE public.t ENABLE ELECTRIC;
ELECTRIC ENABLE
issue_625=# \quit
✨  Done in 55.68s.

$ yarn client:generate
yarn run v1.22.19
warning package.json: No license field
$ yarn electric:check && npx electric-sql generate --service http://localhost:5133 --proxy postgresql://prisma:proxy_password@localhost:65432/electric
warning package.json: No license field
$ node ./check-electric-is-running.js
Generating Electric client...
Successfully generated Electric client at: ./src/generated/client
Building migrations...
Successfully built migrations
✨  Done in 4.61s.

The generated Electric client i get does not contain errors. This is how the IntFilterSchema looks like:

export const IntFilterSchema: z.ZodType<Prisma.IntFilter> = z.object({
  equals: z.number().optional(),
  in: z.number().array().optional(),
  notIn: z.number().array().optional(),
  lt: z.number().optional(),
  lte: z.number().optional(),
  gt: z.number().optional(),
  gte: z.number().optional(),
  not: z.union([ z.number(),z.lazy(() => NestedIntFilterSchema) ]).optional(),
}).strict();

[kevin-dp]

@ccapndave When i migrate PG with your DB schema and try to generate an Electric client for it using latest Electric i get another problem regarding the foreign keys which i believe is a problem of the proxy. We will look into that. So best i could do for now is to try reproducing your problem from your intermediate Prisma schema. I ran the latest generator on your intermediate Prisma schema but i do not get missing schemas in the generated client (i do get some type errors however).

[ccapndave]

That's very strange - I can reproduce it reliably here every time I run the generator script. Could you confirm my dependencies are correct (I've put the ones that could possibly be relevant):

    "@capacitor-community/sqlite": "^5.4.2-2",
    "electric-sql": "^0.7.0",
    "wa-sqlite": "https://github.com/rhashimoto/wa-sqlite#buildless",
    "zod": "^3.22.4"
    "@electric-sql/prisma-generator": "^1.1.0",
    "@prisma/client": "5.5.2",

i do get some type errors however

To be clear, type errors are the problem; the generated file does actually work.

[kevin-dp]

That's very strange - I can reproduce it reliably here every time I run the generator script. Could you confirm my dependencies are correct (I've put the ones that could possibly be relevant):

    "@capacitor-community/sqlite": "^5.4.2-2",
    "electric-sql": "^0.7.0",
    "wa-sqlite": "https://github.com/rhashimoto/wa-sqlite#buildless",
    "zod": "^3.22.4"
    "@electric-sql/prisma-generator": "^1.1.0",
    "@prisma/client": "5.5.2",

i do get some type errors however

To be clear, type errors are the problem; the generated file does actually work.

I managed to reproduce the errors about missing schemas like StringFieldRefInputSchema. These errors come from the fact that you are using prisma version 5.5.2 but the generator expects version 4. When using the following versions the generated client for your schema works fine:

"@prisma/client": "4.8.1",
"prisma": "4.8.1"

You should also be using zod version 3.21.1 because of a type problem in the generator we use which occurs because of a change in Zod since version 3.21.2.

[barbern]

@kevin-dp what you've attempted works because npx create-electric-app installs the non-problematic versions of @prisma/client/prisma by default. The manual setup steps suggest it should be possible to install electric-sql and then npx electric-sql generate will just work (it will also fail if @electric-sql/prisma-generator is not installed which electric-sql does not handle).

[kevin-dp]

@barbern Agreed. We are working on updating the docs. I will close the issue as I believe it is solved (with a note to update the docs). Feel free to re-open it in case some problem persists.

[ccapndave]

It works with these versions! Thanks for your help.