Using zod for data validation

[olehmisar]

Problem

Right now, defining a collection looks like this and is based on type assertions, i.e., the type of data is NOT validated in runtime.

import * as typesaurus from 'typesaurus'
type User = {
  username: string
}
const users = typesaurus.collection<User>("users")

It means that firebase may contain data with a different shape:

{
  users: {
    alice: { username: "Alice" },
    bob: { username: 47 }
  }
}

...and the following code will now be unsound:

import * as typesaurus from 'typesaurus'
const bob = (await typesaurus.get(users, 'bob'))!.data; // type: { username: string }
bob.username // typescript type is inferred to be `string` but the value is number 47
bob.username // 47
bob.username.toLowerCase() // runtime error: bob.username.toLowerCase is not a function

Proposal

I propose using zod validation library for runtime validation of data. The new API will look like this:

import * as typesaurus from 'typesaurus'
const User = z.object({
  username: z.string()
})
// before it was: typesaurus.collection<User>("users")
const users = typesaurus.collection("users", User); // no need for type assertion anymore

const bob = (await typesaurus.get(users, 'bob'))!.data; // runtime error: validation failed, 47 is not a string
const alice = (await typesaurus.get(users, 'alice'))!.data // type: { username: string }
user.username // "Alice"
user.username.toLowerCase() // "alice"

Backwards compatibility

This feature is fully backwards compatible:

1. If zod schema is not provided, require a generic type (as it is now)
2. If zod schema is provided, infer data type from the schema (new API)

[tianhuil]

We're implementing some version of this; it turns out there's some subtlety about how it works (e.g. with transform). On the bright side, we use zod to process away the weird Firestore custom time (we just get back Date). However, there's actually a lot of complexity, depending on how complex you want to make it (e.g. a zod transform for adding a field name firstName + lastName.

[kossnocorp]

I understand the desire to add it and can see how it can be helpful, but I'm not a Zod user and am trying to figure out how to approach it with the new v10 API. If you have ideas and use cases, please share them with me.

I would love to see validation added to Typesaurus, but I don't want to bloat the library. Defining full schema in Zod seems like an overhead, and I would not recommend it to anyone. Zod is also one of many libraries that allow data parsing/validation.

[drewdearing]

While it may add bloat to the project, Zod is starting to become the standard for this kind of thing. Defining the Typesaurus schema entirely in Zod might be overkill for this project, but I think there's definitely room for type validations in the name of making Firestore viable/safe to use.

For example, if I wanted a string field to only be an email, I could use z.string().email() in my schema.

One way to make Zod validation optional for this project is to allow the dev to provide a validate function for each collection type provided.

const userSchema = z.object({
  email: z.string().email(),
  username: z.string(),
});

type User = z.infer<typeof userSchema>;

export const db = schema(($) => ({
  user: $.collection<User>({ validate: (data) => userSchema.parse(data) }),
}));

You can take it one step further, by providing optional packages for Zod integration

import { TypesaurusServerDate, zodSchema } from "@typesaurus/zod";

const userSchema = z.object({
  email: z.string().email(),
  username: z.string(),
  createdAt: TypesaurusServerDate(), //custom zod type to handle the server date validation
});

export const db = zodSchema(($) => ({
  user: $.collection(userSchema), // automatically infer collection type based on schema provided
}));

This does break variable models, however. This kind of takes me back to our conversation in #135, where it could be possible to define collections as a list of variable models

export const db = zodSchema(($) => ({
  user: $.var({
    userA: $.collection(userASchema).sub({
      items: $.collection(userAItemSchema),
    }),
    userB: $.collection(userBSchema).sub({
      items: $.collection(userBItemSchema),
    }),
  }),
}));