colinhacks / zod

TypeScript-first schema validation with static type inference
https://zod.dev
MIT License
34.17k stars 1.2k forks source link

[Suggestion] Stricter boolean coerce option #1630

Open pheuter opened 1 year ago

pheuter commented 1 year ago

Since we're currently using Svelte Kit with form actions that make heavy use of the FormData browser api, the new Zod 3.20 release is a very much welcome improvement, specifically the new coercion functionality (since all form post values are serialized as strings over the wire).

One suggestion/request I have is to allow boolean coerce to take a strict option that would check for explicit boolean strings:

Current:

const schema = z.coerce.boolean();
schema.parse("true"); // => true
schema.parse("false"); // => true

Suggestion:

const schema = z.coerce.boolean({ strict: true });
schema.parse("true"); // => true
schema.parse("false"); // => false
schema.parse("123"); // => ZodError

Update: If you're using SvelteKit like me, then worth taking a look at sveltekit-superforms. We're using it now and haven't looked back.

colinhacks commented 1 year ago

Hm :(

This is definitely an important use case but I think the proposed behavior is pretty weird. It's inconsistent with how coercion is handled with other data types. I don't think z.coerce is good for this particular use case.

Seems like a better approach for the example you mentioned is something like this:

z.string().transform(JSON.stringify).pipe(z.boolean());

I've been thinking about ways to make it easier to parse FormData with Zod for a while but struggling. FormData is so astonishingly bad as to be nearly unsalvagable and I hate it 🤷‍♂️

colinhacks commented 1 year ago

Perhaps a zod-formdata lib is called for

import * as zfd from "zod-formdata";

zfd.formdata({
  checkbox: zfd.boolean(),
  field: zfd.string(),
  array: zfd.all(z.string()) // uses `FormData.getAll()`,
  jsonified: zfd.json() // input must be string, Zod calls `JSON.parse` automatically
})

IDK this is a very early concept

pheuter commented 1 year ago

I think the proposed behavior is pretty weird.

I haven't used Zod nearly enough to have a good sense of what's idiomatic, so completely defer to your intuitions here, was mainly trying to convey intent and not implementation.

FormData is so astonishingly bad as to be nearly unsalvagable and I hate it 🤷‍♂️

Couldn't agree more :)

Perhaps a zod-formdata lib is called for

Yeah maybe. There's this one but it doesn't play too great with the latest Zod library, would definitely love nothing more than to have the official library handle FormData better. I anticipate this use case becoming only more popular over time as more frameworks embrace progressive enhancement and built-in web apis.

colinhacks commented 1 year ago

Handling form submissions server-side using FormData is egregious and I hope Remix moves away from it. FormData isn't actually supported by Node or Bun. I don't understand how people handle forms with nested fields or variable-number fields (e.g. "+ Add another") but I would really miss something like RHF's register("children[0].name") that is able to resolve the form data to a nested object.

Anyway if you're using Remix then Zodix is worth looking into: https://github.com/rileytomasek/zodix

pheuter commented 1 year ago

We're actually using Svelte Kit, which has the concept of Form Actions to handle await request.formData() server-side: https://kit.svelte.dev/docs/form-actions

pheuter commented 1 year ago

FormData isn't actually supported by Node or Bun.

Maybe I'm misunderstanding, but the following works for me on Node v19:

// In `node` repl:
> new FormData()
kibertoad commented 1 year ago

It is supported since Node 18.0.0 (see https://developer.mozilla.org/en-US/docs/Web/API/FormData)

colinhacks commented 1 year ago

Ah good catch.

@pheuter want to try to propose an API/behavior that would work for you? I've never gotten very far here. Some open questions:

pheuter commented 1 year ago

Some (not exhaustive) thoughts and ideas:

z.formData() as a z.object() analog:

const schema = z.formData({
  channelType: z.enum(["public", "private"]), // formData.get("channelType")
  channelName: z.string().min(3), // formData.get("channelName")
  members: z.array(z.string()) // formData.getAll("members")
})

schema.parse(await request.formData())

I think the trickiest bit here is coercion. Prob can safely ignore File from the FormDataEntryValue union type and just assume .get() => string | null and .getAll() => string[]. This does assume that Zod can handle coercing "true" and "false" to true and false, and more broadly being able to coerce FormData strings into Zod numbers, enums, etc...

denniscalazans commented 1 year ago

Workaround:

const booleanParamSchema = z.enum(["true", "false"]).transform((value) => value === "true")
egecavusoglu commented 1 year ago

Hey another workaround from me, treats "0" and "false" as false, "1" and "true" astrue. Defaults to false. Works with string enums since I'm using this with query params in my endpoint. eg. api.com/property=1

// Coerces a string to true if it's "true" or "1", false if "false" or "0"
export const coerceBoolean = z
  .enum(["0", "1", "true", "false"])
  .catch("false")
  .transform((value) => value == "true" || value == "1");
ryanflorence commented 1 year ago

Ignoring any strong opinions about FormData, supporting it would mean you'd get support for validating URLSearchParams as well, since they are nearly identical interfaces with all the same capabilities and constraints for zod.

and I hope Remix moves away from it

Won't happen. It's the JavaScript object representation of the HTML <form> interface. Remix supports HTML. That means being able to submit forms with client side JavaScript but also letting the browser make the FormData request the way the web has always worked.

<3 zod :)

pheuter commented 1 year ago

Prob safe to say same for SvelteKit.

ryanflorence commented 1 year ago

I'm not sure this needs to be zod's job though. Something else can parse the form data and then hand it to zod. FormData can get complicated with multipart.

WORMSS commented 1 year ago

Workaround:

const booleanParamSchema = z.enum(["true", "false"]).transform((value) => value === "true")

I was just writing this very thing, but with the addition of True False and TRUE FALSE because, there are heathens around us.

exsesx commented 1 year ago

+1 for a strict boolean check. Workarounds are okay, but having a pretty one-liner for checking "true" and "false" booleans without transforms would be great.

noveogroup-amorgunov commented 1 year ago

hey, any plans about this suggestion? It will be very usefull for parsing env variables:

process.env.IS_ENABLED = "false";

// now
const schema = z.coerce.boolean()
const result = schema.parse(process.env.IS_ENABLED) // true

// want
const schema = z.coerce.boolean({ strict: true })
const result = schema.parse(process.env.IS_ENABLED) // false
oljimenez commented 1 year ago

Workaround:

z.preprocess((v) => z.enum(['true', 'false']).transform((v) => JSON.parse(v)).catch(v).parse(v),z.boolean())
lzehrung commented 1 year ago

Here's another workaround that's case-insensitive but expects true/false explicitly or adds an error. (@oljimenez when I paste yours the .catch(v) doesn't seem to exist - at least in my version of zod)

.optional() can be called from this as needed. Hopefully the params passed to addIssue make sense, first time using them 🙂.

export const booleanStrict = z.string().transform<boolean>((v, ctx) => {
  v = v.toLowerCase();
  switch (v) {
    case 'true':
      return true;
    case 'false':
      return false;
    default:
      ctx.addIssue({
        code: z.ZodIssueCode.invalid_type,
        expected: z.ZodParsedType.boolean,
        received: z.ZodParsedType.string,
        message: 'Expected "true" or "false"',
      });
      return false;
  }
});
oljimenez commented 1 year ago

Here's another workaround that's case-insensitive but expects true/false explicitly or adds an error. (@oljimenez when I paste yours the .catch(v) doesn't seem to exist - at least in my version of zod)

.optional() can be called from this as needed. Hopefully the params passed to addIssue make sense, first time using them 🙂.

export const booleanStrict = z.string().transform<boolean>((v, ctx) => {
  v = v.toLowerCase();
  switch (v) {
    case 'true':
      return true;
    case 'false':
      return false;
    default:
      ctx.addIssue({
        code: z.ZodIssueCode.invalid_type,
        expected: z.ZodParsedType.boolean,
        received: z.ZodParsedType.string,
        message: 'Expected "true" or "false"',
      });
      return false;
  }
});

https://zod.dev/?id=catch

chambaz commented 1 year ago

+1 for strict option

ronymmoura commented 1 year ago

+1 for strict boolean option. If I'm coercing, it should at least have an option to truly transform the value

casder-succ commented 1 year ago

+1 for strict option. it would be very helpful

syahmifauzi commented 1 year ago

My workaround. Since I'm validating both on frontend and backend using the same schema, this works for me:

  const booleanSchema = z
    .union([z.boolean(), z.literal('true'), z.literal('false')])
    .transform((value) => value === true || value === 'true')
joshuat commented 1 year ago

Coercion is a difficult thing to nail down for all use cases, but I really think "false" when being coerced into a boolean should be considered false - this is primarily driven by the AJV coercion table

While AJV is far from the first or only library to support schema validation with coercion it is certainly the most widely adopted and I think for primitive coercion they really have hit the mark.

LarsOlt commented 1 year ago
function envBoolean(params: { optional: boolean; defaultValue: boolean }) {
    type BoolEnum = ['true', 'false'];
    let variable: z.ZodCatch<z.ZodEnum<BoolEnum>> | z.ZodEnum<BoolEnum>;

    if (params.optional) {
        // if undefined assign the defaultValue
        variable = z.enum(['true', 'false']).catch(params.defaultValue ? 'true' : 'false');
    } else {
        // not optional so "true" or "false" is enforced
        variable = z.enum(['true', 'false']);
    }

    // convert string to bool
    return variable.transform((v) => v === 'true');
}

This works for me. I am using this to parse .env

const envSchema= z.object({
  IS_COOL: envBoolean({ optional: true, defaultValue: true })
})

export const env = envSchema.parse(process.env);
jlandowner commented 1 year ago

Finally I reached this issue since I tried to parse environment variables.

As @colinhacks mentioned in https://github.com/colinhacks/zod/issues/1630#issuecomment-1338124864, I agree that strict option is inconsistent with other data types. It is clear thatz.boolean() represents Boolean().

However we also need a utility to transform string to boolean in many use cases. So I suggest a new string transformation: z.string().boolean()

I think the transforming specification is good to be the same with Go standard library's strconv.ParseBool()

https://pkg.go.dev/strconv#ParseBool

ParseBool returns the boolean value represented by the string. It accepts 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False. Any other value returns an error.

elie222 commented 11 months ago

Another option for people to use:

z
    .enum(["true", "false"])
    .nullish()
    .transform((v) => v === "true")
Elalfy74 commented 11 months ago

My solution work for all ['true', 'false', true, false] also returns boolean type

const booleans = ['true', 'false', true, false];
const BooleanOrBooleanStringSchema = z
  .any()
  .refine((val) => booleans.includes(val), { message: 'must be boolean' })
  .transform((val) => {
    if (val === 'true' || val === true) return true;
    return false;
  });
type BooleanOrBooleanStringType = z.infer<typeof BooleanOrBooleanStringSchema>;  //boolean
markomitranic commented 10 months ago

Another option for people to use:

z
    .enum(["true", "false"])
    .nullish()
    .transform((v) => v === "true")

I've found this onefrom @elie222 the most useful for my use case, as it enforces strict-ish string values. I use it to coerce booleans during env var parsing and validations. For reference here is how it works:

expect(coercer.parse("false")).toBe(false);
expect(coercer.parse("true")).toBe(true);
expect(() => coercer.parse(undefined)).toThrow();

For future readers, there is also some degree of advice about this over at https://env.t3.gg/docs/recipes#booleans one of the examples has a broader brain:

const coercer = z.string().nullish().transform((s) => !!s && s !== "false" && s !== "0");

expect(coercer.parse("false")).toBe(false);
expect(coercer.parse("true")).toBe(true);
expect(coercer.parse("asdf")).toBe(true);
expect(coercer.parse("1")).toBe(true);
expect(coercer.parse("0")).toBe(false);
expect(coercer.parse(undefined)).toBe(false);
expect(coercer.parse(null)).toBe(false);

Edit: url typo fix

JacobWeisenburger commented 10 months ago

Perhaps this would be helpful:

const toBool = z.union( [
    z.enum( [ 'false', '0' ] ).transform( () => false ),
    z.boolean(),
    z.string(),
    z.number(),
] ).pipe( z.coerce.boolean() )

console.log( toBool.parse( true ) )    // true
console.log( toBool.parse( 'true' ) )  // true
console.log( toBool.parse( 'foo' ) )   // true
console.log( toBool.parse( '1' ) )     // true 
console.log( toBool.parse( 1 ) )       // true

console.log( toBool.parse( false ) )   // false
console.log( toBool.parse( 'false' ) ) // false
console.log( toBool.parse( '0' ) )     // false
console.log( toBool.parse( 0 ) )       // false

console.log( toBool.safeParse( undefined ).success ) // false
console.log( toBool.safeParse( null ).success )      // false

I find it very helpful when dealing with z.coerce to explicitly define the types that I want to allow. This way, I can be sure that I am getting the behavior that I expect.

If you found my answer helpful, please consider supporting me. Even a small amount is greatly appreciated. Thanks friend! 🙏 https://github.com/sponsors/JacobWeisenburger

beetroop commented 8 months ago

Yet another vote for strictmode option. For those who dip in and out of zod and use it for query string validation etc, the current behaviour is pretty non-intuitive.

ThomasClague commented 8 months ago

+1 for strict mode. My first use of this package was parsing query strings

hamzasial1911 commented 8 months ago

You can also do it like this. This will also check the explicit boolean strings:

Current:

const schema = z.coerce.boolean();
schema.parse("true"); // => true
schema.parse("false"); // => true

Suggestion:

const schema = z.coerce.string().transform((val) => val === "true")
schema.parse("true"); // => true
schema.parse("false"); // => false
hugocruzlfc commented 2 months ago

Any update on this issue of z.coerce.boolean("false) // ‘true’.

Any update?

twiddler commented 1 month ago

For anyone who is wondering why this happens:

  1. https://github.com/colinhacks/zod/blob/1f4f0dacf313a2dba45563d78171e6f016096925/src/types.ts#L1753
  2. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Boolean#boolean_coercion

    The empty string "" turns into false; other strings turn into true.