go-playground / validator

:100:Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving
MIT License
16.14k stars 1.29k forks source link

Weird panic - Undefined validation function 'required' on field XXX #1169

Open kubeplusplus opened 10 months ago

kubeplusplus commented 10 months ago

Package version eg. v9, v10: v10.15.4

image

Issue, Question or Enhancement:

I am developing a consumer that is using nats.io. In the consumer, I am using the validator to validate my message before process next step but got a weird bug panic: Undefined validation function 'required' on field 'Subject'. Definitely the validator must have the built-in required function so something went wrong

Code sample, to showcase or reproduce:

import govalidator "github.com/go-playground/validator/v10"

func UseConsumer() func(msg *nats.Msg) {
    v := govalidator.New(govalidator.WithRequiredStructEnabled())

    // some init tasks

    func(msg *nats.Msg) {
        event := natsMsgToEvent(msg)
        if err := v.Struct(event); err != nil {
            subscriber.logger.Errorw(err.Error(), "nats_msg", utils.Stringify(msg))
            if err := msg.Nak(); err != nil {
                // it's important to log entire event here because we can trace it in log
                subscriber.logger.Errorw(ErrSubNakFail.Error(), "nats_msg", utils.Stringify(msg))
            }
            return
        }
        // my logic here
    }
}

And the error I got

panic: Undefined validation function 'required' on field 'Subject'

goroutine 47 [running]:
github.com/go-playground/validator/v10.(*Validate).parseFieldTagsRecursive(0xc00089ca80, {0x13c1c96?, 0x167d31d?}, {0x13c1c75, 0x7}, {0x0, 0x0}, 0x0)
    /app/vendor/github.com/go-playground/validator/v10/cache.go:294 +0x9d9
github.com/go-playground/validator/v10.(*Validate).extractStructCache(0xc00089ca80, {0x15d6f40?, 0xc0187c79e0?, 0x0?}, {0x137eb47, 0x5})
    /app/vendor/github.com/go-playground/validator/v10/cache.go:155 +0x569
github.com/go-playground/validator/v10.(*validate).validateStruct(0xc01feffb00, {0x18d7060, 0x2cc0ec0}, {0x14a9840?, 0xc0187c79e0?, 0xffffffffffffffff?}, {0x15d6f40?, 0xc0187c79e0?, 0xc0025f4230?}, {0x18eeb48, ...}, ...)
    /app/vendor/github.com/go-playground/validator/v10/validator.go:37 +0x196
github.com/go-playground/validator/v10.(*Validate).StructCtx(0xc00089ca80, {0x18d7060, 0x2cc0ec0}, {0x14a9840?, 0xc0187c79e0?})
    /app/vendor/github.com/go-playground/validator/v10/validator_instance.go:401 +0x5ac
github.com/go-playground/validator/v10.(*Validate).Struct(0xc002b7f500?, {0x14a9840?, 0xc0187c79e0?})
    /app/vendor/github.com/go-playground/validator/v10/validator_instance.go:370 +0x31
github.com/scrapnode/kanthor/infrastructure/streaming.(*NatsSubscriberPushing).Sub.func1(0xc0011d40d0?)
    /app/infrastructure/streaming/nats_subscriber_pushing.go:105 +0x6d
github.com/nats-io/nats%2ego.(*Conn).waitForMsgs(0xc0003cd800, 0xc0011d40d0)
    /app/vendor/github.com/nats-io/nats.go/nats.go:3001 +0x3f2
created by github.com/nats-io/nats%2ego.(*Conn).subscribeLocked in goroutine 30
    /app/vendor/github.com/nats-io/nats.go/nats.go:4232 +0x3c8

After take a look in the source code, I found where the panic was threw and put some test code by myself and test again

image

There is the error out I got after added my debug code. You can see the has variable is false but in the map v.validations we have required key

2023/09/17 13:45:54  false | map[alpha:{fn:0x699860 runValidatinOnNil:false} alphanum:{fn:0x699860 runValidatinOnNil:false} alphanumunicode:{fn:0x699860 runValidatinOnNil:false} alphaunicode:{fn:0x699860 runValidatinOnNil:false} ascii:{fn:0x699860 runValidatinOnNil:false} base64:{fn:0x699860 runValidatinOnNil:false} base64rawurl:{fn:0x699860 runValidatinOnNil:false} base64url:{fn:0x699860 runValidatinOnNil:false} bcp47_language_tag:{fn:0x699860 runValidatinOnNil:false} bic:{fn:0x699860 runValidatinOnNil:false} boolean:{fn:0x699860 runValidatinOnNil:false} btc_addr:{fn:0x699860 runValidatinOnNil:false} btc_addr_bech32:{fn:0x699860 runValidatinOnNil:false} cidr:{fn:0x699860 runValidatinOnNil:false} cidrv4:{fn:0x699860 runValidatinOnNil:false} cidrv6:{fn:0x699860 runValidatinOnNil:false} contains:{fn:0x699860 runValidatinOnNil:false} containsany:{fn:0x699860 runValidatinOnNil:false} containsrune:{fn:0x699860 runValidatinOnNil:false} credit_card:{fn:0x699860 runValidatinOnNil:false} cron:{fn:0x699860 runValidatinOnNil:false} cve:{fn:0x699860 runValidatinOnNil:false} datauri:{fn:0x699860 runValidatinOnNil:false} datetime:{fn:0x699860 runValidatinOnNil:false} dir:{fn:0x699860 runValidatinOnNil:false} dirpath:{fn:0x699860 runValidatinOnNil:false} dns_rfc1035_label:{fn:0x699860 runValidatinOnNil:false} e164:{fn:0x699860 runValidatinOnNil:false} email:{fn:0x699860 runValidatinOnNil:false} endsnotwith:{fn:0x699860 runValidatinOnNil:false} endswith:{fn:0x699860 runValidatinOnNil:false} eq:{fn:0x699860 runValidatinOnNil:false} eq_ignore_case:{fn:0x699860 runValidatinOnNil:false} eqcsfield:{fn:0x699860 runValidatinOnNil:false} eqfield:{fn:0x699860 runValidatinOnNil:false} eth_addr:{fn:0x699860 runValidatinOnNil:false} eth_addr_checksum:{fn:0x699860 runValidatinOnNil:false} excluded_if:{fn:0x6998c0 runValidatinOnNil:true} excluded_unless:{fn:0x6998c0 runValidatinOnNil:true} excluded_with:{fn:0x6998c0 runValidatinOnNil:true} excluded_with_all:{fn:0x6998c0 runValidatinOnNil:true} excluded_without:{fn:0x6998c0 runValidatinOnNil:true} excluded_without_all:{fn:0x6998c0 runValidatinOnNil:true} excludes:{fn:0x699860 runValidatinOnNil:false} excludesall:{fn:0x699860 runValidatinOnNil:false} excludesrune:{fn:0x699860 runValidatinOnNil:false} fieldcontains:{fn:0x699860 runValidatinOnNil:false} fieldexcludes:{fn:0x699860 runValidatinOnNil:false} file:{fn:0x699860 runValidatinOnNil:false} filepath:{fn:0x699860 runValidatinOnNil:false} fqdn:{fn:0x699860 runValidatinOnNil:false} gt:{fn:0x699860 runValidatinOnNil:false} gtcsfield:{fn:0x699860 runValidatinOnNil:false} gte:{fn:0x699860 runValidatinOnNil:false} gtecsfield:{fn:0x699860 runValidatinOnNil:false} gtefield:{fn:0x699860 runValidatinOnNil:false} gtfield:{fn:0x699860 runValidatinOnNil:false} hexadecimal:{fn:0x699860 runValidatinOnNil:false} hexcolor:{fn:0x699860 runValidatinOnNil:false} hostname:{fn:0x699860 runValidatinOnNil:false} hostname_port:{fn:0x699860 runValidatinOnNil:false} hostname_rfc1123:{fn:0x699860 runValidatinOnNil:false} hsl:{fn:0x699860 runValidatinOnNil:false} hsla:{fn:0x699860 runValidatinOnNil:false} html:{fn:0x699860 runValidatinOnNil:false} html_encoded:{fn:0x699860 runValidatinOnNil:false} http_url:{fn:0x699860 runValidatinOnNil:false} image:{fn:0x699860 runValidatinOnNil:false} ip:{fn:0x699860 runValidatinOnNil:false} ip4_addr:{fn:0x699860 runValidatinOnNil:false} ip6_addr:{fn:0x699860 runValidatinOnNil:false} ip_addr:{fn:0x699860 runValidatinOnNil:false} ipv4:{fn:0x699860 runValidatinOnNil:false} ipv6:{fn:0x699860 runValidatinOnNil:false} isbn:{fn:0x699860 runValidatinOnNil:false} isbn10:{fn:0x699860 runValidatinOnNil:false} isbn13:{fn:0x699860 runValidatinOnNil:false} isdefault:{fn:0x699860 runValidatinOnNil:false} iso3166_1_alpha2:{fn:0x699860 runValidatinOnNil:false} iso3166_1_alpha3:{fn:0x699860 runValidatinOnNil:false} iso3166_1_alpha_numeric:{fn:0x699860 runValidatinOnNil:false} iso3166_2:{fn:0x699860 runValidatinOnNil:false} iso4217:{fn:0x699860 runValidatinOnNil:false} iso4217_numeric:{fn:0x699860 runValidatinOnNil:false} json:{fn:0x699860 runValidatinOnNil:false} jwt:{fn:0x699860 runValidatinOnNil:false} latitude:{fn:0x699860 runValidatinOnNil:false} len:{fn:0x699860 runValidatinOnNil:false} longitude:{fn:0x699860 runValidatinOnNil:false} lowercase:{fn:0x699860 runValidatinOnNil:false} lt:{fn:0x699860 runValidatinOnNil:false} ltcsfield:{fn:0x699860 runValidatinOnNil:false} lte:{fn:0x699860 runValidatinOnNil:false} ltecsfield:{fn:0x699860 runValidatinOnNil:false} ltefield:{fn:0x699860 runValidatinOnNil:false} ltfield:{fn:0x699860 runValidatinOnNil:false} luhn_checksum:{fn:0x699860 runValidatinOnNil:false} mac:{fn:0x699860 runValidatinOnNil:false} max:{fn:0x699860 runValidatinOnNil:false} md4:{fn:0x699860 runValidatinOnNil:false} md5:{fn:0x699860 runValidatinOnNil:false} min:{fn:0x699860 runValidatinOnNil:false} mongodb:{fn:0x699860 runValidatinOnNil:false} multibyte:{fn:0x699860 runValidatinOnNil:false} ne:{fn:0x699860 runValidatinOnNil:false} ne_ignore_case:{fn:0x699860 runValidatinOnNil:false} necsfield:{fn:0x699860 runValidatinOnNil:false} nefield:{fn:0x699860 runValidatinOnNil:false} number:{fn:0x699860 runValidatinOnNil:false} numeric:{fn:0x699860 runValidatinOnNil:false} oneof:{fn:0x699860 runValidatinOnNil:false} postcode_iso3166_alpha2:{fn:0x699860 runValidatinOnNil:false} postcode_iso3166_alpha2_field:{fn:0x699860 runValidatinOnNil:false} printascii:{fn:0x699860 runValidatinOnNil:false} required:{fn:0x699860 runValidatinOnNil:false} required_if:{fn:0x6998c0 runValidatinOnNil:true} required_unless:{fn:0x6998c0 runValidatinOnNil:true} required_with:{fn:0x6998c0 runValidatinOnNil:true} required_with_all:{fn:0x6998c0 runValidatinOnNil:true} required_without:{fn:0x6998c0 runValidatinOnNil:true} required_without_all:{fn:0x6998c0 runValidatinOnNil:true} rgb:{fn:0x699860 runValidatinOnNil:false} rgba:{fn:0x699860 runValidatinOnNil:false} ripemd128:{fn:0x699860 runValidatinOnNil:false} ripemd160:{fn:0x699860 runValidatinOnNil:false} semver:{fn:0x699860 runValidatinOnNil:false} sha256:{fn:0x699860 runValidatinOnNil:false} sha384:{fn:0x699860 runValidatinOnNil:false} sha512:{fn:0x699860 runValidatinOnNil:false} skip_unless:{fn:0x6998c0 runValidatinOnNil:true} spicedb:{fn:0x699860 runValidatinOnNil:false} ssn:{fn:0x699860 runValidatinOnNil:false} startsnotwith:{fn:0x699860 runValidatinOnNil:false} startswith:{fn:0x699860 runValidatinOnNil:false} tcp4_addr:{fn:0x699860 runValidatinOnNil:false} tcp6_addr:{fn:0x699860 runValidatinOnNil:false} tcp_addr:{fn:0x699860 runValidatinOnNil:false} tiger128:{fn:0x699860 runValidatinOnNil:false} tiger160:{fn:0x699860 runValidatinOnNil:false} tiger192:{fn:0x699860 runValidatinOnNil:false} timezone:{fn:0x699860 runValidatinOnNil:false} udp4_addr:{fn:0x699860 runValidatinOnNil:false} udp6_addr:{fn:0x699860 runValidatinOnNil:false} udp_addr:{fn:0x699860 runValidatinOnNil:false} ulid:{fn:0x699860 runValidatinOnNil:false} unique:{fn:0x699860 runValidatinOnNil:false} unix_addr:{fn:0x699860 runValidatinOnNil:false} uppercase:{fn:0x699860 runValidatinOnNil:false} uri:{fn:0x699860 runValidatinOnNil:false} url:{fn:0x699860 runValidatinOnNil:false} url_encoded:{fn:0x699860 runValidatinOnNil:false} urn_rfc2141:{fn:0x699860 runValidatinOnNil:false} uuid:{fn:0x699860 runValidatinOnNil:false} uuid3:{fn:0x699860 runValidatinOnNil:false} uuid3_rfc4122:{fn:0x699860 runValidatinOnNil:false} uuid4:{fn:0x699860 runValidatinOnNil:false} uuid4_rfc4122:{fn:0x699860 runValidatinOnNil:false} uuid5:{fn:0x699860 runValidatinOnNil:false} uuid5_rfc4122:{fn:0x699860 runValidatinOnNil:false} uuid_rfc4122:{fn:0x699860 runValidatinOnNil:false}]
panic: Undefined validation function 'required' on field 'Subject'

goroutine 29 [running]:
github.com/go-playground/validator/v10.(*Validate).parseFieldTagsRecursive(0xc00041ad90, {0x13c1c91?, 0x167d2de?}, {0x13c1c70, 0x7}, {0x0, 0x0}, 0x0)
    /app/vendor/github.com/go-playground/validator/v10/cache.go:297 +0xade
github.com/go-playground/validator/v10.(*Validate).extractStructCache(0xc00041ad90, {0x15d6f00?, 0xc0224f43c0?, 0xc000f81ba8?}, {0x137eb47, 0x5})
    /app/vendor/github.com/go-playground/validator/v10/cache.go:156 +0x569
github.com/go-playground/validator/v10.(*validate).validateStruct(0xc01e0f0000, {0x18d7080, 0x2cc0ee0}, {0x14a9800?, 0xc0224f43c0?, 0x1685ccb?}, {0x15d6f00?, 0xc0224f43c0?, 0x19?}, {0x18eece8, ...}, ...)
    /app/vendor/github.com/go-playground/validator/v10/validator.go:37 +0x196
github.com/go-playground/validator/v10.(*Validate).StructCtx(0xc00041ad90, {0x18d7080, 0x2cc0ee0}, {0x14a9800?, 0xc0224f43c0?})
    /app/vendor/github.com/go-playground/validator/v10/validator_instance.go:393 +0x445
github.com/go-playground/validator/v10.(*Validate).Struct(0xc00930ae70?, {0x14a9800?, 0xc0224f43c0?})
    /app/vendor/github.com/go-playground/validator/v10/validator_instance.go:366 +0x31
github.com/scrapnode/kanthor/infrastructure/streaming.(*NatsSubscriberPushing).Sub.func1(0xc0010b2750?)
    /app/infrastructure/streaming/nats_subscriber_pushing.go:105 +0x6d
github.com/nats-io/nats%2ego.(*Conn).waitForMsgs(0xc0006f5100, 0xc0010b2750)
    /app/vendor/github.com/nats-io/nats.go/nats.go:3001 +0x3f2
created by github.com/nats-io/nats%2ego.(*Conn).subscribeLocked in goroutine 26
    /app/vendor/github.com/nats-io/nats.go/nats.go:4232 +0x3c8
deankarn commented 10 months ago

Can you please post what event struct looks like and the validations on it. I can’t really help without it.

MysteriousPotato commented 10 months ago

Is it possible the required tag contains an invisible char? I'd try deleting the tag and rewriting it just in case.

Ex.:

func TestNastyInvisibleChar(t *testing.T) {
    v := validator.New(validator.WithRequiredStructEnabled())

    type myTypeWithInvisibleChar struct {
                // The "required" tag here contains an invisible char at the end
        V string `validate:"required­"`
    }

    type myType struct {
        V string `validate:"required"`
    }

    // All good
    if err := v.Struct(myType{"potato"}); err != nil {
        t.Fatal(err)
    }

    // Panics with "Undefined validation function 'required­' on field 'V'"
    if err := v.Struct(myTypeWithInvisibleChar{"potato"}); err != nil {
        t.Fatal(err)
    }
}
kubeplusplus commented 10 months ago

@deankarn sorry about missing the event, here it is


type Event struct {
    Subject string `json:"subject" validate:"required"`
    AppId   string `json:"app_id" validate:"required"`
    Type    string `json:"type" validate:"required"`

    Id       string            `json:"id" validate:"required"`
    Data     []byte            `json:"data" validate:"required"`
    Metadata map[string]string `json:"metadata"`
}

And an example of actual data that was dumped when the bug was happen

{
  "subject": "kanthor.default.msg.app_2VXx8Qpkd9MAaKhYhEwnBLSliYJ.testing.traffic.stability",
  "app_id": "app_2VXx8Qpkd9MAaKhYhEwnBLSliYJ",
  "type": "testing.traffic.stability",
  "id": "msg_2VXxhrM959iCT4pLFJI7TlOojZC",
  "data": "eyJpZCI6Im1zZ18yVlh4aHJNOTU5aUNUNHBMRkpJN1RsT29qWkMiLCJ0aW1lc3RhbXAiOjE2OTQ5OTYzNDc4MjUsImJ1Y2tldCI6IjIwMjMwOTE4IiwiYXR0X2lkIjoiYXR0XzJWWHhocVROZzdaMEp1cmlpdnk3MFNya0xrSiIsInRpZXIiOiJkZWZhdWx0IiwiYXBwX2lkIjoiYXBwXzJWWHg4UXBrZDlNQWFLaFloRXduQkxTbGlZSiIsInR5cGUiOiJ0ZXN0aW5nLnRyYWZmaWMuc3RhYmlsaXR5IiwibWV0YWRhdGEiOnsia2FudGhvci5pZGVtcG90ZW5jeV9rZXkiOiI0MmY1NjUwNi1lN2ZmLTQ3MGYtODBhMS01Y2Q2M2Q4MTAyNjMifSwiaGVhZGVycyI6eyJIZWFkZXIiOnsiWC1DbGllbnQiOlsiazYuaW8iXX19LCJib2R5IjoiZXlKMWMyVnlibUZ0WlNJNkluVmZkbXhpWTNwNmEzRm5jVzF2Ym1weWIyZGhZMmRyZDNKdGFuTnNaRUJyWVc1MGFHOXliR0ZpY3k1amIyMGlmUT09In0=",
  "metadata": {}
}

Some other information that may help

kubeplusplus commented 10 months ago

Is it possible the required tag contains an invisible char? I'd try deleting the tag and rewriting it just in case.

Ex.:

func TestNastyInvisibleChar(t *testing.T) {
  v := validator.New(validator.WithRequiredStructEnabled())

  type myTypeWithInvisibleChar struct {
                // The "required" tag here contains an invisible char at the end
      V string `validate:"required­"`
  }

  type myType struct {
      V string `validate:"required"`
  }

  // All good
  if err := v.Struct(myType{"potato"}); err != nil {
      t.Fatal(err)
  }

  // Panics with "Undefined validation function 'required­' on field 'V'"
  if err := v.Struct(myTypeWithInvisibleChar{"potato"}); err != nil {
      t.Fatal(err)
  }
}

I have test with this debug log image And it shows the normal required text image

I also did a check with online tool Your example shows this output image And this is the mine image

Note: And the bug only happen during my stress test. In normal traffic, it's just fine