Closed johanstokking closed 5 years ago
Here are the formalized responsibilities of the API Boundary and the individual components.
API Boundary: (A1) The API boundary validators are responsible for making sure that for each path specified in the request's field mask, the corresponding field should either have a valid value, or no value at all. (A2) The API boundary may unset any fields that are not specified in the field mask. (A3) The ids
path is implicitly part of the field mask. (A4) As far as the API boundary is concerned, only the TTN identifiers are mandatory (so not the EUIs in end devices, also note that the JS's GetByEUI's are not in our gRPC API). (A5) For each RPC a list of field mask paths that are allowed to be in the request message can be registered. The API boundary will then validate the field mask paths in the request message. (#167)
Components: (C1) Are responsible for validating presence non-TTN identifiers if required. (C2) Are responsible for validating that the result of an operation results in a valid state (i.e. it does not leave mandatory fields unset, and it does not leave related fields in an invalid state).
Updated issue description
I don't think everything in this umbrella issue is actually done with #194 getting merged.
cc: @KrishnaIyer @rvolosatovs
Can you clarify what isn't?
I don't know, but if we can answer all questions below with "yes", then I guess we have everything.
With the numbers of What is missing? What do you want to see?:
2: Is Validate() error
or ValidateContext(ctx) error
really implemented on all request messages now?
3: Are field that aren't specified in the field mask now unset on Set-like operations?
4: Do we have these assertions?
From https://github.com/TheThingsNetwork/lorawan-stack/issues/69#issuecomment-465184422:
A1: Are all fields now validated with (generated) validators?
Validate() error
or ValidateContext(context.Context) error
on all requests, since we just call ValidateFields()
if they're not.
All requests with fieldmasks have a ValidateContext(context.Context) error
definedreq.Field.SetFields(req.FieldMask.Paths...)
at the top of the RPC, which needs this. Otherwise, for consistency sake, we would need to come up with a new method signature and handle it accordingly, which seems to just be a waste of time.@johanstokking no, it's about unsetting the fields in the message itself. E.g. keys.app_s_key
in EndDevice
if keys.app_s_key
is not specified in the fieldmask.
Ok, so it looks like all we have left from this umbrella is point (3) from my comment above. I agree with @rvolosatovs that implementing this in the generic part of the API boundary would be quite a lot for work, while it would be only a few lines in RPCs.
With that, I think we can close this issue.
Summary:
This is an umbrella issue that covers validation of requests in the API boundary, and the contracts with components beind the API boundary.
Why do we need this?
The idea, which came from this article, is that we introduce validators on each message, responsible for validating and cleaning incoming data.
What is already there? What do you see now?
mwitkow/go-proto-validators
) that validate our protos against rules defined in our protos.Validate() error
orValidateContext(ctx) error
What is missing? What do you want to see?
github.com/TheThingsIndustries/protoc-gen-validate
) to validate specific fields in our protos against rules defined in our protos (and not the entire message if only few fields are masked).Validate() error
orValidateContext(ctx) error
on all request messages, and make that func callValidateFields(...string) error
with the appropriate fields. On Get-like requests this means the fields of the request itself. On Set-like request this means the fields of the entity being updated.How do you propose to implement this?
194
Original issue: https://github.com/TheThingsIndustries/lorawan-stack/issues/1058 by @htdvisser