API: Handle data validation

pbastia commented 7 months ago

Description:

Ensure data passed to the API is valid. Examples may include checking data formats, data bounds, date ranges, upload sizes, valid table entries and other similar attributes.

How is the API validating the data passed in through the endpoints? I imagine this is highly dependent on how flexible the endpoints are?

@DataVillage are there any solutions you have in mind, or issues with the current implementation that you'd like to bring up?

My recommendation would be to do something like this https://www.django-rest-framework.org/api-guide/validators/. The basic idea is to put "validation logic in reusable components". There are some good examples in the above URL. Ideally validation logic will not be duplicated across different endpoints.

Acceptance Criteria:

Given I am a developer When I want to Then

Development Checklist:

[x] Date Validation
[x] Numeric Validation
[x] Field size validation
[x] Upload size validation
[x] Data with values from predefined tables is valid
[x] Meets the DOD

Definition of Ready (Note: If any of these points are not applicable, mark N/A)

[NA] User story is included
[NA] User role and type are identified
[x] Acceptance criteria are included
[NA] Wireframes are included (if required)
[x] Design / Solution is accepted by Product Owner
[x] Dependencies are identified (technical, business, regulatory/policy)
[x] Story has been estimated (under 13 pts)

·Definition of Done (Note: If any of these points are not applicable, mark N/A)

[x] Acceptance criteria are tested by the CI pipeline
[NA] UI meets accessibility requirements
[x] Configuration changes are documented, documentation and designs are updated
[x ] Passes code peer-review
[x] Passes QA of Acceptance Criteria with verification in Dev and Test
[x] Ticket is ready to be merged to main branch
[x] Can be demoed in Sprint Review
[x] Bugs or future work cards are identified and created
[x] Reviewed and approved by Product Owner

Notes:

Dependencies

Blocked by Endpoint Definitions
Blocking Endpoint Operations

pbastia commented 7 months ago

Data validation:

in the database?
in the forms?

pbastia commented 7 months ago

@Sepehr-Sobhani confirms this already exists!

pydantic as part of django-ninja does payload validation
Example here for data-in and here for data-out

Custom validators can be used for complex data types: example

@patriciarussellCAS @hannavovk closing this one

bcgov / cas-obps