Feature: handling files from `multipart/form-data` request

lsdch commented 4 months ago

This PR aims to improve the handling of multipart/form-data requests by:

3570bf1 allowing users to document the request body at the level of operations, which is currently not possible since it is overwritten by huma when RawBody is multipart.FormData. The generated schema is more of an example and does not reflect the actual form data structure, which should be specified by the user.
d6eb219 implementing MultipartFormFiles, which can be used to type RawBody in request inputs and provides logic for decoding and validating files from a multipart/form-data request as multipart.File values, as well as generating the associated schema in the spec. Although this could be extended to support data types other than files, it would require significant refactoring to reuse some of the logic from body/query/path/header parameters decoding, and I was not confident enough to engage into it.
01ca953 implementing mimetype validation for submitted files

There is currently no automated tests for this PR, I would like to first get some feedback before spending more time on this (and still have to get familiar with humatest).

Example usage:

type FileData struct {
        // This is an example, any number of `multipart.File` fields can be defined.
        // Nested structs are not supported.
    SomeFile multipart.File   `form-data:"some_file" content-type:"image/png" required:"true"`
    SeveralFiles []multipart.File `form-data:"several_files" content-type:"image/png,image/jpeg" required:"true"`
}

type FileHandlerInput struct {
    RawBody huma.MultipartFormFiles[FileData]
}

func FileHandler(ctx context.Context, input *FileHandlerInput) (*struct{}, error) {
  fileData := input.RawBody.Data()
  DoSomeThingWith(fileData.SomeFile)
  OrSomethingElseWith(fileData.SeveralFiles)
}

huma.Register(api,
  huma.Operation{
      Path:             "/handle-files",
      Method:        http.MethodPost,
      OperationID:     "Handle files",
  }, FileHandler)

When using MultipartFormFiles[T], only the fields in T that implement multipart.File or []multipart.File are considered when generating the spec or resolving a request. The raw form data can be accessed at MultipartFormFiles.Form which is a multipart.Form.

form-data tag specifies the lookup key to retrieve the encoded file in the multipart.Form. In its absence, the name of the field is used instead. content-type specifies the acceptable mime types for a file. It is used to document the endpoint in the spec, and validate file format using http.DetectContentType required makes the field required. In the case of []multipart.File, validation checks that at least one file was submitted.

Generated operation spec :

{
  "/handle-files": {
    "post": {
      "operationId": "HandleFiles",
      "requestBody": {
        "content": {
          "multipart/form-data": {
            "encoding": {
              "some_file": {
                "contentType": "image/png"
              },
              "several_files": {
                "contentType": "image/png,image/jpeg"
              }
            },
            "schema": {
              "properties": {
                "some_file": {
                  "format": "binary",
                  "type": "string"
                },
                "several_files": {
                  "items": {
                    "format": "binary",
                    "type": "string"
                  },
                  "type": "array"
                }
              },
              "required": [
                "some_file",
                "several_files"
              ],
              "type": "object"
            }
          }
        }
      }
    }
  }
}

Summary by CodeRabbit

New Features
- Introduced functionality for handling multipart form data in HTTP requests.
- Added support for validating MIME types in uploaded files.
- Enhanced the ability to read and decode form data, including single and multiple file uploads.
Tests
- Added multiple test cases to ensure robust handling of multipart form data, including validation, error handling, and content type detection.

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 93.93939% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 92.80%. Comparing base (e1b7179) to head (f1a58f1).

Files	Patch %	Lines
formdata.go	94.87%	4 Missing and 4 partials :warning:
huma.go	90.47%	2 Missing and 2 partials :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #415 +/- ## ========================================== + Coverage 92.76% 92.80% +0.03% ========================================== Files 21 22 +1 Lines 3567 3750 +183 ========================================== + Hits 3309 3480 +171 - Misses 220 226 +6 - Partials 38 44 +6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

danielgtaylor commented 4 months ago

@lsdch this looks amazing! I will dig in and give it a thorough review soon, just wanted to say thanks for all the work on this and I like the idea of better file handling & documentation.

coderabbitai[bot] commented 3 months ago

Walkthrough

The changes introduce extensive support for handling multipart form data in HTTP requests within the huma package. This includes defining structures and methods for reading, validating, and decoding form files, as well as adding comprehensive test cases to ensure robust functionality.

Changes

Files	Change Summary
`formdata.go`	Added structures and methods for handling multipart form data, including file reading, validation, and decoding.
`huma.go`	Renamed `rawBodyMultipart` to `rawBodyDecodedMultipart` in the `Register` function.
`huma_test.go`	Added multiple test cases for multipart file handling, including validation, optional/required files, and content type checks.

🐰 In bytes and streams, we trust,
To handle forms, precise and just.
With MIME types checked and files read right,
Our code now gleams, a coder's delight.
So cheers to tests, both new and bold,
Our multipart forms, a tale well told.
🌟

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

Tips

### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger a review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

lsdch commented 3 months ago

@danielgtaylor hey, finally took some time to write tests for this feature. Coverage is almost 100%, except for 3 error checks on file I/O that are hard to trigger.

API has changed a little, I introduced small convenience features to help working with optional files and content-type. Also changed tag names to be more consistent with the naming conventions in Huma -- I don't mind changing them again to whatever you like.

I expect that some of the code could be better optimized, especially the parts using reflect, which I do not know in depth yet.

New usage example :

type FileData struct {
        // Now using huma.FormFile instead of multipart.File
        // FormFile is a simple wrapper around multipart.File which provides some convenience features
    SomeFile huma.FormFile   `form:"some_file" contentType:"image/png" required:"true"`
    SomeImage huma.FormFile   `form:"some_image" contentType:"image/*"`
    SeveralFiles []huma.FormFile `form:"several_files" contentType:"image/png,image/jpeg" required:"true"`
}

type FileHandlerInput struct {
    RawBody huma.MultipartFormFiles[FileData]
}

func FileHandler(ctx context.Context, input *FileHandlerInput) (*struct{}, error) {
  fileData := input.RawBody.Data()
  DoSomeThingWith(fileData.SomeFile)
  OrSomethingElseWith(fileData.SeveralFiles)

  if fileData.SomeImage.IsSet { // We can now check the presence of optional files
    fmt.Printf("Content type: %s", fileData.SomeImage.ContentType) // Actual content type (declared in the form or detected as fallback)
  } 
}

huma.Register(api,
  huma.Operation{
      Path:             "/handle-files",
      Method:        http.MethodPost,
      OperationID:     "Handle files",
  }, FileHandler)

danielgtaylor commented 3 months ago

@lsdch awesome work! I will dig in and do a thorough review ASAP! Thank you.

danielgtaylor / huma