RicoSuter / NSwag

The Swagger/OpenAPI toolchain for .NET, ASP.NET Core and TypeScript.
http://NSwag.org
MIT License
6.77k stars 1.29k forks source link

OpenAPI 3.0.1 upload/download file example #2495

Open joshmouch opened 4 years ago

joshmouch commented 4 years ago

Do you have an example of what an ASP.Net Core 3.0 API operation that uploads and downloads a file and what the OpenAPI 3.0.1 yaml should look like in order to properly generate a client using NSwag code generator?

I've tried every combination I can think of, and nothing seems to work. Either I get errors trying to deserialize to a FileContent type, or I get errors about not finding the Content boundary or else the client has random parameters that aren't used or a myriad of other issues. At this point, my yaml is identical to what's on the OpenApi documentation page, but this doesn't work, either: https://swagger.io/docs/specification/describing-request-body/file-upload/ I just need a working example to go off of so I can stop guessing.

lprichar commented 4 years ago

Seconded. I've got the same problem, although it's looking suspiciously like a bug to me. For downloading specifically I've tried:

The result is always the same: the proxy attempts to call blobToText on the result and it throws an error. What am I doing wrong?

lprichar commented 4 years ago

I got it!! :) With Swashbuckle anyway. File download can be accomplished with a custom operation filter, and an reusable attribute to indicate content-type result:

Custom attribute:

/// <summary>
/// Indicates swashbuckle should expose the result of the method as a file in open api (see https://swagger.io/docs/specification/describing-responses/)
/// </summary>
[AttributeUsage(AttributeTargets.Method)]
public class FileResultContentTypeAttribute : Attribute
{
    public FileResultContentTypeAttribute(string contentType)
    {
        ContentType = contentType;
    }

    /// <summary>
    /// Content type of the file e.g. image/png
    /// </summary>
    public string ContentType { get; }
}

Operation filter:

public class FileResultContentTypeOperationFilter : IOperationFilter
{
    public void Apply(OpenApiOperation operation, OperationFilterContext context)
    {
        var requestAttribute = context.MethodInfo.GetCustomAttributes(typeof(FileResultContentTypeAttribute), false)
            .Cast<FileResultContentTypeAttribute>()
            .FirstOrDefault();

        if (requestAttribute == null) return;

        operation.Responses.Clear();
        operation.Responses.Add("200", new OpenApiResponse
        {
            Content = new Dictionary<string, OpenApiMediaType>
            {
                {
                    requestAttribute.ContentType, new OpenApiMediaType
                    {
                        Schema = new OpenApiSchema
                        {
                            Type = "string",
                            Format = "binary"
                        }
                    }
                }
            }
        });
    }
}

Register it in Startup.cs:

services.AddSwaggerGen(options =>
{
    ...
    options.OperationFilter<FileResultContentTypeOperationFilter>();
}

Then annotate your controller with the attribute. Here's an example that generates an Excel file:

[HttpPost]
[Route("{fileName}.xlsx")]
[FileResultContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")]
public async Task<ActionResult> Generate(string fileName, [FromBody]MyDto myDto)
{
    var fileMemoryStream = GetExcelFileAsBytes(myDto);
    return File(fileMemoryStream,
        "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
        fileName + ".xlsx");
}

Oh, and the result looks like this:

"responses": {
    "200": {
        "content": {
            "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": {
                "schema": {
                    "type": "string",
                    "format": "binary"
                }
            }
        }
    }
}

And then nswag will finally stop calling the blobToText function on it.

joshmouch commented 4 years ago

I'm using Swashbuckle to generate the yaml, so my download operation is already generating the yaml you have, above. So the issue is with the generated client, I guess. I wonder if the issue is with no double quotes being around "text/plain"...? I get the following error:

Could not deserialize the response body stream as FileResponse

 / api / Test / DownloadFile: {
    get: {
        tags: [
            "Test"
        ],
        responses: {
            200: {
                description: "Success",
                content: {
                    text / plain: {
                        schema: {
                            type: "string",
                            format: "binary"
                        }
                    }
                }
            }
        }
    }
},
lprichar commented 4 years ago

The "text/plain" looks suspicious to me, maybe try a different content-type? Also, yes quotes.

joshmouch commented 4 years ago

With the download, it appears the API operation is writing the file content directly to the body, instead of to a serialized FileResult object. However, the NSwag client is expecting a FileResult object.

joshmouch commented 4 years ago

@lprichar Does your generated client use a FileResult object? If so, is that filename you sent back availabled? I can't figure out why mine's just sending the content back in the body instead of an entire serialized FileResult.

Developers - do you have a working upload example for ASP.Net Core 3.0? It sounds like we're close to figuring out the download with what @lprichar has (though I'm still getting a serialization error).

lprichar commented 4 years ago

The generated client uses a FileResponse. Once I swapped out "text/plain" for the actual content type ("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" for me, see example code above) then method called process[METHODNAME]fieldSerializer changed significantly and the "Accept" line in the [METHODNAME]FieldSerializer method changed to show my content-type. But everything still operates on a FileResponse e.g. protected downloadExcelFile(response: HttpResponseBase): Observable<FileResponse> { ... }

joshmouch commented 4 years ago

@lprichar Just FYI, I think you can do the same thing as your custom attribute by adding this, instead:

[ProducesResponseType(typeof(FileResult), 200)]
[Produces(contentType: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")]
joshmouch commented 4 years ago

@lprichar You're right about the content-type. When it's a "text\plain", there's a bug triggered in the NSwag generated client.

With content-type="text/plain":

                        if (status_ == "200") 
                        {
                            var objectResponse_ = await ReadObjectResponseAsync<FileResponse>(response_, headers_).ConfigureAwait(false);
                            return objectResponse_.Object;
                        }

With content-type = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"

                        if (status_ == "200" || status_ == "206") 
                        {
                            var responseStream_ = response_.Content == null ? System.IO.Stream.Null : await response_.Content.ReadAsStreamAsync().ConfigureAwait(false);
                            var fileResponse_ = new FileResponse((int)response_.StatusCode, headers_, responseStream_, null, response_); 
                            client_ = null; response_ = null; // response and client are disposed by FileResponse
                            return fileResponse_;
                        }

In the first one, it's trying to deserialize the body, which is wrong.

lprichar commented 4 years ago

Nice, thanks. I'll be tackling file uploads on Monday, please do share if you make any progress on that else I'll post back here with any progress then.

joshmouch commented 4 years ago

This is what my yaml looks like for the file upload. I can upload just fine from the Swagger UI. However, the generated NSwag client does not send a "content-type boundary". The exception is below:

    "/api/Test/UploadFile": {
      "post": {
        "tags": [
          "Test"
        ],
        "requestBody": {
          "content": {
            "multipart/form-data": {
              "schema": {
                "type": "object",
                "properties": {
                  "file": {
                    "type": "string",
                    "format": "binary"
                  }
                }
              }
            }
          }
        },
        "responses": {
          "200": {
            "description": "Success"
          }
        }
      }
    },
System.IO.InvalidDataException : Missing content-type boundary.
   at Microsoft.AspNetCore.Http.Features.FormFeature.GetBoundary(MediaTypeHeaderValue contentType, Int32 lengthLimit)
   at Microsoft.AspNetCore.Http.Features.FormFeature.InnerReadFormAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.ModelBinding.FormValueProviderFactory.AddValueProviderAsync(ValueProviderFactoryContext context)
lprichar commented 4 years ago

Sounds like it wants quotes around "multipart/form-data"

joshmouch commented 4 years ago

I should add that I have a custom IOperation to generate this YAML. The default YAML generated by Swashbuckle didn't work in Swagger UI. None of the properties names matched up with the Api method parameters, so ASP.Net Core couldn't bind the model:


            operation.RequestBody.Content["multipart/form-data"].Schema.Properties.Clear();
            operation.RequestBody.Content["multipart/form-data"].Encoding.Clear();

            var parameters = context.ApiDescription.ActionDescriptor.Parameters;

            var formFileParams = (from parameter in parameters
                                  where parameter.ParameterType.IsAssignableFrom(typeof(IFormFile))
                                  select parameter).ToArray();

            foreach (var formFileParam in formFileParams)
            {
                var argumentName = formFileParam.Name;
                operation.RequestBody.Content["multipart/form-data"].Schema.Properties.Add(argumentName, new OpenApiSchema() { Type = "string", Format = "binary" });
            }
RicoSuter commented 4 years ago

Maybe this processor solves your problem: https://github.com/RicoSuter/NSwag/pull/2353

But looking into file upload/download is on my TODO list...

joshmouch commented 4 years ago

@RicoSuter The processor doesn't help me, since it's just for updating the yaml, right? The yaml is correct in this case. It's what's on the OpenAPI example website. It works in SwaggerUI. It doesn't work with the c# generated NSwag client.

joshmouch commented 4 years ago

@RicoSuter

The issue seems to be in the Client.Class.liquid template around here:

{%     if operation.HasContent -%}
{%         if operation.HasBinaryBodyParameter -%}
                var content_ = new System.Net.Http.StreamContent({{ operation.ContentParameter.VariableName }});
...
{%     else -%}
{%         if operation.HasFormParameters -%}
...
{%             else -%}
                var boundary_ = System.Guid.NewGuid().ToString();
                var content_ = new System.Net.Http.MultipartFormDataContent(boundary_);
                content_.Headers.Remove("Content-Type");
                content_.Headers.TryAddWithoutValidation("Content-Type", "multipart/form-data; boundary=" + boundary_);

When using OpenApi v2 the latter part of that code is generated (which includes a boundary= in the header. When using OpenApi v3, the first if block is used. So the difference must be with v2, operation.HasContent=false, and with v3, operation.HasContent=true.

joshmouch commented 4 years ago

@RicoSuter When do you think this could be fixed? Looks like a simple template fix, but I'm not sure what you're trying to do with the different flags.

joshmouch commented 4 years ago

Another issue could be that the generated method expects a type Stream in OpenApi v3 but expected a FileResult in OpenApi v2.

joshmouch commented 4 years ago

@RicoSuter I wrote a test case for you:

using System;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using NJsonSchema;
using NSwag.Generation.WebApi;
using Xunit;

namespace NSwag.CodeGeneration.CSharp.Tests
{
    public class FileUploadTests
    {
        [Fact]
        public async Task When_openapi3_contains_octet_stream_response_then_FileResponse_is_generated()
        {
            // Arrange
            var json = @"{
                                ""openapi"": ""3.0.1"",
                                ""paths"": {
                                    ""/api/Test/UploadFile"": {
                                        ""post"": {
                                            ""tags"": [""UploadFile""],
                                            ""requestBody"": {
                                                ""content"": {
                                                    ""multipart/form-data"": {
                                                        ""schema"": {
                                                            ""type"": ""object"",
                                                            ""properties"": {
                                                                ""file"": {
                                                                    ""type"": ""file"",
                                                                    ""format"": ""binary""
                                                                }
                                                            }
                                                        },
                                                        ""encoding"": {
                                                            ""file"": {
                                                                ""style"": ""form""
                                                            }
                                                        }
                                                    }
                                                }
                                            },
                                            ""responses"": {
                                                ""200"": {
                                                    ""description"": ""Success"",
                                                    ""content"": {
                                                        ""text/plain"": {
                                                            ""schema"": {
                                                                ""type"": ""boolean""
                                                            }
                                                        }
                                                    }
                                                }
                                            }
                                        }
                                    }
                                }
                            }";
            var document = await OpenApiDocument.FromJsonAsync(json, null, SchemaType.OpenApi3, null);

            //// Act
            var codeGenerator = new CSharpClientGenerator(document, new CSharpClientGeneratorSettings
            {
                GenerateClientInterfaces = true
            });
            var code = codeGenerator.GenerateFile();

            //// Assert
            Assert.Contains("public async System.Threading.Tasks.Task UploadFileAsync(FileParameter file = null", code);
            Assert.Contains("content_.Headers.TryAddWithoutValidation(\"Content-Type\", \"multipart/form-data; boundary=\" + boundary_);", code);
            Assert.Contains("content_.Add(content_file_, \"file\", file.FileName ?? \"file\");", code);
        }
    }
}
joshmouch commented 4 years ago

@RicoSuter Do you think this will be addressed soon?

RicoSuter commented 4 years ago

I dont think its implemented soon as it is completely different than in swagger 2.0 (form parameters vs json schema) and we cannot just use the same implementation i think.

dtaalbers commented 4 years ago

I would love to have an example as well. My dotnet core API 3.1 doesn't receive the binary file at all. It is sent via the generated typescript client.

public async Task<IActionResult> UploadAttachment([FromForm] IFormFile file, [FromRoute] string id, [FromQuery] bool createThumbnail = false)
 uploadAttachment(id: string, createThumbnail?: boolean | undefined, body?: Blob | undefined): Promise<void> {
        let url_ = this.baseUrl + "/workspaces/{id}/attach?";
        if (id === undefined || id === null)
            throw new Error("The parameter 'id' must be defined.");
        url_ = url_.replace("{id}", encodeURIComponent("" + id)); 
        if (createThumbnail === null)
            throw new Error("The parameter 'createThumbnail' cannot be null.");
        else if (createThumbnail !== undefined)
            url_ += "createThumbnail=" + encodeURIComponent("" + createThumbnail) + "&"; 
        url_ = url_.replace(/[?&]$/, "");

        const content_ = body;

        let options_ = <RequestInit>{
            body: content_,
            method: "PUT",
            headers: {
                "Content-Type": "multipart/form-data",
            }
        };

Before dotnet 3.1 and Open Api v3 (SwashBuckle 5.0.0) it would add the content in FormData

 uploadAttachment(id: string, createThumbnail?: boolean, file?: FileParameter | null | undefined): Promise<void> {
        let url_ = this.baseUrl + "/workspaces/{id}/attach";
        if (id === undefined || id === null)
            throw new Error("The parameter 'id' must be defined.");
        url_ = url_.replace("{id}", encodeURIComponent("" + id)); 
        url_ = url_.replace(/[?&]$/, "");

        const content_ = new FormData();
        if (file !== null && file !== undefined)
            content_.append("file", file.data, file.fileName ? file.fileName : "file");

I used an OperationFilter to get the working generated typescript client. I've tried a lot of different OperationFilter solutions to get the client to work. But my file always remain null. Looking forward to some help with this.

mahinthan commented 4 years ago

On your client side, if you would replace the FileParameter/Blob into FormData and send the file as below, the IFormFile would get filled in.

const formData = new FormData();
formData.append('file', fileObj)
dtaalbers commented 4 years ago

On your client side, if you would replace the FileParameter/Blob into FormData and send the file as below, the IFormFile would get filled in.

const formData = new FormData();
formData.append('file', fileObj)

Are you suggesting this to me? If so, thanks for the reply! But I know that the client needs to have FormData. But I don't want to do that manually every time after I generate the client. The trick is to get nswag to do that. But the solution that I used for dotnet core 2.x and 4.x swashbuckle, doesn't work for dotnet core 3.x and swashbuckle 5.x.

dellel-firas commented 4 years ago

Hello, I had exactly the same issue, so I did like @mahinthan suggested, but it was not enough, in the request I got payload instead of formData, so the missing part of the solution is to delete the content type which have this value "multipart/form-data" from the generated code and it worked well "Content-Type": "multipart/form-data" But it's not really practice to change manually a generated code, that's why I added some code to the useTransformOptionsMethod remove the content type when it is equal to "multipart/form-data" Hope that it helps, and that will be fixed soon.