[Question] How can I set `mutipartBody` to receive Chinese filename in form part ?

[counter2015]

Tapir version: 0.19.3

Scala version: 2.13.7

Describe the Problem

In rfc6266

The parameters "filename" and "filename" differ only in that "filename" uses the encoding defined in [RFC5987], allowing the use of characters not present in the ISO-8859-1 character set ([ISO-8859-1]).

I wrote an endpoint to hanle file upload , like this

val uploadEndpoint =
    endpoint
      .post
      .in("file")
      .in(path[String]("id"))
      .in(multipartBody[FileForm])
      .out(stringBody)
      .tag("File")

  def uploadLogic(id: String, form: FileForm): ZIO[Logging, Nothing, String] = {
    for {
      _ <- Logging.info(s"got upload file request, id is $id")
    } yield form.file.fileName.getOrElse("file name not found")
  }

It works well in old version of tapir 0.17.12, but not work in 0.19.3

since https://github.com/softwaremill/sttp/pull/877 add encoding in client side mutipart body, does tapir provide something like this in server side ? So I can use default UTF-8 Codec ?

How to reproduce?

$  sbt new counter2015/tapir-zio-http4s.g8
a template of tapir-zio-http4s project. 

name [tapir-exmple]: 

Template applied in /private/tmp/./tapir-exmple

$ cd tapir-exmple

$ sbt run

In another window, or in swagger page

$ curl -X 'POST' \
  'http://localhost:8080/file/1' \
  -H 'accept: text/plain' \
  -H 'Content-Type: multipart/form-data' \
  -F 'uploader=me' \
  -F 'file=@中文文件名.json;type=application/json'

response: Bad Request Invalid value for: body

Additional information

0.17.12 version can be found here, it works well https://github.com/counter2015/tapir-zio-http4s.g8/tree/0.17/src/main/g8

Raw log from postman

POST /file/2222 HTTP/1.1
accept: text/plain
Content-Type: multipart/form-data
User-Agent: PostmanRuntime/7.26.5
Postman-Token: cf00388a-a15a-4463-98ba-a48eea00de14
Host: localhost:8080
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 416
----------------------------postman
Content-Disposition: form-data; name="uploader"

counter
----------------------------postman
Content-Disposition: form-data; name="file"; filename="中文文件名.json"; filename*="UTF-8''%E4%B8%AD%E6%96%87%E6%96%87%E4%BB%B6%E5%90%8D.json"

<中文文件名.json>
----------------------------postman--
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=UTF-8
Date: Tue, 28 Dec 2021 13:22:30 GMT
Content-Length: 23
Invalid value for: body

[adamw]

Hm I think it's not about encoding of the body (the linked sttp issue sets the body encoding, as far as I know), but rather about using filename/filename* appropriately as you write.

Which interpreter are you using, http4s? I wonder what changed since 0.17 🤔

[counter2015]

yes, http4s. the full depenience can be found here

0.17 version

import sbt._

object Dependencies {

  val tapirVersion = "0.17.12"

  val tapir = Seq(
    "com.softwaremill.sttp.tapir" %% "tapir-zio"                % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-zio-http4s-server"  % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-json-circe"         % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-openapi-docs"       % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-openapi-circe-yaml" % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-swagger-ui-http4s"  % tapirVersion,
  )

  val zioLoggingVersion = "0.5.14"
  val log =  Seq(
    "ch.qos.logback" % "logback-classic" % "1.2.10",
    "dev.zio" %% "zio-logging" % zioLoggingVersion,
    "dev.zio" %% "zio-logging-slf4j" % zioLoggingVersion,
  )

  val coreDependency = tapir ++ log
}

0.19 version

import sbt._

object Dependencies {

  val tapirVersion = "0.19.3"

  val tapir = Seq(
    "com.softwaremill.sttp.tapir" %% "tapir-zio"                % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-zio-http4s-server"  % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-json-circe"         % tapirVersion,
    "com.softwaremill.sttp.tapir" %% "tapir-swagger-ui-bundle"  % tapirVersion,
  )

  val zioLoggingVersion = "0.5.14"
  val log =  Seq(
    "ch.qos.logback" % "logback-classic" % "1.2.10",
    "dev.zio" %% "zio-logging" % zioLoggingVersion,
    "dev.zio" %% "zio-logging-slf4j" % zioLoggingVersion,
  )

  val coreDependency = tapir ++ log
}

[adamw]

I think this is a problem with http4s, see: https://github.com/http4s/http4s/issues/5809

[adamw]

For future testing, here's my test code reproducing the problem:

import cats.effect._
import org.http4s.HttpRoutes
import org.http4s.server.Router
import org.http4s.blaze.server.BlazeServerBuilder
import org.http4s.syntax.kleisli._
import sttp.model.Part
import sttp.tapir._
import sttp.tapir.server.http4s.Http4sServerInterpreter

import java.io.File
import scala.concurrent.ExecutionContext
import scala.io.Source
import sttp.tapir.generic.auto._

/*
curl -X 'POST' 'http://localhost:8080/file/1' -H 'accept: text/plain' -H 'Content-Type: multipart/form-data' -F 'uploader=me' -F 'file=@中文文件名.json;type=application/json'
 */

object Test extends IOApp {

  case class FileForm(file: Part[File])

  val uploadEndpoint =
    endpoint.post
      .in("file")
      .in(path[String]("id"))
      .in(multipartBody[FileForm])
      .tag("File")

  def uploadLogic(id: String, form: FileForm): IO[Unit] = IO {
    println(s"\n\n\nGot: $id ${form.file}\n${Source.fromFile(form.file.body).mkString("\n")}")
  }

  val uploadRoutes: HttpRoutes[IO] =
    Http4sServerInterpreter[IO]().toRoutes(uploadEndpoint.serverLogicSuccess { case (id, form) => uploadLogic(id, form) })

  implicit val ec: ExecutionContext = scala.concurrent.ExecutionContext.Implicits.global

  override def run(args: List[String]): IO[ExitCode] = {
    BlazeServerBuilder[IO]
      .withExecutionContext(ec)
      .bindHttp(8080, "localhost")
      .withHttpApp(Router("/" -> uploadRoutes).orNotFound)
      .resource
      .use { _ => IO.never }
      .as(ExitCode.Success)
  }
}

[counter2015]

Thanks!

[weili96]

so , this question can be solve?

[adamw]

@weili96 well, we don't really have a solution to this problem. So I'd keep this open.