search

rwf2 / multer

An async parser for multipart/form-data content-type in Rust
MIT License
158 stars 37 forks source link

"Multipart stream is incomplete" when form data has leading newlines. #16

Closed vicky5124 closed 3 years ago

vicky5124 commented 4 years ago

When trying to read fields from a Multipart where the original stream had leading data such as newlines, the query will error with "Multipart stream is incomplete"; this behavior differs from other web libraries, such as python's flask, where those leading newlines get ignored.

(ss) Example data where the issue happens

This request was made by the UnityWebRequest, so it's a completely valid real world scenario where this issue could happen, and it should be supported.

Currently the solution is to skip the first 2 bytes of the stream before passing it to the field iterator, but this is not ergonomic, and in some situations (such as being combined with rocket), the stream becomes useless as the entire data had to be written from the AsyncRead to accomplish this.

rousan commented 4 years ago

in some situations (such as being combined with rocket), the stream becomes useless as the entire data had to be written from the AsyncRead to accomplish this.

This should not happen as the http body stream in rocket framework does not include the leading new line, so multer-rs will work with any Rust web framework.

In most of the cases, the http request body does not include the trailing new line. Are you using it in any customized http client?

vicky5124 commented 4 years ago

This should not happen as the http body stream in rocket framework does not include the leading new line, so multer-rs will work with any Rust web framework.

yes, when a request is sent by rocket, this does not happen, the issue is when it receives it.

In most of the cases, the http request body does not include the trailing new line. Are you using it in any customized http client?

The client is an unmodified install of the Unity Web Request library, the data sent has those leading newlines; using d.stream_to_file("AAAA").await.unwrap(); returned the file on the screenshot, with those leading newlines.

    let mut d = data.open(1.megabytes()).stream_to_vec().await.unwrap();
    d.remove(0);
    d.remove(0);

    let (stream, _) = get_stream_from_bytes(d).await;

this is the solution i went with, which downloads the entire thing into the vector, defeating the point of the initial stream.

rousan commented 4 years ago

Okay, got it, will fix it tomorrow and release a new version.

SergioBenitez commented 3 years ago

@rousan Do you have plans to release a new version with a fix for this issue?

rousan commented 3 years ago

Hey @nitsuga5124 @SergioBenitez, I am really sorry for slow response, actually, I got stuck in some other work lately. Can you please want to be a contributor? PR is welcome.

vicky5124 commented 3 years ago

I already tried to look into PRing this, but i was unable to figure out a way to do it.

jebrosen commented 3 years ago

For reference, this paragraph from RFC 2046 section 5.1.1 seems relevant (emphasis mine).

There appears to be room for additional information prior to the first boundary delimiter line and following the final boundary delimiter line. These areas should generally be left blank, and implementations must ignore anything that appears before the first boundary delimiter line or after the last one.

It looks like implementing this may require adding an additional Stage: currently Stage::ReadingBoundary is used both before the first boundary and between fields, but additional data is only allowed before the first boundary.

I have some time this week to try this or another approach if nobody else is working on it.