search

http-rs / surf

Fast and friendly HTTP client framework for async Rust
https://docs.rs/surf
Apache License 2.0
1.45k stars 119 forks source link

Incorrect Content-Type value returned #280

Open hipstermojo opened 3 years ago

hipstermojo commented 3 years ago

Surf seems to return a Content-Type: application/json header when fetching images from webpages built with Notion. I tried making the same requests over curl and it logged the correct headers e.g Content-Type: image/gif for this gif which is an image from this article.

Fishrock123 commented 3 years ago

Can you share one of these files? That would indicate it has incorrect magic bytes somehow.

hipstermojo commented 3 years ago

That is the gif linked to in the first comment. aaf09519-3f8a-42f9-b794-43dc0d322cad

That would indicate it has incorrect magic bytes somehow.

Does it mean that surf relies on magic bytes first, then the content-type header, to determine a file type?

Fishrock123 commented 3 years ago

Oh sorry, on remote read it should be content-type first.

It seems that when doing client decode it is not actually set properly.

Though, I would expect it to be application/octet-stream, since that's what Body::from_reader would set it to.

From deepest outwards to Surf:

  1. http_types::Body::from_reader()
  2. async_h1::client::decode()
  3. http_client::h1::H1Client::send()
  4. surf::Client::send()
Fishrock123 commented 3 years ago

I suspect this code should probably live in async-h1.