test_https_client() fails to read http response body

[ardave]

It looks like something goes wrong converting byte array into a string, and the body winds up presented as an empty string. I've attached a log output excerpt.

Log excerpt is generated using a TinyPICO (pretty sure V2) and commit 2918d36.

Let me know anything I can help with, more information I can provide, etc.

I (20158) rust_esp32_std_demo::experimental: About to fetch content from https://google.com I (20478) esp-x509-crt-bundle: Certificate validated I (21868) HTTP_CLIENT: Body received in fetch header state, 0x3ffdd35d, 220 I (21868) esp_idf_svc::http::client: Got response 301, about to follow redirect I (22138) esp-x509-crt-bundle: Certificate validated I (23568) HTTP_CLIENT: Body received in fetch header state, 0x3ffdd326, 283 I (23578) rust_esp32_std_demo::experimental: Body (truncated to 3K): "" I (23588) esp_idf_svc::http::server: Started Httpd server with config Configuration { http_port: 80, https_port: 443, max_sessions: 16, session_timeout

[Squirrelcoding]

I changed the function such that it uses response.reader().read_exact(&mut body)?; instead of let (body, _) = io::read_max(response.reader(), &mut body)?; which is used in the demo. I'm still trying to figure out why this works lol

    fn test_https_client() -> anyhow::Result<()> {
        use embedded_svc::http::{self, client::*, status, Headers, Status};
        use esp_idf_svc::http::client::*;
        use embedded_svc::io::Read;

        let url = String::from("https://google.com");

        info!("About to fetch content from {}", url);

        let mut client = EspHttpClient::new(&EspHttpClientConfiguration {
            crt_bundle_attach: Some(esp_idf_sys::esp_crt_bundle_attach),

            ..Default::default()
        })?;

        let mut response = client.get(&url)?.submit()?;

        let mut body = [0_u8; 3048];

        response.reader().read_exact(&mut body)?;
        info!(
            "Body (truncated to 3K):\n{:?}",
            String::from_utf8_lossy(&body).into_owned()
        );

        Ok(())
    }

UPDATE

Ok so I have investigated further and the problem is this line. The code should not shadow the body variable as for some reason (which I'm still investigating) the read_max function always returns an empty array but still modifies the input array.

// io::read_max always returns an empty array for the first element of the tuple, which assigns body to an empty array
let (body, _) = io::read_max(response.reader(), &mut body)?;

// The solution is to remove `let (body, _) =`. That way io::read_max still writes to the buffer
io::read_max(response.reader(), &mut body)?;

Basically what is happening is that io::read_max is still reading to &mut body, but it returns an empty buffer for reasons I'm still investigating. Assigning body to the tuple returned basically sets body to the empty buffer I just mentioned.

ok I made a pull request fixing the bug

[ivmarkov]

io::read_max has a bug which is fixed in embedded-svc master in the meantime, however read_max is renamed now to try_read_full to follow the Rust STD naming convention.

Returning a buffer (not that try_read_full does that anymore) is not a problem. The returned buffer being empty is the problem, but that was because read_max was buggy.

[ivmarkov]

Oh and read_exact has a different semantics compared to read_max / try_read_full in that it will error if it reads less than the size of the buffer. So you probably don't want to use it.

[ardave]

Removing the shadowed variable binding and mutating the vector in-place does the trick.

I greatly appreciate both of you looking into this.

Dave