Kong / unirest-java

Unirest in Java: Simplified, lightweight HTTP client library.
http://kong.github.io/unirest-java/
MIT License
2.58k stars 591 forks source link

Strange behavior of response.getRequestSummary().getUrl() #523

Closed plantaest closed 2 months ago

plantaest commented 2 months ago

Describe the bug Hello Unirest project maintainers,

Recently, I've been exploring and using the Unirest library for a personal project. I've encountered a rather odd issue and would appreciate your assistance.

With Python's requests library, I can inspect the redirect history when making a call to a link using Response.history. For Unirest, since there isn't a similar feature, I've been using the asPaged method to achieve this.

To Reproduce Here's the code I'm using:

package io.github.plantaest;

import kong.unirest.core.HttpRequest;
import kong.unirest.core.PagedList;
import kong.unirest.core.Unirest;

import java.util.List;

public class Main {
    public static void main(String[] args) {
        Unirest.config()
                .followRedirects(false)
                .verifySsl(false);

        PagedList<String> responses = Unirest.get("http://w.wiki/9ke3")
                .asPaged(
                        HttpRequest::asString,
                        response -> List.of(301, 302).contains(response.getStatus())
                                ? response.getHeaders().getFirst("Location")
                                : null
                );

        for (var response : responses) {
            System.out.println("==========");
            System.out.println("Request HTTP Method: " + response.getRequestSummary().getHttpMethod());
            System.out.println("Request URL: " + response.getRequestSummary().getUrl());
            System.out.println("Response Header Location: " + response.getHeaders().getFirst("Location"));
            System.out.println("Response Status: " + response.getStatus());
            System.out.println("Response Body Length: " + response.getBody().length());
        }
    }
}

When running the above code, it prints out as follows:

==========
Request HTTP Method: GET
Request URL: https://meta.wikimedia.org/wiki/User:Plantaest
Response Header Location: https://w.wiki/9ke3
Response Status: 301
Response Body Length: 0
==========
Request HTTP Method: GET
Request URL: https://meta.wikimedia.org/wiki/User:Plantaest
Response Header Location: https://meta.wikimedia.org/wiki/User:Plantaest
Response Status: 301
Response Body Length: 0
==========
Request HTTP Method: GET
Request URL: https://meta.wikimedia.org/wiki/User:Plantaest
Response Header Location: 
Response Status: 200
Response Body Length: 34953

Expected behavior The "Request URL" value is the same across all 3 responses, which doesn't seem right. According to the javadoc, you note: (HttpResponse::getRequestSummary) @return a Summary of the HttpRequest that created this response.

Therefore, I expect that the response.getRequestSummary().getUrl() method should return the correct URL of the request when used with asPaged. The expected printout might look like:

==========
Request HTTP Method: GET
Request URL: http://w.wiki/9ke3
Response Header Location: https://w.wiki/9ke3
Response Status: 301
Response Body Length: 0
==========
Request HTTP Method: GET
Request URL: https://w.wiki/9ke3
Response Header Location: https://meta.wikimedia.org/wiki/User:Plantaest
Response Status: 301
Response Body Length: 0
==========
Request HTTP Method: GET
Request URL: https://meta.wikimedia.org/wiki/User:Plantaest
Response Header Location: 
Response Status: 200
Response Body Length: 34953

Screenshots None.

Environmental Data:

Additional context Although it may not be directly related, I believe Unirest should have a feature similar to Response.history in requests, even though asPaged can currently serve as a workaround.

Aside from the issues mentioned above, I find this library to be very good. Thank you!

ryber commented 2 months ago

This is because the asPaged method re-uses the same request object changing its URL every time, but keeping everything else. So you only ever get the last one. It should create a all new request with everything copied from the last one. That will take some work

ryber commented 2 months ago

This is complete in 4.3.2

plantaest commented 1 month ago

Thanks!