bogdanfinn / tls-client

net/http.Client like HTTP Client with options to select specific client TLS Fingerprints to use for requests.
BSD 4-Clause "Original" or "Old" License
668 stars 133 forks source link

[Bug]: No "pre_shared_key" when using shared lib #115

Closed qiu-x closed 1 month ago

qiu-x commented 1 month ago

TLS client version

master and 1.7.5

System information

Arch Linux

Issue description

When using the shared library version of tls-client, the next request does not contain the "pre_shared_key" in the "extensions" section of the test at https://tls.peet.ws/api/all, yet the Go version works as expected.

Steps to reproduce / Code Sample

I added the tls.UtlsPreSharedKeyExtension extension to the chrome_124 profile in the following way:

diff --git a/profiles/internal_browser_profiles.go b/profiles/internal_browser_profiles.go
index 350bcc9..197b6f1 100644
--- a/profiles/internal_browser_profiles.go
+++ b/profiles/internal_browser_profiles.go
@@ -182,6 +182,7 @@ var Chrome_124 = ClientProfile{
                        {Group: tls.X25519},
                    }},
                    &tls.UtlsGREASEExtension{},
+                   &tls.UtlsPreSharedKeyExtension{OmitEmptyPsk: true},
                },
            }, nil
        },

and after building the library, I tested it with this program:

#include <stdio.h>
#include "tls-client-linux-ubuntu-amd64-with_psk.h"

char* requestPayload = 
  "{"
    "\"tlsClientIdentifier\": \"chrome_124\","
    "\"withDefaultCookieJar\": true,"
    "\"withDebug\": true,"
    "\"catchPanics\": false,"
    "\"timeoutSeconds\": 60,"
    "\"sessionId\": \"test_session\","
    "\"requestUrl\": \"https://tls.peet.ws/api/all\","
    "\"requestMethod\": \"GET\","
    "\"requestCookies\": []"
"}";

int main() {
    char* resp = request(requestPayload);
    printf("%s\n", resp);

    printf("\n-----\n\n");

    resp = request(requestPayload);
    printf("%s\n", resp);
    return 0;
}

program output: shared_lib_output.txt (The second request is missing the "pre_shared_key")

I also made a Go program to test whether the issue was only present in the shared lib: main.go

However the Go version worked correctly (golang_output.txt) and the "pre_shared_key" is present in the second request response, which leads me to believe that there is some kind of issue in the shared lib.

Please let me know if you noticed any issues with my testing approach or if there is a way to get around this problem.

qiu-x commented 1 month ago

Ok, I already figured out what caused the problem. I had to remove go.mod and go.sum in cffi_dist, because the files made the shared libs use the code on GitHub and ignore my modifications. Perhaps those files should be generally removed from the repo to avoid similar confusion in the future?

bogdanfinn commented 1 month ago

@qiu-x These two files are for the dependency managenent for the shared lib and required for the build process with xgo. Yeah if you are doing modifications in the tls client code itself you need to the the shared library project to actualy use the specific modified version of the tls client and not the one published on github.

You can use a replace statement inside the go.mod targeting the local modified tls_client:

I also commented out the line in the go.mod file.

// replace github.com/bogdanfinn/tls-client => ../