seanmonstar / reqwest

An easy and powerful Rust HTTP Client
https://docs.rs/reqwest
Apache License 2.0
9.92k stars 1.12k forks source link

Segmentation fault when making an HTTPS connection #1766

Closed bonigarcia closed 1 year ago

bonigarcia commented 1 year ago

Hello. I am a Selenium committer and the developer of selenium-manager, a CLI tool developed in Rust belonging to the Selenium project. selenium-manager is currently used by many Selenium users nowadays, and recently, one of them has reported a segmentation fault problem.

Internally, selenium-manager uses reqwest to manage the HTTP connections, and it seems the problem happens when making an HTTPS connection. The last logged trace is this one.

$ ./selenium-manager --driver chromedriver --trace
TRACE   Reading metadata from /home/ben/.cache/selenium/selenium-manager.json
DEBUG   Using shell command to find out chrome version
DEBUG   Running command: "google-chrome --version"
DEBUG   Output: "Google Chrome 110.0.5481.177 "
DEBUG   The version of chrome is 110.0.5481.177
TRACE   Writing metadata to /home/ben/.cache/selenium/selenium-manager.json
DEBUG   Detected browser: chrome 110
TRACE   Reading metadata from /home/ben/.cache/selenium/selenium-manager.json
DEBUG   Reading chromedriver version from https://chromedriver.storage.googleapis.com/LATEST_RELEASE_110
DEBUG   starting new connection: https://chromedriver.storage.googleapis.com/
[1]    750214 segmentation fault (core dumped)  ./selenium-manager --driver chromedriver --trace

The problem happened in Ubuntu 64, and the binary is built in GitHub Actions (see job): selenium-manager_linux-x64

We use the following setup in Cargo:

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [ "rustls-tls" ] }

[profile.release]
opt-level = 'z'     # Optimize for size
lto = true          # Enable Link Time Optimization
codegen-units = 1   # Reduce number of codegen units to increase optimizations
panic = 'abort'     # Abort on panic
strip = true        # Strip symbols from binary*

... and the binary is built using static linkage (RUSTFLAGS: '-C target-feature=+crt-static') as follows:

cargo build --release --target x86_64-pc-windows-msvc

Do you have an idea of the cause of the problem? I really appreciate any help you can provide.

seanmonstar commented 1 year ago

You said the problem occurs on Ubuntu 64, but you're build target is Windows MSVC? That sounds problematic...

Otherwise, I don't have an idea of the cause. I've not seen a report of a segfault. I'd probably start by removing some of those release optimizations, to see if that makes a difference. And generally, I'd run the program with gdb to capture the stack when the segfault is raised.

bonigarcia commented 1 year ago

You said the problem occurs on Ubuntu 64, but you're build target is Windows MSVC? That sounds problematic...

Sorry, that was a mistake copying and paste. The actual command to build it on Linux is:

cargo build --release --target x86_64-unknown-linux-gnu
bonigarcia commented 1 year ago

And generally, I'd run the program with gdb to capture the stack when the segfault is raised.

Thanks a lot for the advice. I'm trying to generate a build release without the optimization flags but including debugging symbols (to debug it with gdb). To that aim, I included the following in my Cargo.toml:

[profile.release]
debug = true

Nevertheless, I get the following error when compiling:

/usr/bin/ld: /home/runner/work/selenium/selenium/rust/target/x86_64-unknown-linux-gnu/release/deps/libselenium_manager-9f78026e3142334c.rlib(selenium_manager-9f78026e3142334c.selenium_manager.6674b1a6-cgu.15.rcgu.o): undefined reference to symbol '__tls_get_addr@@GLIBC_2.3' 
[284](https://github.com/SeleniumHQ/selenium/actions/runs/4354767567/jobs/7610480788#step:4:285) /usr/bin/ld: /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2: error adding symbols: DSO missing from command line 
[285](https://github.com/SeleniumHQ/selenium/actions/runs/4354767567/jobs/7610480788#step:4:286) collect2: error: ld returned 1 exit status

I looked for that error but could not find a solution. Do you know something about it? (I am aware this is not related to reqwest, but I am struggling with this error)

seanmonstar commented 1 year ago

Is LTO stripping it, perhaps? Or z opt-level?

bonigarcia commented 1 year ago

I removed all optimizations from Cargo.toml (including opt-level = 'z' and strip = true ), so I suppose the problem is other.

satakuma commented 1 year ago

I think it's because of the static linking, see hyperium/hyper#2537.

seanmonstar commented 1 year ago

Ah, that must be it! Thanks for reminding us.

bonigarcia commented 1 year ago

Indeed, static linking did the trick! Thanks a lot, @satakuma and @seanmonstar!