Closed makorne closed 5 months ago
Hello! This isn't really a complete project, but a quick hack. The problem you were facing seemed to be related to Google changing download URLs for the chromedriver. I've updated my fork of the Rust-undetected-chromedriver crate to use the newer URL and it seems to be working now.
You can use the crate the following way:
let trends = gtrends34::GTrends::new("/tmp/directory/to/store/the/downloaded/files/").await?;
let result = trends.get_keyword("keyword").await?;
Thanks for prompt reply!
But now the new error:
Starting chromedriver...
Starting ChromeDriver 124.0.6367.207 (a9001a6e39fbaa559510ca866052950457dd4e6b-refs/branch-heads/6367_201@{#3}) on port 3813
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
error creating new session: webdriver did not create session: session not created: DevToolsActivePort file doesn't exist
thread 'main' panicked at /home/.cargo/git/checkouts/rust-undetected-chromedriver-791f14f9b72420ec/1e97d6e/src/lib.rs:135:25:
called `Option::unwrap()` on a `None` value
And if I add a googled option caps.add_chrome_arg("--remote-debugging-port=9222")?;
ChromeDriver was started successfully.
thread 'tokio-runtime-worker' panicked at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fantoccini-0.19.3/src/session.rs:921:34:
internal error: entered unreachable code: received unknown error (chrome not reachable) for INTERNAL_SERVER_ERROR status code
stack backtrace:
0: rust_begin_unwind
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panicking.rs:645:5
1: core::panicking::panic_fmt
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/panicking.rs:72:14
2: fantoccini::session::Session<C>::issue_wd_cmd::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fantoccini-0.19.3/src/session.rs:921:34
3: <T as futures_util::fns::FnOnce1<A>>::call_once
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/fns.rs:15:9
4: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/map.rs:57:73
5: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/lib.rs:91:13
6: <futures_util::future::either::Either<A,B> as core::future::future::Future>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/either.rs:109:32
7: fantoccini::session::Ongoing::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fantoccini-0.19.3/src/session.rs:383:51
8: <fantoccini::session::Session<C> as core::future::future::Future>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fantoccini-0.19.3/src/session.rs:448:30
9: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/core.rs:328:17
10: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/loom/std/unsafe_cell.rs:16:9
11: tokio::runtime::task::core::Core<T,S>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/core.rs:317:13
12: tokio::runtime::task::harness::poll_future::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:485:19
13: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/panic/unwind_safe.rs:272:9
14: std::panicking::try::do_call
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panicking.rs:552:40
15: __rust_try
16: std::panicking::try
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panicking.rs:516:19
17: std::panic::catch_unwind
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panic.rs:146:14
18: tokio::runtime::task::harness::poll_future
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:473:18
19: tokio::runtime::task::harness::Harness<T,S>::poll_inner
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:208:27
20: tokio::runtime::task::harness::Harness<T,S>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:153:15
21: tokio::runtime::task::raw::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/raw.rs:271:5
22: tokio::runtime::task::raw::RawTask::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/raw.rs:201:18
23: tokio::runtime::task::LocalNotified<S>::run
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/mod.rs:427:9
24: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/scheduler/multi_thread/worker.rs:639:17
25: tokio::runtime::coop::with_budget
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/coop.rs:107:5
26: tokio::runtime::coop::budget
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/coop.rs:73:5
27: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/scheduler/multi_thread/worker.rs:575:9
28: tokio::runtime::scheduler::multi_thread::worker::Context::run
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/scheduler/multi_thread/worker.rs:526:24
29: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/scheduler/multi_thread/worker.rs:491:21
30: tokio::runtime::context::scoped::Scoped<T>::set
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/context/scoped.rs:40:9
31: tokio::runtime::context::set_scheduler::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/context.rs:176:26
32: std::thread::local::LocalKey<T>::try_with
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/thread/local.rs:284:16
33: std::thread::local::LocalKey<T>::with
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/thread/local.rs:260:9
34: tokio::runtime::context::set_scheduler
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/context.rs:176:9
35: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/scheduler/multi_thread/worker.rs:486:9
36: tokio::runtime::context::runtime::enter_runtime
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/context/runtime.rs:65:16
37: tokio::runtime::scheduler::multi_thread::worker::run
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/scheduler/multi_thread/worker.rs:478:5
38: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/scheduler/multi_thread/worker.rs:447:45
39: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/blocking/task.rs:42:21
40: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/core.rs:328:17
41: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/loom/std/unsafe_cell.rs:16:9
42: tokio::runtime::task::core::Core<T,S>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/core.rs:317:13
43: tokio::runtime::task::harness::poll_future::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:485:19
44: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/panic/unwind_safe.rs:272:9
45: std::panicking::try::do_call
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panicking.rs:552:40
46: __rust_try
47: std::panicking::try
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panicking.rs:516:19
48: std::panic::catch_unwind
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panic.rs:146:14
49: tokio::runtime::task::harness::poll_future
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:473:18
50: tokio::runtime::task::harness::Harness<T,S>::poll_inner
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:208:27
51: tokio::runtime::task::harness::Harness<T,S>::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:153:15
52: tokio::runtime::task::raw::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/raw.rs:271:5
53: tokio::runtime::task::raw::RawTask::poll
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/raw.rs:201:18
54: tokio::runtime::task::UnownedTask<S>::run
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/mod.rs:464:9
55: tokio::runtime::blocking::pool::Task::run
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/blocking/pool.rs:159:9
56: tokio::runtime::blocking::pool::Inner::run
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/blocking/pool.rs:513:17
57: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
at /home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/blocking/pool.rs:471:13
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error creating new session: webdriver server disconnected: WebDriver session was closed while waiting
thread 'main' panicked at /home/.cargo/git/checkouts/rust-undetected-chromedriver-791f14f9b72420ec/1e97d6e/src/lib.rs:135:25:
called `Option::unwrap()` on a `None` value
The caps.add_chrome_arg("--remote-debugging-pipe")?;
// https://bugs.chromium.org/p/chromedriver/issues/detail?id=4403#c35
Does not help too.
Any ideas?
I was unable to recreate the error. After a bit of googling I've found a StackOverflow entry that suggests using --disable-dev-shm-usage
argument.
Also this entry talks about Chrome crashing when running as root.
Not sure if any of this will help, but I can't test the solutions on my end.
I already tried using --disable-dev-shm-usage argument. and many others from google :( My chrome not crashing but dont have debugging connect :( What OS do you use?
I'm using Debian Bookworm. You can try updating your Chrome install. I'm currently using chromium and this is the output of chromium --version
on my machine:
Chromium 125.0.6422.60 built on Debian 12.5, running on Debian 12.5
I have Chromium too: Version 119.0.6045.199 (Official Build) Fedora Project (64-bit) 37 But rust-undetected-chromedriver download and patch only chrome from google, is`nt it? How do you use chromium with rust-undetected-chromedriver ?
The Rust-undetected-chromedriver crate checks the installed chrome/chromium version, downloads the chromedriver that matches the version and then patches it to avoid detection by websites.
I have both chrome/chromium... May be this is a problem ...
Remove chrome, chromedriver and reinstall chromium does not help :(
Starting chromedriver...
Starting ChromeDriver 119.0.6045.105 (38c72552c5e15ba9b3117c0967a0fd105072d7c6-refs/branch-heads/6045@{#1103}) on port 2155
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
error creating new session: webdriver did not create session: session not created: DevToolsActivePort file doesn't exist
thread 'main' panicked at /home/.cargo/git/checkouts/rust-undetected-chromedriver-791f14f9b72420ec/1e97d6e/src/lib.rs:135:25:
called `Option::unwrap()` on a `None` value
On Debian 12.5 it works! Thank you!!
That's interesting. Probably has something to do with the chromium version in the Debian and Fedora repositories.
Hi! Thank you for your interesting crate. But how to use it?