platonai / PulsarRPA

Automate webpages at scale, scrape web data completely and accurately with high performance, distributed AI-RPA.
Apache License 2.0
778 stars 119 forks source link

Using Java sample code "BasicUsage" reports an error. #64

Closed kuekiko closed 8 months ago

kuekiko commented 8 months ago

OS: Windows11 Java Version: JDK-17 Log info: "C:\Program Files\Google\Chrome\Application\chrome.exe" --disable-gpu --hide-scrollbars --remote-debugging-port=0 --no-default-browser-check --no-first-run --no-startup-window --mute-audio --disable-background-networking --disable-background-timer-throttling --disable-client-side-phishing-detection --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-translate --disable-geolocation --disable-blink-features=AutomationControlled --metrics-recording-only --safebrowsing-disable-auto-update --no-sandbox --ignore-certificate-errors --remote-allow-origins=* --window-size=1920,1080 --pageLoadStrategy=none --throwExceptionOnScriptError=true --user-data-dir=C:\Users\xxx\AppData\Local\Temp\pulsar-xxx\context\tmp\3\cx.03281GEX261\pulsar_chrome 2024-03-28 00:34:34.985 WARN [main] a.p.p.p.b.d.c.ChromeDevtoolsBrowser - ai.platon.pulsar.crawl.fetch.driver.WebDriverException: createDevTools at ai.platon.pulsar.protocol.browser.driver.cdt.ChromeDevtoolsBrowser.createDevTools(ChromeDevtoolsBrowser.kt:221) at ai.platon.pulsar.protocol.browser.driver.cdt.ChromeDevtoolsBrowser.newDriver(ChromeDevtoolsBrowser.kt:141) at ai.platon.pulsar.protocol.browser.driver.cdt.ChromeDevtoolsBrowser.newDriver(ChromeDevtoolsBrowser.kt:66) at ai.platon.pulsar.protocol.browser.driver.cdt.ChromeDevtoolsBrowser.newDriver(ChromeDevtoolsBrowser.kt:21) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.computeBrowserAndDriver0(LoadingWebDriverPool.kt:388) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.computeBrowserAndDriver(LoadingWebDriverPool.kt:345) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.resourceSafeCreateDriverIfNecessary(LoadingWebDriverPool.kt:338) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.pollWebDriver(LoadingWebDriverPool.kt:314) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.poll(LoadingWebDriverPool.kt:202) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.poll(LoadingWebDriverPool.kt:197) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.pollWithEvents(LoadingWebDriverPool.kt:304) at ai.platon.pulsar.protocol.browser.driver.LoadingWebDriverPool.poll(LoadingWebDriverPool.kt:224) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager.runWithDriverPool(WebDriverPoolManager.kt:493) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager.access$runWithDriverPool(WebDriverPoolManager.kt:32) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager$runWithDriverPool$2.invokeSuspend(WebDriverPoolManager.kt:461) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager$runWithDriverPool$2.invoke(WebDriverPoolManager.kt) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager$runWithDriverPool$2.invoke(WebDriverPoolManager.kt) at ai.platon.pulsar.common.PreemptChannelSupport.whenNormalDeferred(PreemptChannelSupport.kt:58) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager.runWithDriverPool(WebDriverPoolManager.kt:449) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager.doRun(WebDriverPoolManager.kt:398) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager.run(WebDriverPoolManager.kt:157) at ai.platon.pulsar.protocol.browser.driver.WebDriverPoolManager.run(WebDriverPoolManager.kt:137) at ai.platon.pulsar.protocol.browser.emulator.context.WebDriverContext.run(WebDriverContext.kt:77) at ai.platon.pulsar.protocol.browser.emulator.context.BrowserPrivacyContext.doRun$suspendImpl(BrowserPrivacyContext.kt:68) at ai.platon.pulsar.protocol.browser.emulator.context.BrowserPrivacyContext.doRun(BrowserPrivacyContext.kt) at ai.platon.pulsar.crawl.fetch.privacy.PrivacyContext.run$suspendImpl(PrivacyContext.kt:285) at ai.platon.pulsar.crawl.fetch.privacy.PrivacyContext.run(PrivacyContext.kt) at ai.platon.pulsar.protocol.browser.emulator.context.BasicPrivacyContextManager.run1(BasicPrivacyContextManager.kt:96) at ai.platon.pulsar.protocol.browser.emulator.context.BasicPrivacyContextManager.run0(BasicPrivacyContextManager.kt:84) at ai.platon.pulsar.protocol.browser.emulator.context.BasicPrivacyContextManager.run(BasicPrivacyContextManager.kt:35) at ai.platon.pulsar.protocol.browser.emulator.impl.BrowserEmulatedFetcherImpl.fetchTaskDeferred(BrowserEmulatedFetcherImpl.kt:93) at ai.platon.pulsar.protocol.browser.emulator.impl.BrowserEmulatedFetcherImpl.fetchContentDeferred$suspendImpl(BrowserEmulatedFetcherImpl.kt:80) at ai.platon.pulsar.protocol.browser.emulator.impl.BrowserEmulatedFetcherImpl.fetchContentDeferred(BrowserEmulatedFetcherImpl.kt) at ai.platon.pulsar.protocol.browser.emulator.impl.BrowserEmulatedFetcherImpl$fetchContent$1.invokeSuspend(BrowserEmulatedFetcherImpl.kt:57) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:280) at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85) at kotlinx.coroutines.BuildersKtBuildersKt.runBlocking(Builders.kt:59) at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source) at kotlinx.coroutines.BuildersKtBuildersKt.runBlocking$default(Builders.kt:38) at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source) at ai.platon.pulsar.protocol.browser.emulator.impl.BrowserEmulatedFetcherImpl.fetchContent(BrowserEmulatedFetcherImpl.kt:56) at ai.platon.pulsar.protocol.browser.BrowserEmulatorProtocol.getResponse(BrowserEmulatorProtocol.kt:43) at ai.platon.pulsar.crawl.protocol.http.AbstractHttpProtocol.getProtocolOutputWithRetry(AbstractHttpProtocol.kt:118) at ai.platon.pulsar.crawl.protocol.http.AbstractHttpProtocol.getProtocolOutput(AbstractHttpProtocol.kt:88) at ai.platon.pulsar.crawl.component.FetchComponent.fetchContent0(FetchComponent.kt:108) at ai.platon.pulsar.crawl.component.FetchComponent.fetchContent(FetchComponent.kt:75) at ai.platon.pulsar.crawl.component.LoadComponent.fetchContent(LoadComponent.kt:505) at ai.platon.pulsar.crawl.component.LoadComponent.fetchContentIfNecessary(LoadComponent.kt:266) at ai.platon.pulsar.crawl.component.LoadComponent.load1(LoadComponent.kt:233) at ai.platon.pulsar.crawl.component.LoadComponent.load0(LoadComponent.kt:227) at ai.platon.pulsar.crawl.component.LoadComponent.loadWithRetry(LoadComponent.kt:129) at ai.platon.pulsar.crawl.component.LoadComponent.load(LoadComponent.kt:117) at ai.platon.pulsar.context.support.AbstractPulsarContext.load(AbstractPulsarContext.kt:367) at ai.platon.pulsar.session.AbstractPulsarSession.loadAndCache(AbstractPulsarSession.kt:493) at ai.platon.pulsar.session.AbstractPulsarSession.load(AbstractPulsarSession.kt:184) at ai.platon.pulsar.session.AbstractPulsarSession.load(AbstractPulsarSession.kt:171) at ai.platon.pulsar.session.AbstractPulsarSession.load(AbstractPulsarSession.kt:169) at org.example.BasicUsage.main(BasicUsage.java:24) Caused by: java.lang.RuntimeException: Failed creating proxy from abstract class at ai.platon.pulsar.browser.driver.chrome.util.ProxyClasses.createProxyFromAbstract(Reflects.kt:47) Caused by: java.lang.RuntimeException: Failed creating proxy from abstract class

at ai.platon.pulsar.browser.driver.chrome.impl.ChromeImpl.createDevTools0(ChromeImpl.kt:134)
at ai.platon.pulsar.browser.driver.chrome.impl.ChromeImpl.access$createDevTools0(ChromeImpl.kt:21)
at ai.platon.pulsar.browser.driver.chrome.impl.ChromeImpl$createDevTools$1.invoke(ChromeImpl.kt:92)
at ai.platon.pulsar.browser.driver.chrome.impl.ChromeImpl$createDevTools$1.invoke(ChromeImpl.kt:92)
at ai.platon.pulsar.browser.driver.chrome.impl.ChromeImpl.createDevTools$lambda$0(ChromeImpl.kt:92)
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
at ai.platon.pulsar.browser.driver.chrome.impl.ChromeImpl.createDevTools(ChromeImpl.kt:92)
at ai.platon.pulsar.protocol.browser.driver.cdt.ChromeDevtoolsBrowser.createDevTools(ChromeDevtoolsBrowser.kt:220)
... 59 more

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @1b919693 at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354) Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @1b919693

at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:200)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:194)
at javassist.util.proxy.SecurityActions.setAccessible(SecurityActions.java:102)
at javassist.util.proxy.FactoryHelper.toClass2(FactoryHelper.java:180)
at javassist.util.proxy.FactoryHelper.toClass(FactoryHelper.java:163)
at javassist.util.proxy.ProxyFactory.createClass3(ProxyFactory.java:501)
at javassist.util.proxy.ProxyFactory.createClass2(ProxyFactory.java:486)
at javassist.util.proxy.ProxyFactory.createClass1(ProxyFactory.java:422)
at javassist.util.proxy.ProxyFactory.createClass(ProxyFactory.java:394)
at javassist.util.proxy.ProxyFactory.create(ProxyFactory.java:649)
at javassist.util.proxy.ProxyFactory.create(ProxyFactory.java:634)
at ai.platon.pulsar.browser.driver.chrome.util.ProxyClasses.createProxyFromAbstract(Reflects.kt:42)
... 67 more

2024-03-28 00:34:34.985 WARN [main] a.p.p.p.b.e.c.WebDriverContext - 3. Retry task 1 in crawl scope | caused by: [Unexpected] Failed to create chrome devtools driver 2024-03-28 00:34:35.014 INFO [main] a.p.p.c.c.L.Task - 3. ? ? U for RT got 1601 0 <- 0 in 0s, last fetched 1m30s ago, fc:3/3 Retry(1601) rs: Driver exception, rsp: CRAWL | 03281GEX261 | file://C:\Users\xxx\AppData\Local\Temp\ln\b91271c0fc754bbc603037ac3cce9b5d.htm | https://list.jd.com/list.html?cat=652,12345,12349 -expires PT24H 2024-03-28 00:34:35.014 INFO [main] a.p.p.c.c.L.Task - Log explanation: https://github.com/platonai/PulsarRPA/blob/master/docs/log-format.adoc 2024-03-28 00:34:35.038 INFO [main] a.p.p.c.i.StreamingCrawlLoop - Registered 15 link collectors | loop#1 @1989332276 2024-03-28 00:34:35.053 INFO [r-worker-1] a.p.p.c.i.StreamingCrawler - Starting StreamingCrawler #1 ... == document

Optional.empty == document2

Optional.empty == pages [] == fields [] == fields2 [] 2024-03-28 00:34:35.169 INFO [Thread-0] a.p.p.c.s.AbstractPulsarContext - Closing context #1/2 | StaticPulsarContext 2024-03-28 00:34:35.170 INFO [Thread-0] a.p.p.s.AbstractPulsarSession - Session is closed | #1000002 2024-03-28 00:34:35.170 INFO [l-worker-1] a.p.p.s.AbstractPulsarSession - Session is closed | #1000001 2024-03-28 00:34:35.171 INFO [Thread-0] a.p.p.p.b.d.WebDriverPoolManager - Web driver pool manager is closed 2024-03-28 00:34:35.171 INFO [Thread-0] a.p.p.c.f.p.PrivacyManager - Closing privacy contexts ... 2024-03-28 00:34:35.172 INFO [Thread-0] a.p.p.p.b.e.c.BrowserPrivacyContext - Privacy context #03281GEX261 has lived for 1s | success: 0(0.00 pages/s) | small: 0(0.0%) | traffic: 0 B(0 B/s) | tasks: 1 total run: 1 | null 2024-03-28 00:34:35.172 INFO [Thread-0] a.p.p.p.b.e.c.WebDriverContext - Shutdown the underlying layer immediately 2024-03-28 00:34:35.265 INFO [Thread-2] a.p.p.c.Runtimes - Exit | AAA\xxx25204 22484 2024-03-27T16:34:34.564Z PT0.109375S

qpcyet commented 8 months ago

@kuekiko Hello, I have the same issue. Any solution and advice of this problem, Very thankful!

kuekiko commented 8 months ago

@kuekiko Hello, I have the same issue. Any solution and advice of this problem, Very thankful!

Setting JVM runtime arguments : --add-opens java.base/java.lang=ALL-UNNAMED

qpcyet commented 7 months ago

@kuekiko Hello, I have the same issue. Any solution and advice of this problem, Very thankful!

Setting JVM runtime arguments : --add-opens java.base/java.lang=ALL-UNNAMED

This work for me Thanks a lot