search

platonai / PulsarRPA

Automate webpages at scale, scrape web data completely and accurately with high performance, distributed AI-RPA.
Apache License 2.0
778 stars 118 forks source link

build fail at ubuntu 18.04 #8

Closed elife33 closed 2 years ago

elife33 commented 2 years ago

bin/build.sh ->

[INFO] Running ai.platon.pulsar.common.sql.TestSQLTemplate [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 s - in ai.platon.pulsar.common.sql.TestSQLTemplate [INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] TestAppRuntimes.testDeleteBrokenSymbolicLinksUsingJava:92->testDeleteBrokenSymbolicLinksUsingJava$lambda-14:92 » FileSystem [INFO] [ERROR] Tests run: 149, Failures: 0, Errors: 1, Skipped: 8 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Pulsar 1.8.4-SNAPSHOT: [INFO] [INFO] Pulsar ............................................. SUCCESS [ 1.669 s] [INFO] Pulsar Common ...................................... FAILURE [ 44.509 s] [INFO] Pulsar Third ....................................... SKIPPED

platonai commented 2 years ago

Can you paste the detailed error message?

Or you can just delete the file to move on: rm pulsar-common/src/test/kotlin/ai/platon/pulsar/common/TestAppRuntimes.kt

elife33 commented 2 years ago

Thanks! After delete the file to move on, [INFO] Reactor Summary for Pulsar 1.8.4-SNAPSHOT: [INFO] [INFO] Pulsar ............................................. SUCCESS [ 1.769 s] [INFO] Pulsar Common ...................................... SUCCESS [ 27.686 s] [INFO] Pulsar Third ....................................... SUCCESS [ 0.027 s] [INFO] Pulsar Jsoup ....................................... SUCCESS [ 1.818 s] [INFO] Pulsar Persist ..................................... SUCCESS [ 7.388 s] [INFO] Pulsar Boilerpipe .................................. SUCCESS [ 2.196 s] [INFO] Pulsar DOM ......................................... SUCCESS [ 6.343 s] [INFO] Pulsar Tools ....................................... SUCCESS [ 0.023 s] [INFO] Pulsar Browser ..................................... SUCCESS [ 3.400 s] [INFO] Pulsar QL Common ................................... SUCCESS [ 1.214 s] [INFO] Pulsar Skeleton .................................... SUCCESS [ 19.970 s] [INFO] Pulsar Resources ................................... SUCCESS [ 0.076 s] [INFO] Pulsar Plugins ..................................... SUCCESS [ 0.017 s] [INFO] Pulsar Filter ...................................... SUCCESS [ 3.101 s] [INFO] Pulsar Parse ....................................... SUCCESS [ 2.990 s] [INFO] Pulsar Protocol .................................... SUCCESS [ 4.729 s] [INFO] Pulsar Scoring ..................................... SUCCESS [ 1.147 s] [INFO] Pulsar Index ....................................... SUCCESS [ 2.530 s] [INFO] Pulsar Schedule .................................... SUCCESS [ 0.973 s] [INFO] Pulsar Spring Support .............................. SUCCESS [ 0.016 s] [INFO] Pulsar Beans ....................................... SUCCESS [ 2.200 s] [INFO] Pulsar QL .......................................... SUCCESS [ 15.444 s] [INFO] Pulsar All-in-one .................................. FAILURE [ 18.181 s] [INFO] Pulsar Boot ........................................ SKIPPED [INFO] Pulsar Rest ........................................ SKIPPED [INFO] Pulsar Client ...................................... SKIPPED [INFO] Pulsar App ......................................... SKIPPED [INFO] Pulsar App Resources ............................... SKIPPED [INFO] Pulsar App Common Resources ........................ SKIPPED [INFO] Pulsar Tests ....................................... SKIPPED [INFO] Pulsar QA .......................................... SKIPPED [INFO] Pulsar Sites Support ............................... SKIPPED [INFO] Pulsar Site Amazon ................................. SKIPPED [INFO] Pulsar Site CN Gov ................................. SKIPPED [INFO] Pulsar Master ...................................... SKIPPED [INFO] Pulsar Examples .................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:03 min [INFO] Finished at: 2022-04-24T18:21:38+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.codehaus.mojo:flatten-maven-plugin:1.2.7:flatten (flatten) on project pulsa r-all: failed to create a clean pom: unable to create flattened dependencies: caught exception when flattening dependencies: Failed to read artifact descriptor for com.jamesmurty.utils:java-xmlbuilder::0.4: Could not find artifact org.sonatype.oss:oss-parent:pom:3 in maven2-repository.dev.java.net (http://download.java.net/maven /2/) -> [Help 1] ...... Downloading from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/ai/platon/pulsar/pulsar-app-common-resources/1.8.4-SNAPSHOT/pulsar-app-common-resources-1.8.4-SNAPSHOT.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 12.367 s [INFO] Finished at: 2022-04-24T18:21:52+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project pulsar-master: Could not resolve dependencies for project ai.platon.pulsar:pulsar-master:jar:1.8.4-SNAPSHOT: The following artifacts could not be resolved: ai.platon.pulsar:pulsar-all:jar:1.8.4-SNAPSHOT, ai.platon.pulsar:pulsar-rest:jar:1.8.4-SNAPSHOT, ai.platon.pulsar:pulsar-app-common-resources:jar:1.8.4-SNAPSHOT: Could not find artifact ai.platon.pulsar:pulsar-all:jar:1.8.4-SNAPSHOT in Central (https://repo1.maven.org/maven2/) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException /home/elife/work/lab/pulsar cat: /tmp/pulsar-app/pulsar-master.classpath: No such file or directory Error: Could not find or load main class ai.platon.pulsar.app.master.PulsarMasterKt Caused by: java.lang.ClassNotFoundException: ai.platon.pulsar.app.master.PulsarMasterKt elife@rtx:~/work/lab/pulsar$

elife33 commented 2 years ago

elife@rtx:~/work/lab/pulsar$ mvn compile -rf :pulsar-all WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release [INFO] Scanning for projects... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] Pulsar All-in-one [jar] [INFO] Pulsar Boot [jar] [INFO] Pulsar Rest [jar] [INFO] Pulsar Client [jar] [INFO] Pulsar App [pom] [INFO] Pulsar App Resources [pom] [INFO] Pulsar App Common Resources [jar] [INFO] Pulsar Tests [jar] [INFO] Pulsar QA [jar] [INFO] Pulsar Sites Support [pom] [INFO] Pulsar Site Amazon [jar] [INFO] Pulsar Site CN Gov [jar] [INFO] Pulsar Master [jar] [INFO] Pulsar Examples [jar] [INFO] [INFO] --------------------< ai.platon.pulsar:pulsar-all >--------------------- [INFO] Building Pulsar All-in-one 1.8.4-SNAPSHOT [1/14] [INFO] --------------------------------[ jar ]--------------------------------- [WARNING] The POM for ai.platon.pulsar:pulsar-h2:jar:1.4.196 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details [INFO] [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-maven-version) @ pulsar-all --- [INFO] [INFO] --- duplicate-finder-maven-plugin:1.3.0:check (default) @ pulsar-all --- [INFO] Skipping duplicate-finder execution! [INFO] [INFO] --- maven-remote-resources-plugin:1.7.0:process (process-resource-bundles) @ pulsar-all --- [INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4 [INFO] Copying 3 resources from 1 bundle. [WARNING] Invalid POM for ai.platon.pulsar:pulsar-h2:jar:1.4.196, transitive dependencies (if any) will not be available, enable debug logging for more details [WARNING] Invalid POM for ai.platon.pulsar:pulsar-h2:jar:1.4.196, transitive dependencies (if any) will not be available, enable debug logging for more details [WARNING] Invalid project model for artifact [pulsar-h2:ai.platon.pulsar:1.4.196]. It will be ignored by the remote resources Mojo. [INFO] [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ pulsar-all --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/elife/work/Trading/pulsar/pulsar-all/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- flatten-maven-plugin:1.2.7:flatten (flatten) @ pulsar-all --- [INFO] Generating flattened POM of project ai.platon.pulsar:pulsar-all:jar:1.8.4-SNAPSHOT... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Pulsar All-in-one 1.8.4-SNAPSHOT: [INFO] [INFO] Pulsar All-in-one .................................. FAILURE [ 4.256 s] [INFO] Pulsar Boot ........................................ SKIPPED [INFO] Pulsar Rest ........................................ SKIPPED [INFO] Pulsar Client ...................................... SKIPPED [INFO] Pulsar App ......................................... SKIPPED [INFO] Pulsar App Resources ............................... SKIPPED [INFO] Pulsar App Common Resources ........................ SKIPPED [INFO] Pulsar Tests ....................................... SKIPPED [INFO] Pulsar QA .......................................... SKIPPED [INFO] Pulsar Sites Support ............................... SKIPPED [INFO] Pulsar Site Amazon ................................. SKIPPED [INFO] Pulsar Site CN Gov ................................. SKIPPED [INFO] Pulsar Master ...................................... SKIPPED [INFO] Pulsar Examples .................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.572 s [INFO] Finished at: 2022-04-24T19:21:55+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.codehaus.mojo:flatten-maven-plugin:1.2.7:flatten (flatten) on project pulsar-all: failed to create a clean pom: unable to create flattened dependencies: caught exception when flattening dependencies: Failed to read artifact descriptor for org.mortbay.jetty:jetty::6.1.26: Could not find artifact org.mortbay.jetty:jetty-parent:pom:10 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException elife@rtx:~/work/lab/pulsar$

elife33 commented 2 years ago

I pulled the latest commit 54989ef511049c70ac7542ebb7d6e7586027d8fe and bin/build.sh -> [INFO] Pulsar Skeleton .................................... SUCCESS [ 30.322 s] [INFO] Pulsar Resources ................................... SUCCESS [ 0.060 s] [INFO] Pulsar Plugins ..................................... SUCCESS [ 0.020 s] [INFO] Pulsar Filter ...................................... SUCCESS [ 6.097 s] [INFO] Pulsar Parse ....................................... SUCCESS [ 5.759 s] [INFO] Pulsar Protocol .................................... FAILURE [ 6.060 s] [INFO] Pulsar Scoring ..................................... SKIPPED [INFO] Pulsar Index ....................................... SKIPPED [INFO] Pulsar Schedule .................................... SKIPPED [INFO] Pulsar Spring Support .............................. SKIPPED [INFO] Pulsar Beans ....................................... SKIPPED [INFO] Pulsar QL .......................................... SKIPPED [INFO] Pulsar All-in-one .................................. SKIPPED [INFO] Pulsar Boot ........................................ SKIPPED [INFO] Pulsar Rest ........................................ SKIPPED [INFO] Pulsar Client ...................................... SKIPPED [INFO] Pulsar App ......................................... SKIPPED [INFO] Pulsar App Resources ............................... SKIPPED [INFO] Pulsar App Common Resources ........................ SKIPPED [INFO] Pulsar Tests ....................................... SKIPPED [INFO] Pulsar QA .......................................... SKIPPED [INFO] Pulsar Sites Support ............................... SKIPPED [INFO] Pulsar Site Amazon ................................. SKIPPED [INFO] Pulsar Site CN Gov ................................. SKIPPED [INFO] Pulsar Master ...................................... SKIPPED [INFO] Pulsar Examples .................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:59 min [INFO] Finished at: 2022-04-24T19:54:09+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test (default-test) on project pulsar-protocol: There are test failures. [ERROR] [ERROR] Please refer to /home/elife/work/Trading/pulsar/pulsar-plugins/pulsar-protocol/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :pulsar-protocol elife@rtx:~/work/Trading/pulsar$

elife33 commented 2 years ago

Test set: ai.platon.pulsar.protocol.browser.emulator.context.BrowserContextTests

Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.717 s <<< FAILURE! - in ai.platon.pulsar.protocol.browser.emulator.context.BrowserContextTests When run tasks the contexts rotates(ai.platon.pulsar.protocol.browser.emulator.context.BrowserContextTests) Time elapsed: 0.691 s <<< ERROR! ai.platon.pulsar.protocol.browser.DriverLaunchException: Failed to create web driver | pulsar_chrome at ai.platon.pulsar.protocol.browser.emulator.context.BrowserContextTests.When run tasks the contexts rotates(BrowserContextTests.kt:48)

platonai commented 2 years ago

I saw the complaint: ”Failed to create web driver“, it's usually caused by the missing of a google chrome installation. Pulsar searches the following paths for google chrome:

val CHROME_BINARY_SEARCH_PATHS = arrayOf(
    "/usr/bin/google-chrome-stable",
    "/usr/bin/google-chrome",
    "/opt/google/chrome/chrome",
    "C:/Program Files (x86)/Google/Chrome/Application/chrome.exe",
    "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
    "/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary",
    "/Applications/Chromium.app/Contents/MacOS/Chromium",
    "/usr/bin/chromium",
    "/usr/bin/chromium-browser"
)
platonai commented 2 years ago

Try

bin/build-run.sh

to install google chrome automatically.

Thanks for the bug reporting, we will add code to install chrome in bin/build.sh in the next version.

elife33 commented 2 years ago

use bin/build-run.sh then [INFO] Pulsar QL .......................................... SUCCESS [ 15.444 s]
[INFO] Pulsar All-in-one .................................. FAILURE [ 18.181 s]
[INFO] Pulsar Boot ........................................ SKIPPED
[INFO] Pulsar Rest ........................................ SKIPPED
[INFO] Pulsar Client ...................................... SKIPPED
[INFO] Pulsar App ......................................... SKIPPED
[INFO] Pulsar App Resources ............................... SKIPPED
[INFO] Pulsar App Common Resources ........................ SKIPPED
[INFO] Pulsar Tests ....................................... SKIPPED
[INFO] Pulsar QA .......................................... SKIPPED
[INFO] Pulsar Sites Support ............................... SKIPPED
[INFO] Pulsar Site Amazon ................................. SKIPPED
[INFO] Pulsar Site CN Gov ................................. SKIPPED
[INFO] Pulsar Master ...................................... SKIPPED
[INFO] Pulsar Examples .................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:03 min
[INFO] Finished at: 2022-04-24T18:21:38+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:flatten-maven-plugin:1.2.7:flatten (flatten) on project pulsa r-all: failed to create a clean pom: unable to create flattened dependencies: caught exception when flattening dependencies: Failed to read artifact descriptor for com.jamesmurty.utils:java-xmlbuilder::0.4: Could not fin d artifact org.sonatype.oss:oss-parent:pom:3 in maven2-repository.dev.java.net (http://download.java.net/maven /2/) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :pulsar-all

elife33 commented 2 years ago

elife@rtx:~/work/Trading/pulsar$ mvn --version Apache Maven 3.6.0 Maven home: /usr/share/maven Java version: 11.0.14.1, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64 Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "5.4.0-91-generic", arch: "amd64", family: "unix" elife@rtx:~/work/Trading/pulsar$

platonai commented 2 years ago

Seems to be caused by a problem with maven source, what about retrying the building?

== "Could not fin d artifact org.sonatype.oss:oss-parent:pom:3 in maven2-repository.dev.java.net (http://download.java.net/maven /2/)"

elife33 commented 2 years ago

Seems to be caused by a problem with maven source, what about retrying the building?

== "Could not fin d artifact org.sonatype.oss:oss-parent:pom:3 in maven2-repository.dev.java.net (http://download.java.net/maven /2/)"

I upgraded Maven from the default 3.6.0 to latest 3.8.5, then pulsar built successfully! Thanks a lot!