commoncrawl / news-crawl

News crawling with StormCrawler - stores content as WARC
Apache License 2.0
323 stars 35 forks source link

mvn clean package fails on Mac on Apple M1 Pro chip #56

Closed raphaelzhou1 closed 1 year ago

raphaelzhou1 commented 1 year ago

Hi friends,

I am using news-crawl for academic research but am unable to set it up on my Mac computer. But I encountered build error and would really appreciate some help!

Environment: MacOS Monterey 12.2.1, Apple M1 Pro javac 20.0.2 openjdk 20.0.2 2023-07-18 : OpenJDK Runtime Environment (build 20.0.2+9-78); OpenJDK 64-Bit Server VM (build 20.0.2+9-78, mixed mode, sharing) Apache Maven 3.9.3 (21122926829f1ead511c958d89bd2f672198ae9f) Apache Storm 2.4.0 elasticsearch 8.8.2 On bash shell news-crawler commit 4194f9c

Steps to reproduce:

  1. I installed elasticsearch, Apache Storm; started elasticsearch; built bin/ES_IndexInit.sh
  2. Cloned news-crawler, direct into directory and put "mvn clean package"

Expected result

  1. Successful build

Actual result:

  1. Screenshot

Screen Shot 2023-07-22 at 4 42 53 PM Screen Shot 2023-07-22 at 4 43 04 PM Screen Shot 2023-07-22 at 4 43 20 PM

I am new to Java development. Please let me know if I can provide additional context! Thanks

wumpus commented 1 year ago

Our typical news crawl user just downloads the WARCs we generate. We don't test this software outside of our environment.

However, if you want some clues from someone who also is not that familiar with Java, in the screenshots there's a clue near the end of all of that confusing output:

Please refer to /Users/user/Desktop/Coding/AI/Context/financial_news_extractor/news-crawl/target/surefire-reports for individual test results

sebastian-nagel commented 1 year ago

Hi @raphaelzhou1, news-crawl is built with Java 8. Sorry, this information wasn't given in the README (updated now) but only in the pom.xml. Please, install JDK 1.8, make JAVA_HOME point to it and try again. I'm also not 100% sure whether Apache Storm 1.2.3 or 1.2.4 runs on ARM. However, I expect that development and testing should be possible.

raphaelzhou1 commented 1 year ago

Thanks!

On Tue, Jul 25, 2023 at 9:56 AM Sebastian Nagel @.***> wrote:

Hi @raphaelzhou1 https://github.com/raphaelzhou1, news-crawl is built with Java 8. Sorry, this information wasn't given in the README (updated now) but only in the pom.xml. Please, install JDK 1.8, make JAVA_HOME point to it and try again. I'm also not 100% sure whether Apache Storm https://storm.apache.org/ 1.2.3 or 1.2.4 runs on ARM. However, I expect that development and testing should be possible.

— Reply to this email directly, view it on GitHub https://github.com/commoncrawl/news-crawl/issues/56#issuecomment-1649897164, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASSCYKDNYRWMQSNAZQCV6HLXR7GB7ANCNFSM6AAAAAA2UBNUGE . You are receiving this because you were mentioned.Message ID: @.***>

-- 国无道,处贱不闷,贫而乐 Our life is frittered away by detail. Simplify, simplify اطلب العلم ولو كان في الصين

wumpus commented 1 year ago

@raphaelzhou1 Did Java 8 work for you? Can we close this issue?

raphaelzhou1 commented 1 year ago

Yes! Thanks! ᐧ

On Tue, Aug 15, 2023 at 8:43 AM Greg Lindahl @.***> wrote:

@raphaelzhou1 https://github.com/raphaelzhou1 Did Java 8 work for you? Can we close this issue?

— Reply to this email directly, view it on GitHub https://github.com/commoncrawl/news-crawl/issues/56#issuecomment-1679165943, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASSCYKHNFSIR6XBY764LDSLXVOKKPANCNFSM6AAAAAA2UBNUGE . You are receiving this because you were mentioned.Message ID: @.***>

-- 国无道,处贱不闷,贫而乐 Our life is frittered away by detail. Simplify, simplify اطلب العلم ولو كان في الصين