search

sryza / aas

Code to accompany Advanced Analytics with Spark from O'Reilly Media
Other
1.52k stars 1.03k forks source link

Increased maven memory for build project #143

Open zizu1985 opened 3 years ago

zizu1985 commented 3 years ago

Hi,

Problem appeared with chapter 9 data. I had to use export MAVEN_OPTS="-Xmx8192m -XX:MaxPermSize=256m" to be able to build entire solution.

With lower heap size I've ended with below error I wonder if it makes sens to add information about memory requirement for build in some README.

[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Advanced Analytics with Spark 2.0.0 ................ SUCCESS [ 2.847 s] [INFO] Introduction to Data Analysis with Scala and Spark 2.0.0 SUCCESS [ 6.911 s] [INFO] Recommender Engines with Audioscrobbler data 2.0.0 . SUCCESS [ 0.210 s] [INFO] Covtype with Random Decision Forests 2.0.0 ......... SUCCESS [ 0.194 s] [INFO] Anomaly Detection with K-means 2.0.0 ............... SUCCESS [ 0.229 s] [INFO] Wikipedia Latent Semantic Analysis 2.0.0 ........... SUCCESS [ 20.339 s] [INFO] Network Analysis with GraphX 2.0.0 ................. SUCCESS [ 2.014 s] [INFO] Temporal and Geospatial Analysis 2.0.0 ............. SUCCESS [ 0.361 s] [INFO] Value at Risk through Monte Carlo Simulation 2.0.0 . FAILURE [06:44 min] [INFO] Genomics Analysis with ADAM 2.0.0 .................. SKIPPED [INFO] Simple Spark Project 0.1.0 ......................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 07:20 min [INFO] Finished at: 2021-05-19T06:25:27-07:00 [INFO] ------------------------------------------------------------------------ [ERROR] Java heap space -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError

srowen commented 3 years ago

Are you just trying to build? I don't think it should take that much memory but the default may be too low, sure. Is there any way to set that in the build itself rather than an env variable?