FileSystem#listStatus error , Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded ,

crs4 / pydoop

A Python MapReduce and HDFS API for Hadoop

Apache License 2.0

237 stars 59 forks source link

FileSystem#listStatus error , Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded , #326

Closed ghost closed 6 years ago

ghost commented 6 years ago

How to deal with this error, how to modify jvm HEAPSIZE

ghost commented 6 years ago

Resolved to set up the "LIBHDFS_OPTS" environment variable export LIBHDFS_OPTS="-server -Xss2m -Xms2000m -Xmx2000m -Xmn800m -XX:PermSize=64m -XX:MaxPermSize=256m"