Added conf attribute to solve Java heap issue

dsp-uga / andromeda

This repository contains a Naive Bayes classifier implemented on document classification which is completed on CSCI 8360, Data Science Practicum at the University of Georgia, Spring 2018.

MIT License

4 stars 1 forks source link

Added conf attribute to solve Java heap issue #41

Closed nihalsoans91 closed 6 years ago

nihalsoans91 commented 6 years ago

args = vars(parser.parse_args())
conf = SparkConf().setAppName("App")
conf = (conf.setMaster('local[*]')
    .set('spark.executor.memory', '4G')
    .set('spark.driver.memory', '35G')
sc = SparkContext(conf=conf