search

eto-ai / rikai

Parquet-based ML data format optimized for working with unstructured data
https://rikai.readthedocs.io/en/latest/
Apache License 2.0
136 stars 19 forks source link

pyspark use local[*] as master #697

Closed da-tubi closed 1 year ago

da-tubi commented 1 year ago

There are 16 cores on my dev machine.

da-tubi commented 1 year ago

Using local[*] without increasing the driver memory will result in a OOM exception.