search

LouisEchard / ML_MiniProjects

Personal db
0 stars 0 forks source link

function to load data more efficiently #22

Open LouisEchard opened 6 years ago

LouisEchard commented 6 years ago

The data is too big, too disparate. Create a function while cleaning variables to load the right amount of diverse data.

LouisEchard commented 6 years ago

Wait for new RAM

LouisEchard commented 6 years ago

Use multithreading when new ram comes :

from multiprocessing.dummy import Pool as ThreadPool pool = ThreadPool(4) results = pool.map(my_function, my_array)