Train model takes too many hours, how to optimize it.

juliandewit / kaggle_ndsb2017

Kaggle datascience bowl 2017

MIT License

623 stars 290 forks source link

Train model takes too many hours, how to optimize it. #63

Open tangguihua opened 5 years ago

tangguihua commented 5 years ago

Hello Julian, thanks for your great work to predict cancer. Here I have meet a problem when I want to study your project. I use all the data(ndsb and Luna) to re-train the model by using your code. while the ETA shows it takes about 244 hours. My configuration is interal storage: 251G and GPU storage is 12G. how can I reduce the time to train the model, can you give me some advice. Thanks.

juliandewit commented 5 years ago

hello, I noticed that newer keras versions have different parameters for the "fit" function. Where I gave the number of training examples as parameter, I believe it's now the number of batches.. Hence the huge train time.. You have to look it up yourself but it's somewhere in this issues list.