search

timeseriesAI / tsai

Time series Timeseries Deep Learning Machine Learning Python Pytorch fastai | State-of-the-art Deep Learning library for Time Series and Sequences in Pytorch / fastai
https://timeseriesai.github.io/tsai/
Apache License 2.0
4.95k stars 625 forks source link

Partial fit vs fine tune? #749

Closed alitirmizi23 closed 1 year ago

alitirmizi23 commented 1 year ago

Hi. I was wondering what would be the better setting for incremental learning? like I want to train my models as new data come in - so that they keep their previous memory as well as learn any new representations on top. Would fine tune work or partial fit? Or do I just snapshot my training data from before, and as new data come in, I just add that to the snapshot-ed data and re-train models from scratch?

oguiza commented 1 year ago

Hi @alitirmizi23, it's difficult to know. A key factor though is the dataset size. If your dataset is not large, you may get better results retraining the model from scratch. If it's large, you may want to "fine-tune" it. Fastai offers 2 options to do it: learn.fine_tune or learn.fit_one_cycle. The first one only trains the last layers, while the latter trains all of them. In my experience, I've usually achieved better results when training all layers.

oguiza commented 1 year ago

Closed due to lack of response.