search

ibm-granite / granite-tsfm

Foundation Models for Time Series
Apache License 2.0
425 stars 186 forks source link

Fine-tuning time consumption #184

Open czczccz opened 3 weeks ago

czczccz commented 3 weeks ago

Hi! I found the model infers extremely fast but takes a long time when being fine-tuned. Training on hundreds of samples could take 1 hour. Is this normal?

wgifford commented 2 weeks ago

Can you provide more information about what dataset you are training on, what hardware, and general details of the trainer config? If the dataset is multivariate with many dimensions training will take longer. For example:

The fine-tuning in this example: https://github.com/ibm-granite-community/granite-timeseries-cookbook/blob/main/recipes/Time_Series/Few-shot_Finetuning_and_Evaluation.ipynb

For that notebook, with batch size = 64, it takes about 26s (on my MacBook Pro) per epoch. For reference this dataset has 12 inputs and 931 records.