search

open-mmlab / mmflow

OpenMMLab optical flow toolbox and benchmark
https://mmflow.readthedocs.io/en/latest/
Apache License 2.0
965 stars 116 forks source link

How to understand "RepeatDataset"? #265

Closed pedroHuang123 closed 1 year ago

pedroHuang123 commented 1 year ago

image This method can reduce the data loading time between epochs.What do it mean?if my dataset is small,it's loading time is long between epochs,so is the args "times " bigger,the better?

MeowZheng commented 1 year ago

hi, I saw you have opened some issues very recently, and would you like to talk on wechat? I think it will be more efficient.

MeowZheng commented 1 year ago

During training, if the dataset is small, and the total training iteration is the same, the worker processes of dataloader will be launched at each epoch, and the time of launching processes is a little long.

If we repeat several times, the dataset will become larger, and the total training iteration is the same, the epoch will become less and the time of launching processes will be acceptable.

pedroHuang123 commented 1 year ago

Yes,thanks,my wechat account is ForeverHolder.

---- Replied Message ---- | From | Miao @.> | | Date | 11/28/2022 14:25 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [open-mmlab/mmflow] How to understand "RepeatDataset"? (Issue #265) |

hi, I saw you have opened some issues very recently, and would you like to talk on wechat? I think it will be more efficient.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>