How to train a video model?

[playma]

Thank you for the great work!

I have some questions

I want to use DeOldify to train a video model like pix2pix. Which notebook should I use? I found that it did not convert color images to black and white image as training data in ColorizeTrainingVideo.ipynb, but in ColorizeTrainingArtistic.ipynb and ColorizeTrainingStable.ipynb

Should I train ColorizeTrainingArtistic.ipynb or ColorizeTrainingStable.ipynb first, and then train the ColorizeTrainingVideo.ipynb?

[jantic]

You have a few options. If you want to start completely from scratch, then you should train ColorizeTrainingStable.ipynb. This is stated at the top of ColorizeTrainingVideo.ipynb:

It's assumed that there's a pretrained generator from the ColorizeTrainingStable notebook available at the specified path.

But this isn't the only way to get a pretrained generator. You can also download its weights from the readme, in the section that looks like this:

Pretrain Only Generator Weights artistic stable video

Pretrain Only Critic Weights artistic stable video

Download the stable ones if you want to start the ColorizeTrainingVideo.ipynb exactly like you would training ColorizeTrainingStable.ipynb on your own first. Or you download the video weights to get straight to GAN training and bypass pre-training on noise augmentation.

[hammadalibutt]

I am newbie. please mention how to place data in which format or folder. Please can you explain training process in detail ... or add new readme file or manual. because i have got an error on this line learn_gen.fit_one_cycle(1, pct_start=pct_start, max_lr=slice(5e-8,5e-5))

---

StopIteration Traceback (most recent call last)

in ----> 1 learn_gen.fit_one_cycle(1, pct_start=pct_start, max_lr=slice(5e-8,5e-5)) ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, final_div, wd, callbacks, tot_epochs, start_epoch) 20 callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor, pct_start=pct_start, 21 final_div=final_div, tot_epochs=tot_epochs, start_epoch=start_epoch)) ---> 22 learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks) 23 24 def lr_find(learn:Learner, start_lr:Floats=1e-7, end_lr:Floats=10, num_it:int=100, stop_div:bool=True, wd:float=None): ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks) 192 if not getattr(self, 'opt', False): self.create_opt(lr, wd) 193 else: self.opt.lr,self.opt.wd = lr,wd --> 194 callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks) 195 if defaults.extra_callbacks is not None: callbacks += defaults.extra_callbacks 196 fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks) ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/basic_train.py in (.0) 192 if not getattr(self, 'opt', False): self.create_opt(lr, wd) 193 else: self.opt.lr,self.opt.wd = lr,wd --> 194 callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks) 195 if defaults.extra_callbacks is not None: callbacks += defaults.extra_callbacks 196 fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks) ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/callbacks/tensorboard.py in __init__(self, learn, base_dir, name, loss_iters, hist_iters, stats_iters, visual_iters) 176 visual_iters:int=100): 177 super().__init__(learn=learn, base_dir=base_dir, name=name, loss_iters=loss_iters, hist_iters=hist_iters, --> 178 stats_iters=stats_iters) 179 self.visual_iters = visual_iters 180 self.img_gen_vis = ImageTBWriter() ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/callbacks/tensorboard.py in __init__(self, learn, base_dir, name, loss_iters, hist_iters, stats_iters) 36 self.data = None 37 self.metrics_root = '/metrics/' ---> 38 self._update_batches_if_needed() 39 40 def _get_new_batch(self, ds_type:DatasetType)->Collection[Tensor]: ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/callbacks/tensorboard.py in _update_batches_if_needed(self) 48 if not update_batches: return 49 self.data = self.learn.data ---> 50 self.trn_batch = self._get_new_batch(ds_type=DatasetType.Train) 51 self.val_batch = self._get_new_batch(ds_type=DatasetType.Valid) 52 ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/callbacks/tensorboard.py in _get_new_batch(self, ds_type) 40 def _get_new_batch(self, ds_type:DatasetType)->Collection[Tensor]: 41 "Retrieves new batch of DatasetType, and detaches it." ---> 42 return self.learn.data.one_batch(ds_type=ds_type, detach=True, denorm=False, cpu=False) 43 44 def _update_batches_if_needed(self)->None: ~/anaconda3/envs/deoldify/lib/python3.7/site-packages/fastai/basic_data.py in one_batch(self, ds_type, detach, denorm, cpu) 166 w = self.num_workers 167 self.num_workers = 0 --> 168 try: x,y = next(iter(dl)) 169 finally: self.num_workers = w 170 if detach: x,y = to_detach(x,cpu=cpu),to_detach(y,cpu=cpu) StopIteration:

[jantic]

@hammadalibutt Don't take this the wrong way but I can tell you right now that this won't be the end of the list of questions you'll wind up running into if you're asking me this. I'm not saying this to be mean but to be practical: Basic skills (debugging, reading code) are in order first before you can tackle running this training. I just don't have the time to walk you through all this.

[hammadalibutt]

Yes, you are right, but sorry for interruption again. Please, can you help me how to place my data set in your algorithm? another question is video trainer use image datasets?

[jantic]

I've already said no dude. I'm not sure what you think changed between now and then....