how to train my own dataset

[yustaub]

the imagenet dataset is to big, and I want to train with my own dataset , can this repo support train own dataset?very appreciate for your reply!

[nrupatunga]

In order to train on your dataset:

Dataset Preparation

• If your dataset contains only images, refer to https://github.com/nrupatunga/goturn-pytorch/blob/master/src/goturn/dataloaders/imagenet.py

• If your dataset consists of video frames, refer to https://github.com/nrupatunga/goturn-pytorch/blob/master/src/goturn/dataloaders/alov.py

In order to merge both Imagenet and alov, I have used a custom collate_fn in https://github.com/nrupatunga/goturn-pytorch/blob/cd610dd98ca5856ff3944523aa51a058ea7e3c63/src/goturn/dataloaders/goturndataloader.py#L90

Each of the scripts is individually debuggable, please go through each of them in the above order to understand the functionality.

Training Script

• Finally, the curr, prev expected in this line https://github.com/nrupatunga/goturn-pytorch/blob/cd610dd98ca5856ff3944523aa51a058ea7e3c63/src/scripts/train.py#L251 As long as you keep in the same format as now, you should be able to train it on the custom dataset.

• Also, you need to modify train_dataloader and val_dataloader https://github.com/nrupatunga/goturn-pytorch/blob/cd610dd98ca5856ff3944523aa51a058ea7e3c63/src/scripts/train.py#L159 https://github.com/nrupatunga/goturn-pytorch/blob/cd610dd98ca5856ff3944523aa51a058ea7e3c63/src/scripts/train.py#L184

Let me know if you face any issues.

[yustaub]

@nrupatunga thk for your reply, I just find my machine can only support pytorch 1.1 at most( I can not upgrade the driver of the machine),when I use pytorch 1.4 which is in requirements.txt, then cuda can not work, can this repo support pytorch 1.1 ?if it can,where should I change?

[nrupatunga]

Though I haven't tested with PyTorch-1.1, It should work. You just need to install PyTorch 1.1 and try to use it.

You can refer https://pytorch.org/get-started/previous-versions/ for previous versions of PyTorch.

[nrupatunga]

In order to understand the current working of the data loaders, you can use a smaller dataset to understand it, which you can download it here

[yustaub]

@nrupatunga thanks for your quick reply!!.I have install pytorch 1.1 and cuda can work, and I test the repo, it always Please run $source settings.sh from root directory' ,although I have run source settings.sh many times, when I change the pytorch to 1.4 version, it can work but cuda can not support the pytorch1.4.

[nrupatunga]

This might be the case, where you are missing other dependencies. Can you try debugging using pdb.

[yustaub]

ok, thank you very much!!!