search

longcw / yolo2-pytorch

YOLOv2 in PyTorch
1.54k stars 421 forks source link

A `torch.utils.data.Dataset` demo #71

Open jinyu121 opened 6 years ago

jinyu121 commented 6 years ago

Hi~

Look at this~ https://gist.github.com/jinyu121/02a505bb2547006ceb3fdbba3bee556c

Works fine (certainly) to me

longcw commented 6 years ago

For anyone who wants to use this dataloader, you can move imdb.parse to collate_fn and convert numpy array to tensor. Then the data processing part will run on multiple processes, and the data will be shared on the memeroy by using tensor. Another solution is to move _im_processor to __get_item__.

ztyxd commented 6 years ago

@jinyu121 this web https://gist.github.com/jinyu121/02a505bb2547006ceb3fdbba3bee556c may loss efficacy. Can u give me another web please? I really want to konw how to use the torch.utils.data.Dataset instead of multiprocessing

hixiaye commented 5 years ago

For anyone who wants to use this dataloader, you can move imdb.parse to collate_fn and convert numpy array to tensor. Then the data processing part will run on multiple processes, and the data will be shared on the memeroy by using tensor. Another solution is to move _im_processor to __get_item__.

Hi, I don't understand what the moveimdb.parsetocollate_fn meaning. And where I should convert numpy array to tensor? Could you give me some advices? thx

hixiaye commented 5 years ago

Hi~

Look at this~ https://gist.github.com/jinyu121/02a505bb2547006ceb3fdbba3bee556c

Works fine (certainly) to me

Hi, I don't understand what the moveimdb.parsetocollate_fn meaning. And where I should convert numpy array to tensor? Could you give me some advices? thx