search

pytorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration
https://pytorch.org
Other
83.03k stars 22.39k forks source link

drop_last (bool, optional) - more options needed #17075

Closed petertarabawrai closed 5 years ago

petertarabawrai commented 5 years ago

drop_last (bool, optional) – set to True to drop the last incomplete batch, if the dataset size is not divisible by the batch size. If False and the size of dataset is not divisible by the batch size, then the last batch will be smaller. (default: False)

What if I want to redistribute data from last incomplete batch into other batches, so I don't loose any training (or testing) data?

ssnl commented 5 years ago
  1. If you want to discard the notion of epoch, use a custom batch_sampler.
  2. Do it outside dataloader.
petertarabawrai commented 5 years ago

Doesn't notion give more freedom than definition?

ssnl commented 5 years ago

@petertarabawrai What you you mean by notion?

petertarabawrai commented 5 years ago

you said 'if i want to discard the notion of epoch', which I don't. By notion I understand something lighter than theory. what did you mean by notion?

ssnl commented 5 years ago

Notion as similar to concept.

If you are not discarding the concept of epoch, then you basically have to accept that there is only one iterator for each pass/epoch on the dataset, and that you can’t merge data across epochs/iterators.

petertarabawrai commented 5 years ago

I have following order in my head: idea, notion, concept, definition. Idea can be dumb, notion can be whimsy, concept sounds more serious, definition is precise. We don't even have to agree to continue main discussion.

  1. I finally get why you said 'if I want to discard the notion of epoch'. Maybe I could forget about epoch in order to not lose any data.
  2. If iterator knows in advance amount of all samples, it can plan accordingly. If you say iterator does not know amount of all samples in advance, that is only for the first epoch. In all following epochs it can plan accordingly.