libffcv / ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)
https://ffcv.io
Apache License 2.0
2.84k stars 178 forks source link

Indexing #343

Open muradtuk opened 1 year ago

muradtuk commented 1 year ago

Dear authors,

I wanted to ask the following: Is it possible to get the original indices of the instance in each batch during traversal over the data (through the data loader)? In other words, is it possible to have something along the lines of

for images, targets, indices in loader:

without changing the beton file? Or is it possible to retrieve such indices from the loader?

Thanks in advance, M.T.

andrewilyas commented 1 year ago

Hi @muradtuk ! This should be possible - I'm not sure off the top of my head what the cleanest way to do it is, but you can definitely create a Pipeline that applies a transform that simply replaces the data with the index (see https://docs.ffcv.io/ffcv_examples/transform_with_inds.html) - this might be kind of wasteful but probably wouldn't slow things down too much.