search

brando90 / pytorch-meta-dataset

A non-official 100% PyTorch implementation of META-DATASET benchmark for few-shot classification
0 stars 0 forks source link

why don't we need make index? #8

Closed brando90 closed 1 year ago

brando90 commented 1 year ago

https://github.com/brando90/pytorch-meta-dataset/blob/hdb/make_index_files.sh 

data_path=$1
all_sources="ilsvrc_2012 aircraft cu_birds dtd fungi mscoco omniglot quickdraw traffic_sign vgg_flower"

for source in ${all_sources}
do
    source_path=${data_path}/${source}
    find ${source_path} -name '*.tfrecords' -type f \
                        -exec sh -c 'python3 -m pytorch_meta_dataset.tfrecord.tools.tfrecord2idx $2 ${2%.tfrecords}.index' sh ${source_path} {} \;
done
brando90 commented 1 year ago

https://github.com/brando90/pytorch-meta-dataset/issues/7

brando90 commented 1 year ago

can u find the url that mentioned to do this?

patricks-lab commented 1 year ago

Yeah section 1.3 of https://github.com/mboudiaf/pytorch-meta-dataset i actually realized we do need to make index files so sorry I was probably wrong. So you need to make index files AFTER you download the files (records+splits) in meta dataset from the Google research repo.