ViCCo-Group / thingsvision

Python package for extracting representations from state-of-the-art computer vision models
https://vicco-group.github.io/thingsvision/
MIT License
157 stars 21 forks source link

Implement low-memory option for feature extraction #74

Closed andropar closed 2 years ago

andropar commented 2 years ago

Right now, features are stored in RAM when calling extractor.extract_features(...), which quickly makes memory usage explode. It might be a good idea to enable a low-memory option that just writes features to disk immediately.

Maybe add an output path to the extractor extractor = Extractor(out_path='...') and an extra flag extractor.extract_features(store_to_disk=True)?