This changeset compiles Pillow-SIMD with AVX2 support for vectorized image operations (e.g. resize) and compiles against libjpeg-turbo for faster JPEG decoding.
The speedup is quite significant: the dataloaders can load, decode, and pre-process now like running a knife through butter. This takes considerable load off the CPU(s) and it's easier now to keep (potentially multiple) GPUs busy.
Note: downside is we require a machine with AVX2 support now. For GPU boxes this should be a given, and the 2015 laptop I'm writing this on comes with AVX2, too, so I'm happy to make it a default requirement.
Adding a convenient makefile in a second commit to make interaction with docker more pleasant.
This changeset compiles Pillow-SIMD with AVX2 support for vectorized image operations (e.g. resize) and compiles against libjpeg-turbo for faster JPEG decoding.
The speedup is quite significant: the dataloaders can load, decode, and pre-process now like running a knife through butter. This takes considerable load off the CPU(s) and it's easier now to keep (potentially multiple) GPUs busy.
Note: downside is we require a machine with AVX2 support now. For GPU boxes this should be a given, and the 2015 laptop I'm writing this on comes with AVX2, too, so I'm happy to make it a default requirement.
Adding a convenient makefile in a second commit to make interaction with docker more pleasant.