search

libvips / pyvips

python binding for libvips using cffi
MIT License
649 stars 50 forks source link

Slower first run, even without cache #448

Closed rml31 closed 10 months ago

rml31 commented 10 months ago

Hello,

I want to compare the execution time of a program on a specific (large) set of images, with different VIPS settings, in order to find the most efficient ones. The program uses the VIPS functions such as jpegload, colourspace, reduce, gaussblur, jpegsave, to process the images.

My problem is that once I run the program a first time (duration ~24 seconds), no matter what I try afterwards the next runs are much shorter (~8 seconds). I need to restart the OS to retrieve the ~24s duration.

It looks like a caching occurs and is used after the first run. I tried to deactive the caching in vips with pyvips.cache_set_max(0) but it does not change the durations.

I'm using the Python binding pyvips 2.2.2 (installed with pip) with 8.15.1 (installed with conda). I installed the last version to try the "revalidate" argument in jpegload, but it does not really change the durations.

Do you have any ideas of what could happen, or things that I could try?

jcupitt commented 10 months ago

Hi @rml31,

Is this on linux? It caches the drive contents very aggressively, so you are probably working from RAM rather than storage on the second run.

You can drop the disk cache with eg.:

echo 3 > /proc/sys/vm/drop_caches
rml31 commented 10 months ago

Yes I should have mentionned I'm on linux (Ubuntu 22.04).

Thank you very much for the quick answer, that was it! I can now accurately monitor the differences. Perfect!