hoytech / vmtouch

Portable file system cache diagnostics and control
https://hoytech.com/vmtouch/
BSD 3-Clause "New" or "Revised" License
1.79k stars 210 forks source link

Is it possible to crawl multiple ranges and append them to one locked file? #88

Open mgutt opened 3 years ago

mgutt commented 3 years ago

This works to preload head and tail of one file, but it is not persistent:

head -c 60000000 "$file" > /dev/null
tail -c 1000000 "$file" > /dev/null

Now I tried the same through vmtouch as follows:

size=$(stat -c%s $file)
vmtouch -l -p 0-60000000 "$file" &
sleep 2
start_byte="$(($size-1000000))"
vmtouch -l -p $start_byte-$size "$file" &

But I found out that the second command does not append the crawled range to the already locked file. Instead it seems to overwrite the existing one. I mean not really. I have still two different processes for the same file:

ps aux | awk '{print $6/1024 " MB\t\t" $11}' | sort -n | grep vmtouch 2020-10-14 18_45_50

Is there an other combination of commands/flags possible that allows me to lock both file parts?

hoytech commented 3 years ago

I tried a quick test and it seems to work how I expect. I have a 6.7 GB file. In window 1:

$ sudo vmtouch -p 0-100M -l file
LOCKED 25600 pages (100M)

In window 2:

$ sudo vmtouch -p 6.6G- -l file
LOCKED 28778 pages (112M)

Window 3:

$ vmtouch -e file
           Files: 1
     Directories: 0
   Evicted Pages: 1758928 (6G)
         Elapsed: 0.036811 seconds
$ vmtouch -v file
file
[o                                                          o] 54378/1758928

           Files: 1
     Directories: 0
  Resident Pages: 54378/1758928  212M/6G  3.09%
         Elapsed: 0.069962 seconds

Note how the -e doesn't evict the ranges at the start and end.

I agree though, it would be much nicer if it was possible to do this in a single command!