feature request: option to preload file previews

[BPplays]

i would like an option to preload file previews around the cursor. maybe something like set previewpreload 5 in lfrc could preload 5 items above and 5 below the cursor and set previewpreload 0 would disable it.

here is an example of a kinda slowish preview (click to go to the video)

[joelim-work]

I don't think an option like previewpreload as you described makes sense to implement because:

1. lf doesn't really have this kind of mechanism for working with files above/below the cursor
2. The problem would still persist if you jumped to a file outside that range (e.g. page-down/bottom)

I suspect the bottleneck of displaying file previews comes from converting the image to text data, which is then written to stdout of the previewer script. Have you tried caching the data (i.e. if it exists then cat the cached contents, otherwise generate it) and see if it makes any difference?

If caching works then an idea for preloading the cache would be to configure an on-cd hook, which would go through all the images and generate previews for all of them.

[hakan-demirli]

If caching works then an idea for preloading the cache would be to configure an on-cd hook, which would go through all the images and generate previews for all of them.

Here is the implementation for kitty:

#!/usr/bin/env bash

file=$1
w=$2
h=$3
x=$4
y=$5
cache_width=860
cachedir="${XDG_CACHE_HOME:-$HOME/.cache}/lf_images"
mkdir -p "$cachedir"

file="$1"; shift
cachekey=$(echo -n "$file $(stat -c %Y "$file")" | md5sum | cut -d ' ' -f 1)
thumbnail="$cachedir/$cachekey.png"

preview() {
    kitty +kitten icat --silent --stdin no --transfer-mode memory --place "${w}x${h}@${x}x${y}" "$1" < /dev/null > /dev/tty
}

generate_thumbnails() {
    folder=$(dirname "$1")
    for img in "$folder"/*.{avi,mp4,mkv,opus,jpg,jpeg,png,bmp}; do
        img_cachekey=$(echo -n "$img $(stat -c %Y "$img")" | md5sum | cut -d ' ' -f 1)
        img_thumbnail="$cachedir/$img_cachekey.png"
        if [ ! -f "$img_thumbnail" ]; then
            # No thumbnails for ginormous images:
            # BUG: https://video.stackexchange.com/questions/28408/how-to-fix-ffmpeg-picture-size-32768x16384-is-invalid
            ffmpeg -y -i "$img" -vframes 1 -vf "scale=${cache_width}:-1" "$img_thumbnail" &
                # ffmpeg -y -i "$file" -vframes 1 -vf "scale=${w}0:-1" "$thumbnail"
        fi
    done
    wait
}

if [ ! -f "$thumbnail" ]; then
    case "$(basename "$file" | tr '[A-Z]' '[a-z]')" in
    *.tar*) tar tf "$file" ;;
    *.zip) unzip -l "$file" ;;
    *.rar) unrar l "$file" ;;
    *.7z) 7z l "$file" ;;
    *.avi|*.mp4|*.mkv|*.opus|*.jpg|*.jpeg|*.png|*.bmp)
        ffmpeg -y -i "$file" -vframes 1 -vf "scale=${cache_width}:-1" "$thumbnail"
        preview "$thumbnail"
        generate_thumbnails "$file"
        ;;
    *.pdf)
        gs -o "$thumbnail" -sDEVICE=pngalpha -dLastPage=1 "$file" >/dev/null
        preview "$thumbnail"
        ;;
    *.svg)
        convert "$file" -resize "${w}x${h}" "$thumbnail"
        ;;
    *.ttf|*.otf|*.woff)
        fontpreview -i "$file" -o "$thumbnail"
        ;;
    *) bat --terminal-width "${w}" -f "$file" --style=numbers ;;
    esac
else
    preview "$thumbnail"
fi
return 127 # nonzero retcode required for lf previews to reload

Here are the rest of the config files: https://github.com/hakan-demirli/dotfiles/tree/main/.config/lf All of it runs in parallel. So, if you enter a folder with 100 4K images your RAM usage will blow up to ~14GB.

If it is too much work or too slow, you can try yazi. It has bunch of performance improvements over lf. However, it lacks custom previewers/plugins.

[BPplays]

reply to @DusanLesan in #1774

This is duplicate of #1537. I think caching should not be a issue solved by lf. I think preview script should determine surrounding files and cache them, or at most lf could call cache script with the file names surrounding current file

a problem with the scripts caching it is that there is still a noticable flickering even if it's in the script cache. it just doesn't load fast enough even with a fast hash and something compiled like golang

[DusanLesan]

a problem with the scripts caching it is that there is still a noticable flickering even if it's in the script cache. it just doesn't load fast enough even with a fast hash and something compiled like golang

I am not sure I understand. Are you saying that flicker occurs even if the file is already cached, or that some current caching options(I am not aware of any) don't cache fast enough?

[BPplays]

@DusanLesan my preview script is setup to cache images and metadata that it needs so it doesn't have to get them every time, even though it gets stuff from cache quick it still flickers at least on windows terminal if i go over an item that is cached in my script but the flicker doesn't happen after going back over it after it's cached in lf. but lf clears it's cache after closing it, it's not stored on disk or preloaded

[DusanLesan]

Ok. I have not experienced any unexpected issues with caching. I expect to see a delay when loading images for the first time (more noticeable on big images), but cached images load almost instantly for me.

I might be insensitive to flickering since I think even loading of uncached images does not feel bad for average size images.

https://github.com/user-attachments/assets/3c3a9f0f-4d46-495c-8fd0-616eb2f9d790

[joelim-work]

@DusanLesan my preview script is setup to cache images and metadata that it needs so it doesn't have to get them every time, even though it gets stuff from cache quick it still flickers at least on windows terminal if i go over an item that is cached in my script but the flicker doesn't happen after going back over it after it's cached in lf. but lf clears it's cache after closing it, it's not stored on disk or preloaded

For text-based previews, including images that are converted to color escape sequences like chafa -f symbols, the output of the previewer script is indeed cached in lf memory. If you're experiencing flickering when viewing images for the first time then I guess it simply takes too long for the previewer script to run.

The thing about image previews is that there are many different factors involved, including the OS, the previewer script contents, the preview method used (text/kitty/sixel/ueberzug), etc., and therefore different users will experience varying levels of success.

[BPplays]

@joelim-work the total time it takes to display a cache image is only ≈5ms but i still notice flicker, probably because lf seems to clear the screen to print 'Loading...', maybe a fix to improve this would be to only clear the screen and display a loading indicator if the previewer is taking more then a set amount of time. 30-100 ms is probably a reasonable value. only after that amount of time lf will clear the screen and print 'Loading...' if the previewer finishes before that it just prints the new preview to screen over the old one

[joelim-work]

lf already does this though, it waits 100ms (grace period) before displaying the loading... message. I added it myself in #1154.

So what happens is:

1. A new file is selected (for the first time)
2. The timer is set to 100ms
3. The previewer script is then invoked in a background thread.
4. At this point the preview display is blank because the file hasn't loaded yet
5. When the timer expires, it will then display the loading... message
6. Then when the previewer script finishes, it will display the actual preview and the loading... message should disappear

You are right about the fact that lf clears the screen and then displays nothing when a new file is selected for the first time. Maybe you might be able to add some kind of preload command which can load the preview contents into memory beforehand. If the patch is small enough I think it can be accepted.

[og900aero]

@joelim-work the total time it takes to display a cache image is only ≈5ms but i still notice flicker, probably because lf seems to clear the screen to print 'Loading...', maybe a fix to improve this would be to only clear the screen and display a loading indicator if the previewer is taking more then a set amount of time. 30-100 ms is probably a reasonable value. only after that amount of time lf will clear the screen and print 'Loading...' if the previewer finishes before that it just prints the new preview to screen over the old one

The script provided for lf to view pictures is also slow for me. I use another preview script. It's very fast, it doesn't even show the loading label or something else. I use ueberzug. previewer script:

#!/bin/sh

draw() {
    if [ -n "$FIFO_UEBERZUG" ]; then
        printf '{"action": "add", "identifier": "preview", "x": %d, "y": %d, "width": %d, "height": %d, "scaler": "contain", "scaling_position_x": 0.5, "scaling_position_y": 0.5, "path": "%s"}\n' \
    "$4" "$5" "$2" "$3" "$1" >"$FIFO_UEBERZUG"
    fi
  exit 1
}

hash() {
  printf '%s/.cache/lf/%s' "$HOME" \
    "$(stat --printf '%n\0%i\0%F\0%s\0%W\0%Y' -- "$(readlink -f "$1")" | sha256sum | awk '{print $1}')"
}

cache() {
  if [ -f "$1" ]; then
    draw "$@"
  fi
}

batorcat() {
    if command -v batcat > /dev/null 2>&1; then
        batcat --color=always --style=snip --pager=never --decorations=always "$file"
    else
        cat "$file"
    fi
}

glowormdcat() {
    if command -v glow > /dev/null 2>&1; then
        glow "$file"
    else
        mdcat "$file"
    fi
}

file="$1"
shift

if [ -n "$FIFO_UEBERZUG" ]; then
  case "$(file -Lb --mime-type -- "$file")" in
    image/*)
        draw "$file" "$@"
      ;;
    video/*)
      cache="$(hash "$file").jpg"
      cache "$cache" "$@"
      ffmpegthumbnailer -i "$file" -o "$cache" -s 0
      draw "$cache" "$@"
      ;;
    audio/*) exiftool "$file" ;;
    message/rfc822) w3m -dump "$file" ;;
    */*torrent) exiftool "$file" ;;
    */pdf)
      pdftoppm -jpeg -f 1 -singlefile "$1" "$hash"
      draw "$file" "$@"
      ;;
    */rtf) catdoc "$file" ;;
    */msword) catdoc "$file" ;;
    */*wordprocessing*) docx2txt < "$file" | batcat --style=plain ;;
    */*spreadsheet*) xlsx2csv "$file" ;;
    text/*|*/json) batorcat "$file" ;;
    */zip) unzip -l "$file" ;;
    */gzip) 
      tar tzf "$file"
      gzip -l "$file"
      ;;
    */zstd) tar tf "$file" ;;
    */x-bzip2) tar tjf "$file" ;;
    */x-xz) tar tf "$file" ;;
    */x-tar) tar tf "$file" ;;
    */*rar) unrar l "$file" ;;
    *) file -ibL "$file" | grep -q text && batorcat "$file" || file -Lb "$file" ;;
  esac
fi

file -Lb -- "$1" | fold -s -w "$width"
exit 0

[BPplays]

@og900aero what "script provided for lf to view pictures" are you talking about i use a custom one made by me if you mean the other one in this issue i haven't tried that one. my custom previewer is quite fast for cached images ≈5ms have you timed yours? i think the problem is caused by clearing the screen before printing the preview and i don't think Überzug would suffer from that issue

[og900aero]

@og900aero what "script provided for lf to view pictures" are you talking about i use a custom one made by me if you mean the other one in this issue i haven't tried that one. my custom previewer is quite fast for cached images ≈5ms have you timed yours? i think the problem is caused by clearing the screen before printing the preview and i don't think Überzug would suffer from that issue

I can't time it, unfortunately. I think it could only speed up significantly if this function were included in lf (see yazi). And then there could also be an additional function to scroll the preview window. Of course, the precache function could also be implemented this way. But I don't see any chance of that getting into lf. I can partly understand it, because it was designed as a minimalist program, but at the same time, I think it is such a basic function that it would be worth integrating it into it.

[joelim-work]

So I managed to write a preload command which calls the previewer script for any files passed to it, and this can be triggered when entering a directory, something like this:

Click to expand diff

```diff diff --git a/eval.go b/eval.go index 23a6be5..be97d2d 100644 --- a/eval.go +++ b/eval.go @@ -1761,6 +1761,21 @@ func (e *callExpr) eval(app *app, args []string) { } app.readFile(replaceTilde(e.args[0])) app.ui.loadFileInfo(app.nav) + case "preload": + wd, err := os.Getwd() + if err != nil { + log.Printf("getting current directory: %s", err) + } + + for _, arg := range e.args { + path := replaceTilde(arg) + if !filepath.IsAbs(path) { + path = filepath.Join(wd, path) + } else { + path = filepath.Clean(path) + } + app.nav.preload(path, app.ui) + } case "push": if len(e.args) != 1 { app.ui.echoerr("push: requires an argument") diff --git a/nav.go b/nav.go index 592a898..f291664 100644 --- a/nav.go +++ b/nav.go @@ -745,7 +745,7 @@ func (nav *nav) previewLoop(ui *ui) { nav.volatilePreview = false } if len(path) != 0 { - nav.preview(path, win) + nav.preview(path, win, false) prev = path } } @@ -824,7 +824,7 @@ func (nav *nav) previewDir(dir *dir, win *win) { } } -func (nav *nav) preview(path string, win *win) { +func (nav *nav) preview(path string, win *win, preload bool) { reg := ®{loadTime: time.Now(), path: path} defer func() { nav.regChan <- reg }() @@ -835,7 +835,8 @@ func (nav *nav) preview(path string, win *win) { strconv.Itoa(win.w), strconv.Itoa(win.h), strconv.Itoa(win.x), - strconv.Itoa(win.y)) + strconv.Itoa(win.y), + fmt.Sprint(preload)) out, err := cmd.StdoutPipe() if err != nil { @@ -854,7 +855,9 @@ func (nav *nav) preview(path string, win *win) { if e, ok := err.(*exec.ExitError); ok { if e.ExitCode() != 0 { reg.volatile = true - nav.volatilePreview = true + if !preload { + nav.volatilePreview = true + } } } else { log.Printf("loading file: %s", err) @@ -915,6 +918,13 @@ func (nav *nav) preview(path string, win *win) { } } +func (nav *nav) preload(path string, ui *ui) { + win := ui.wins[len(ui.wins)-1] + go func() { + nav.preview(path, win, true) + }() +} + func (nav *nav) loadReg(path string, volatile bool) *reg { r, ok := nav.regCache[path] if !ok || (volatile && r.volatile) { ```

cmd on-cd &{{
    for file in *; do
        # can also pass in multiple files as a single string
        lf -remote "send $id preload '$file'"
    done
}}

However there's a caveat - when invoked like this the previewer script shouldn't have any side effects, including displaying images via ueberzug. So another parameter has to be passed to the previewer script to indicate whether it is from a preload or not.

In any case, this idea is somewhat hacky and I don't have much further interest in it. But I will leave it here in case someone else can make use of it.