Large images inconsistent display using direct mode

[SigmaRichards]

This is something I've been exploring the cause of for a bit. Using kitty in direct-mode when displaying images (e.g. over SSH) occasionally causes images to not be displayed. I find this occurs regularly when the cropped + resized images are over 30kb.

The easiest way to test is by using ssh localhost, however, it occurs outside of ssh if you directly set is_SSH in lua/image/backends/kitty/init.lua. This happens both in and out of TMUX.

During one of the failures, I noted that the characters received by the terminal (i.e. the output of kitty --dump-commands) included a 5 character addition and further in the sequence a 1 character deletion - causing everything within this section to become corrupted once decoded. However, the bytes as reported by lua before being sent appear to be correct.

When using direct-mode, the protocol sends the 4096 base64 characters (effectively sending 3072 bytes each time) so 30kb is roughly 10 write-commands. When the file is small enough for only 9 write commands to be sufficient, the success rate is 90%, 10 commands is about 50% success, and 11 is about 10%.

Potential fix

I'm still trying to explore exactly what causes this issue, however, below is something I found which improves the success rate significantly. Annoyingly, it's another case of I can't explain why it helps.

+++ b/lua/image/backends/kitty/helpers.lua
@@ -92,6 +92,7 @@ local write_graphics = function(config, data)
       else
         control_payload = "m=1"
       end
+      vim.loop.sleep(1)
     end
   else
     -- utils.debug("kitty control payload:", control_payload)

I haven't had the issue occur at all using ssh AND tmux with this change. However, it will occasionally still occur when using JUST ssh. I'm still exploring, but if/when I come across something more conclusive I'll post a PR.

[3rd]

Thank you so much for taking the time to investigate this! We can add the sleep there, we have a sleep(1) in move_cursor as well, maybe it would be ok to add it after each write? It should have no impact on the performance, I think.

[isvicy]

In my case, when using this plugin over ssh in kitty, some pics are rending as expected, some pics don't. When moving the cursor in Neovim, there could be base64 string floated like the screenshot below.

Does the issue I met related to this one? The size of the image referred in the file I opened is 500kb+.

[isvicy]

Btw, I've only encountered this problem when using ssh, it doesn't occur during local development.

[3rd]

Sounds related, but the image should be resized and hopefully much smaller, not sure where this breaks to cause this. Are you running it through tmux as well?

[isvicy]

Sounds related, but the image should be resized and hopefully much smaller, not sure where this breaks to cause this. Are you running it through tmux as well?

no, just ssh

[SigmaRichards]

Unfortunately I haven't had enough time to look into exactly why this is occurring, but even with the resizing, larger images can easily still go over the threshold for when this direct-mode "corruption" occurs (leading to the base64 text).

For me at least, with the additional sleep(1), I haven't run into any issues with this for some time, even though I've uncapped the max image size. I just checked and on the latest commit and it still seems to be a sufficient fix for me. My only guess why this occurs is that when writing the image data, it doesn't flush fast enough, which with enough packets (i.e. for large images) leads to this corruption becoming more likely.

@ak2kay try #158 and see if it fixes, or helps at all