Add a WaitCursor

[Heddesdorfer]

What happened: I opened an other directory, seven bee-icons(?) were displayed and vkdt did not react anymore. After a while when I clicked on the close "X", a message was displayed "vktd reagiert nicht." with buttons "warten" and "beenden". i killed vkdt, started it again, opened the same directory: same procedure. To get a new logfile, I started again and now seven images were displayed.

In this directory are 7 mp4 files, which I think caused the problem. vkdt was working (what ever) in the background on these mp4 (to create these images?) and could not react to any input from the user. It would be nice, if there was something like a "WaitCursor" or another info, so that the user is informed, that vkdt is still alive.

PS: crtl-q doesn't work (minor issue)

OS Manjaro 23.1 Cinnamon Kernel 6.7.0-0 CPU AMD Ryzen 7 5800H Gra Geforce RTX 3050

[hanatos]

re: bee: you apparently never used an atari computer.

re: ctrl-q: it's ctrl-x on a qwerty keyboard

waiticon: whaaaat? certainly not. if stuff is so slow that it would keep me waiting the code needs to be fixed. i'd be interested to find out what it was doing on the mp4.

[Heddesdorfer]

a: no, never. I used a PDP-11/73 with RSX11M+ b: i have a qwertz keyboard. but ctrl-x doesn't work either c: i didn't mean to imply that your code is so slow that it needs to be fixed.

[hanatos]

i may have a problem with an infinite loop in some cases.. investigating..

[hanatos]

PDP-11/73: brilliant :D

no idea about ctrl-x.. works here (tm)

can you try again with the deadlock? i think i might have fixed an infinite loop. if this persists, let me know.. somehow we'll think of a way for me to reproduce (or maybe you can run gdb and kill it to find out what's it doing)

On Tue, Jan 30, 2024 at 11:28 AM Heddesdorfer @.***> wrote:

a: no, never. I used a PDP-11/73 with RSX11M+ b: i have a qwertz keyboard. but ctrl-x doesn't work either c: i didn't mean to imply that your code is so slow that it needs to be fixed.

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1916537522, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMAKKNENT6GSR7GFTDSOPDYRDDNFAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJWGUZTONJSGI . You are receiving this because you commented.Message ID: @.***>

[Heddesdorfer]

Öha! To reproduce the problem I deleted ~/cache and the vkdt.db files. The problem with the mp4 seems to be solved, the referring images were displayed in a few seconds. But for the other folder, containing about 260 images (cr3), it needed about 98 minutes.

Attached the log vkdtlog.txt

and the log from compiling, because there were warnings debug.txt I know, a warning is not an error, but better more than less information.

Addition: I copied a file ()cr3, 50MB) to "download" (where I had seven mp4) and tried to open it with vkdt. Error message: out of memory

[hanatos]

oh wow. i mean the thumbnail processing is on the slow side but the raw loading takes 2 seconds per image which is really remarkable. is this a slow hard drive and super highres images? there is also at least one terrible outlier at like 30sec load time (the first.. harddrive spinning up?)

i'll try to reproduce.. maybe the thumbnail threads aren't waking up fast enough in the background there.

re: out of memory: and it works fine without the CR3? i'd suspect the video code path to be less stable..

[Heddesdorfer]

Hard drive is a SSD. No other disk connected Images: from R3mk2, 24MP sensor, about 34MB each out of memory: I tried again and it worked. yesterday I opened that file right after the thumbnail thing - perhaps the memory wasn't cleared then (I didn't have the nerves for more tests yesterday and wanted ad least to pin the error.

I first read the download directory (with the 7 mp4: movies, 17GB at all) and then my normal working directory for images. I could try the other way to see if it makes any difference.

[Heddesdorfer]

I used the working directory first, the download second: it took about 45min.

Some observations:

• vkdt seems to work on groups of two images
• works on two up to 5 groups, then pause between 30sec and several minutes
• thumbnails of movies much faster than thumbnails of images
• cpu .25%, only on 4-5% when thumbnails are displayed (I think we expected that)
• only when thumbnails are displayed, the amount of disk reads increases (because of the size it must have been the images)

[hanatos]

..does it go any faster if you move the mouse to highlight images in lighttable mode?

On Wed, Jan 31, 2024 at 10:13 AM Heddesdorfer @.***> wrote:

I used the working directory first, the download second: it took about 45min.

Some observations:

• vkdt seems to work on groups of two images
• works on two up to 5 groups, then pause between 30sec and several minutes
• thumbnails of movies much faster than thumbnails of images
• cpu .25%, only on 4-5% when thumbnails are displayed (I think we expected that)
• only when thumbnails are displayed, the amount of disk reads increases (because of the size it must have been the images)

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1918689259, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMAKKJESKGKALARKQSBMQLYRIDMTAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJYGY4DSMRVHE . You are receiving this because you commented.Message ID: @.***>

[Heddesdorfer]

ehm...yes...now it took 5min

I'd really like to know the reason, Never ever heard of that!

[hanatos]

my hypothesis is.. that it's a bs thing with how threading works in vulkan. since you need to synchronise your access to a VkQueue (submit workloads to the gpu), i used to put a mutex around it. some more recent nvidia drivers didn't like this pattern, maybe probably they do something thread local? in any case i decided to do whatever everybody else seems to be doing in their game engines, have one single thread dedicated to calling all vkQueueSubmit. that means the worker threads (thumbnail rendering) need to submit their work not to the gpu, but to this thread, and make sure it wakes up to process their request. now this waking up is a bit tricky because it's different between cli and gui operation.. so it apparently just doesn't work. waking up the gui thread at least is easy (move mouse).. (but should be straight forward to fix this)

On Wed, Jan 31, 2024 at 10:35 AM Heddesdorfer @.***> wrote:

ehm...yes...now it took 5min

I'd really like to the reason, Never ever heard of that!

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1918727001, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMAKKIMP56E4MAIG4WW6Y3YRIF6NAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJYG4ZDOMBQGE . You are receiving this because you commented.Message ID: @.***>

[Heddesdorfer]

I just compiled the daily: I needed 5 min again (about 1 image / sec), but without mouse moves!

[hanatos]

i suppose that's good news. really sad that your raws take so long to decode. my fuji raws are decoded by rawspeed in 20ms, quite a difference.

On Thu, Feb 1, 2024 at 8:22 PM Heddesdorfer @.***> wrote:

I just compiled the daily: I needed 5 min again (about 1 image / sec), but without mouse moves!

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1922062574, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMAKKJA6NXRDKLBMUKYX6LYRPTPNAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRSGA3DENJXGQ . You are receiving this because you commented.Message ID: @.***>

[Heddesdorfer]

Is this the difference between rawspeed and rawler? I could not compile rawspeed on my machine...

johannes hanika @.***> schrieb am Fr., 2. Feb. 2024, 08:29:

i suppose that's good news. really sad that your raws take so long to decode. my fuji raws are decoded by rawspeed in 20ms, quite a difference.

On Thu, Feb 1, 2024 at 8:22 PM Heddesdorfer @.***> wrote:

I just compiled the daily: I needed 5 min again (about 1 image / sec), but without mouse moves!

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1922062574, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAMAKKJA6NXRDKLBMUKYX6LYRPTPNAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRSGA3DENJXGQ>

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1923200690, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVZP4GHAX6SSTYUJRICQWBTYRSIV7AVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRTGIYDANRZGA . You are receiving this because you authored the thread.Message ID: @.***>

[hanatos]

i think is a difference between how well some raw formats are suitable for parallel decoding.

On Fri, Feb 2, 2024 at 11:08 AM Heddesdorfer @.***> wrote:

Is this the difference between rawspeed and rawler? I could not compile rawspeed on my machine...

johannes hanika @.***> schrieb am Fr., 2. Feb. 2024, 08:29:

i suppose that's good news. really sad that your raws take so long to decode. my fuji raws are decoded by rawspeed in 20ms, quite a difference.

On Thu, Feb 1, 2024 at 8:22 PM Heddesdorfer @.***> wrote:

I just compiled the daily: I needed 5 min again (about 1 image / sec), but without mouse moves!

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1922062574, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AAMAKKJA6NXRDKLBMUKYX6LYRPTPNAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRSGA3DENJXGQ>

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1923200690, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AVZP4GHAX6SSTYUJRICQWBTYRSIV7AVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRTGIYDANRZGA>

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1923481760, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMAKKNFYMEPYJ5XXCDJMGDYRS3J7AVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRTGQ4DCNZWGA . You are receiving this because you commented.Message ID: @.***>

[Heddesdorfer]

OK, before bying the next camera I will look for tests of parallel decoding of raw formats :) everything else are minor issues.

johannes hanika @.***> schrieb am Fr., 2. Feb. 2024, 11:33:

i think is a difference between how well some raw formats are suitable for parallel decoding.

On Fri, Feb 2, 2024 at 11:08 AM Heddesdorfer @.***> wrote:

Is this the difference between rawspeed and rawler? I could not compile rawspeed on my machine...

johannes hanika @.***> schrieb am Fr., 2. Feb. 2024, 08:29:

i suppose that's good news. really sad that your raws take so long to decode. my fuji raws are decoded by rawspeed in 20ms, quite a difference.

On Thu, Feb 1, 2024 at 8:22 PM Heddesdorfer @.***> wrote:

I just compiled the daily: I needed 5 min again (about 1 image / sec), but without mouse moves!

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1922062574,

or

unsubscribe <

https://github.com/notifications/unsubscribe-auth/AAMAKKJA6NXRDKLBMUKYX6LYRPTPNAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRSGA3DENJXGQ>

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1923200690, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AVZP4GHAX6SSTYUJRICQWBTYRSIV7AVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRTGIYDANRZGA>

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1923481760, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAMAKKNFYMEPYJ5XXCDJMGDYRS3J7AVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRTGQ4DCNZWGA>

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1923525955, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVZP4GHSTFYGWMUW6EOUSG3YRS6GHAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRTGUZDKOJVGU . You are receiving this because you authored the thread.Message ID: @.***>

[Heddesdorfer]

I couldn't resist: I compiled vkdt with rawspeed (3 warnings, but working). this time vkdt has been very fast...in displaying the bomb for each image. From the log: ...J12A0440.CR3) virtual void rawspeed::Cr3Decoder::checkSupportInternal(const CameraMetaData *), line 603: CR3 compressor version (CNCV: CanonCR3_001/00.11.00/00.00.00) is not supported

[hanatos]

sigh thanks for trying. i guess you're saying it's time to abandon my cr3 rawspeed branch because it's incomplete anyways and go back to vanilla rawspeed completely without cr3 support.

On Fri, Feb 2, 2024 at 5:13 PM Heddesdorfer @.***> wrote:

I couldn't resist: I compiled vkdt with rawspeed (3 warnings, but working). this time vkdt has been very fast...in displaying the bomb for each image. From the log: ...J12A0440.CR3) virtual void rawspeed::Cr3Decoder::checkSupportInternal(const CameraMetaData *), line 603: CR3 compressor version (CNCV: CanonCR3_001/00.11.00/00.00.00) is not supported

— Reply to this email directly, view it on GitHub https://github.com/hanatos/vkdt/issues/108#issuecomment-1924187744, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMAKKPC3QONJXRG4XOW5DTYRUGCPAVCNFSM6AAAAABCQ5UBJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRUGE4DONZUGQ . You are receiving this because you commented.Message ID: @.***>

[Heddesdorfer]

sob no rawspeed for cr3 users...

[hanatos]

right. fwiw i don't think rawler is much slower, if at all. some raw formats it supports better from what i can tell. the difference between CR2 and (uncompressed) RAF is really the file format itself.

thanks for all the observations here. i think i'll close this as resolved now, please feel free to reopen/open another with any more issues you might encounter.

[Heddesdorfer]

ok. imho i think you could communicate, that cr3 support for "new" cameras like r6m2, r8 or r3 doesn't work with rawspeed, whereas rawler does. otherwise it could prevent someone from using vkdt. one has just to choose the right library.