darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.83k stars 1.14k forks source link

Windows fragility #4400

Closed GrahamByrnes closed 4 years ago

GrahamByrnes commented 4 years ago

version 3.1.0_777~gbd7c89a0b-dirty, Windows 10, seems very fragile. Crashes happen in very unpredictable ways... sometimes going to the darkroom, sometimes to the lighttable, sometimes on changing images, very often (but not always) on attempting to apply a style.

Sometimes the lock files are left active, sometimes not. Problems seem equally likely for any image, original or duplicate.

If I recover my DB from back-up and return to v708, all is well again.

So what should I look for in the -d log? It seems just to record ordinary daily life, then stops as though I'd closed the program.

johnny-bit commented 4 years ago

Since you're rebuilding yourself, the best bet would be to use git bisect to find source of instability (what introduced it) :)

GrahamByrnes commented 4 years ago

Here's something... this time it left a loop...

msys_perf_log.txt

[dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 [dt_pthread_create] info: bumping pthread's stacksize from 0 to 2097152 image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] Magick: Unrecognized image format () [No such file or directory]. [dt_imageio_large_thumbnail GM] thumbnail not found? image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [basecurve ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [colorout ] Magick: Unrecognized image format () [No such file or directory]. [dt_imageio_large_thumbnail GM] thumbnail not found? image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [basecurve ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [colorout ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,016 secs (0,109 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [colorcorrection ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [colorcorrection ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ] image colorspace transform Lab-->RGB took 0,000 secs (0,000 CPU) [channelmixer ] image colorspace transform RGB-->Lab took 0,000 secs (0,000 CPU) [tonecurve ]

TurboGit commented 4 years ago

What is the sha1 for v708 ?

TurboGit commented 4 years ago

Anyway you seem to have a broken tree, my last version is release-3.1.0-768-g13c4231f8. So what is your version 777 ????? Looks like some changes on your side, nothing we can do.

GrahamByrnes commented 4 years ago

Yes, there are the preview_downsampling changes... I don't think they are the issue, but accept that it's my responsibility.

Is there a way of getting at the sha1 without opening the program? I would need to reload the old db to enable it to open, because of the db updates of a few days back.

Le sam. 29 févr. 2020 à 14:36, Pascal Obry notifications@github.com a écrit :

Anyway you seem to have a broken tree, my last version is release-3.1.0-768-g13c4231f8. So what is your version 777 ????? Looks like some changes on your side, nothing we can do.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=AN2X2RUIAS4URX4F3ZNXSC3RFEHO3A5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENL2J5Q#issuecomment-592946422, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2X2RQJUNOVTO6J5JDNPWTRFEHO3ANCNFSM4K6V5SVQ .

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

TurboGit commented 4 years ago

@GrahamByrnes : We cannot accept report with local changes, there is already far too more work on dt. You should always check that the issue reproduce with current master.

For the other question, no... I don't know a way to get the sha1 for the given number of commit. But when you are on this commit (this is the case as you've tested it and said there is no issue there) a git describe will give all information we need.

TurboGit commented 4 years ago

If you can reproduce on master, I'll reopen.

johnny-bit commented 4 years ago

is there a way of getting at the sha1 without opening the program?

simply run darktable --version

it'll output in 1st line something like: this is darktable 3.1.0+755~g0a3bcbf08-dirty

0a3bcbf08 is the commit hash.

GrahamByrnes commented 4 years ago

Thanks, useful info :-). Currently building a brand-new nickel chrome version cloned from dt master to answer (more or less) where the problem is. My old working version had the same contaminants as the recent not-so-often working one, so that complicates the story.

Le sam. 29 févr. 2020 à 16:38, Hubert Kowalski notifications@github.com a écrit :

is there a way of getting at the sha1 without opening the program?

simply run darktable --version

it'll output in 1st line something like: this is darktable 3.1.0+755~g0a3bcbf08-dirty

0a3bcbf08 is the commit hash.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=AN2X2RT4H22KZJ773N2YE53RFEVX7A5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENL5CIQ#issuecomment-592957730, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2X2RSQ4FCDFDGYNAA7ZWTRFEVX7ANCNFSM4K6V5SVQ .

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

GrahamByrnes commented 4 years ago

Ok, just started up a freshly built clone of the darktable-org/darktable master, 3.1.0+769~g1ef4476cf

Crashed after a minute or so while trying to apply a style.

here's the last bit of the -d perf trace, in case it helps (looks normal to me):

20,022621 [dev_process_thumbnail] pixel pipeline processing took 0,641 secs (5,094 CPU) 24,661459 [dev] took 0,000 secs (0,000 CPU) to load the image. 25,685327 [dev] took 0,000 secs (0,000 CPU) to load the image. 25,736993 [dev_pixelpipe] took 0,031 secs (0,203 CPU) initing base buffer [full] 25,736993 [dev_pixelpipe] took 0,000 secs (0,000 CPU) processed raw black/white point' on CPU, blended on CPU [full] 25,752597 [dev_pixelpipe] took 0,016 secs (0,031 CPU) processedwhite balance' on CPU, blended on CPU [full] 25,768222 [dev_pixelpipe] took 0,016 secs (0,000 CPU) processed highlight reconstruction' on CPU, blended on CPU [full] 26,459426 [dev_pixelpipe] took 0,691 secs (5,703 CPU) processeddemosaic' on CPU, blended on CPU [full] 26,473422 [dev_pixelpipe] took 0,014 secs (0,016 CPU) processed input color profile' on CPU, blended on CPU [full] 26,561411 [dev_pixelpipe] took 0,088 secs (0,719 CPU) processedoutput color profile' on CPU, blended on CPU [full] 26,569412 [dev_pixelpipe] took 0,008 secs (0,172 CPU) processed gamma' on CPU, blended on CPU [full] 26,570411 [dev_process_image] pixel pipeline processing took 0,865 secs (6,844 CPU) 26,661420 [dev_pixelpipe] took 0,000 secs (0,000 CPU) initing base buffer [preview] 26,662415 [dev_pixelpipe] took 0,001 secs (0,000 CPU) processedraw black/white point' on CPU, blended on CPU [preview] 26,664416 [dev_pixelpipe] took 0,001 secs (0,000 CPU) processed white balance' on CPU, blended on CPU [preview] 26,666414 [dev_pixelpipe] took 0,002 secs (0,000 CPU) processedhighlight reconstruction' on CPU, blended on CPU [preview] 26,676417 [dev_pixelpipe] took 0,010 secs (0,188 CPU) processed demosaic' on CPU, blended on CPU [preview] 26,690414 [dev_pixelpipe] took 0,014 secs (0,016 CPU) processedinput color profile' on CPU, blended on CPU [preview] 26,829419 [dev_pixelpipe] took 0,138 secs (1,125 CPU) processed output color profile' on CPU, blended on CPU [preview] 26,840415 [dev_pixelpipe] took 0,010 secs (0,141 CPU) processedgamma' on CPU, blended on CPU [preview] 26,984421 [dev_process_preview] pixel pipeline processing took 0,328 secs (1,797 CPU)

Le sam. 29 févr. 2020 à 16:48, Graham Byrnes grahamb29@gmail.com a écrit :

Thanks, useful info :-). Currently building a brand-new nickel chrome version cloned from dt master to answer (more or less) where the problem is. My old working version had the same contaminants as the recent not-so-often working one, so that complicates the story.

Le sam. 29 févr. 2020 à 16:38, Hubert Kowalski notifications@github.com a écrit :

is there a way of getting at the sha1 without opening the program?

simply run darktable --version

it'll output in 1st line something like: this is darktable 3.1.0+755~g0a3bcbf08-dirty

0a3bcbf08 is the commit hash.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=AN2X2RT4H22KZJ773N2YE53RFEVX7A5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENL5CIQ#issuecomment-592957730, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2X2RSQ4FCDFDGYNAA7ZWTRFEVX7ANCNFSM4K6V5SVQ .

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

GrahamByrnes commented 4 years ago

It did apply the style, btw... then it crashed again when I tried to remove it from the history stack.

Le sam. 29 févr. 2020 à 17:11, Graham Byrnes grahamb29@gmail.com a écrit :

Ok, just started up a freshly built clone of the darktable-org/darktable master, 3.1.0+769~g1ef4476cf

Crashed after a minute or so while trying to apply a style.

here's the last bit of the -d perf trace, in case it helps (looks normal to me):

20,022621 [dev_process_thumbnail] pixel pipeline processing took 0,641 secs (5,094 CPU) 24,661459 [dev] took 0,000 secs (0,000 CPU) to load the image. 25,685327 [dev] took 0,000 secs (0,000 CPU) to load the image. 25,736993 [dev_pixelpipe] took 0,031 secs (0,203 CPU) initing base buffer [full] 25,736993 [dev_pixelpipe] took 0,000 secs (0,000 CPU) processed raw black/white point' on CPU, blended on CPU [full] 25,752597 [dev_pixelpipe] took 0,016 secs (0,031 CPU) processedwhite balance' on CPU, blended on CPU [full] 25,768222 [dev_pixelpipe] took 0,016 secs (0,000 CPU) processed highlight reconstruction' on CPU, blended on CPU [full] 26,459426 [dev_pixelpipe] took 0,691 secs (5,703 CPU) processeddemosaic' on CPU, blended on CPU [full] 26,473422 [dev_pixelpipe] took 0,014 secs (0,016 CPU) processed input color profile' on CPU, blended on CPU [full] 26,561411 [dev_pixelpipe] took 0,088 secs (0,719 CPU) processedoutput color profile' on CPU, blended on CPU [full] 26,569412 [dev_pixelpipe] took 0,008 secs (0,172 CPU) processed gamma' on CPU, blended on CPU [full] 26,570411 [dev_process_image] pixel pipeline processing took 0,865 secs (6,844 CPU) 26,661420 [dev_pixelpipe] took 0,000 secs (0,000 CPU) initing base buffer [preview] 26,662415 [dev_pixelpipe] took 0,001 secs (0,000 CPU) processedraw black/white point' on CPU, blended on CPU [preview] 26,664416 [dev_pixelpipe] took 0,001 secs (0,000 CPU) processed white balance' on CPU, blended on CPU [preview] 26,666414 [dev_pixelpipe] took 0,002 secs (0,000 CPU) processedhighlight reconstruction' on CPU, blended on CPU [preview] 26,676417 [dev_pixelpipe] took 0,010 secs (0,188 CPU) processed demosaic' on CPU, blended on CPU [preview] 26,690414 [dev_pixelpipe] took 0,014 secs (0,016 CPU) processedinput color profile' on CPU, blended on CPU [preview] 26,829419 [dev_pixelpipe] took 0,138 secs (1,125 CPU) processed output color profile' on CPU, blended on CPU [preview] 26,840415 [dev_pixelpipe] took 0,010 secs (0,141 CPU) processedgamma' on CPU, blended on CPU [preview] 26,984421 [dev_process_preview] pixel pipeline processing took 0,328 secs (1,797 CPU)

Le sam. 29 févr. 2020 à 16:48, Graham Byrnes grahamb29@gmail.com a écrit :

Thanks, useful info :-). Currently building a brand-new nickel chrome version cloned from dt master to answer (more or less) where the problem is. My old working version had the same contaminants as the recent not-so-often working one, so that complicates the story.

Le sam. 29 févr. 2020 à 16:38, Hubert Kowalski notifications@github.com a écrit :

is there a way of getting at the sha1 without opening the program?

simply run darktable --version

it'll output in 1st line something like: this is darktable 3.1.0+755~g0a3bcbf08-dirty

0a3bcbf08 is the commit hash.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=AN2X2RT4H22KZJ773N2YE53RFEVX7A5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENL5CIQ#issuecomment-592957730, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2X2RSQ4FCDFDGYNAA7ZWTRFEVX7ANCNFSM4K6V5SVQ .

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

johnny-bit commented 4 years ago

Get -d all trace since perf ain't telllin jack.

@TurboGit - since @GrahamByrnes can reproduce on clear master can you reopen?

oh and @GrahamByrnes - can you try with completelly clean darktable configs (everything default, no images, no styles), then import the style you want to apply and then try to reproduce?

GrahamByrnes commented 4 years ago

Right, moved all my images somewhere safe. Opened dt and it seems quite stable with the 3 images I imported. First problem is that from the style-> import menu I don't have access to the AppData folder where dt has installed the usual config files, so I can't import the style files from their usual location. I could access "Application Data", but nothing gets stored there...

However, moving the style folder out onto c:/, I could read them and apply them with no drama. Still not possible to delete a style frpm the disk (it disappears from the menu... I'm not sure if this is expected behaviour or not), nor to delete the version-0 xmp if I delete the image file.

I could read a filename_01.dng.xmp copied in from the storage, and the edits were correctly preserved. Oh, but this is cute: after copying in the extra xmp, I see the two versions of the same image in darkroom, but in lighttable I see two different images grouped together... the second one comes from... ?

(I guess I must have copied a non-matching xmp for the same filename in a different folder a couple of years older... so I was seeing the jpg preview from that in lt, but not in dr, despite swapping back & forth. Edit bis: once I had selected the image in DR, the correct preview was seen in LT, do not a problem).

So it seems that some of the problems may be related to Windows file permissions, but some remain simply bizarre. No crashes though now.

GrahamByrnes commented 4 years ago

For those who can bear to look, here's the story on users/application data vs users/appdata

"What I have discovered now is that although the product states that it stores its profiles in the LOCAL subdirectory of "Application Data", "Application Data" is a symbolic link pointing to AppData. Trying to open "Application Data" fails with the access denied message."

It's kind of cool, you can create a folder in there via the styles->export option, save your style, but if you try to read from it, both the file and the containing folder are invisible. They are still there, because you can't re-create the same folder name. Eventually you find the folder you exported to under users/appdata/roaming,... but the path to that folder is barred by dt, so you can't re-read.

You can delete the folder via the windows explorer, however.

GrahamByrnes commented 4 years ago

Back to business: as the next step, I moved a few hundred photos (dng & dng.xmp) into their regular spot, and asked dt to import them.

All went well for a minute or two, then crash.

Re-opened, was asked if I wanted to keep or over-write the xmp from one of the images already in my 3 test images. Clicked "keep"... crash.

Below is the last few images worth of the -d all trace from the first crash. Below, I'll put the the end of the second crash trace.

58,909827 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1341, function dt_history_hash_write_from_history(): prepare "INSERT INTO main.history_hash (imgid, current_hash) VALUES (?1, ?2) ON CONFLICT (imgid) DO UPDATE SET current_hash=?2" 58,909827 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:208, function dt_tag_new(): prepare "SELECT id FROM data.tags WHERE name = ?1" 58,909827 [lighttable] image expose took 0,0000 sec 58,909827 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:633, function dt_set_darktable_tags(): prepare "SELECT COUNT(*) FROM memory.darktable_tags" 58,909827 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:902, function dt_tag_get_tags(): prepare "SELECT DISTINCT T.id FROM main.tagged_images AS I JOIN data.tags T on T.id = I.tagid WHERE I.imgid = 101 AND T.id NOT IN memory.darktable_tags" 58,925453 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 12" 58,925453 [lighttable] image expose took 0,0156 sec 58,925453 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 13" 58,925453 [lighttable] image expose took 0,0000 sec 58,925453 [lighttable] expose took 0,0000 sec 58,956703 [run_job-] 05 292391277,484457 58,956703 load image 100 mip 0 | queue: 1 | priority: 458,956703 58,956703 [run_job+] 05 292391277,484457 58,956703 load image 96 mip 0 | queue: 1 | priority: 458,956703 58,956703 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:220, function dt_image_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id and i.id = ?1" 58,956703 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:248, function _image_local_copy_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id AND i.id = ?1" 58,956703 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 96" 58,956703 [sql] C:/msys64/home/Graham/darktable/src/libs/tools/filmstrip.c:682, function _lib_filmstrip_draw_callback(): prepare "SELECT imgid FROM memory.collected_images ORDER BY rowid LIMIT ?1, ?2" 58,956703 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 5" 58,956703 [lighttable] image expose took 0,0000 sec 58,972327 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:220, function dt_image_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id and i.id = ?1" 58,972327 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:248, function _image_local_copy_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id AND i.id = ?1" 58,972327 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 6" 58,972327 [lighttable] image expose took 0,0156 sec 58,972327 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 7" 58,972327 [lighttable] image expose took 0,0000 sec 58,987950 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 8" 58,987950 [lighttable] image expose took 0,0156 sec 58,987950 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 9" 58,987950 [lighttable] image expose took 0,0000 sec 58,987950 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 10" 58,987950 [lighttable] image expose took 0,0000 sec 58,987950 [dev_pixelpipe] took 0,000 secs (0,000 CPU) processed demosaic' on CPU, blended on CPU [thumbnail] 58,987950 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 11" 58,987950 [lighttable] image expose took 0,0000 sec 58,987950 [dev_pixelpipe] took 0,000 secs (0,000 CPU) processedexposure' on CPU, blended on CPU [thumbnail]



Second crash partial trace:

4,712078 [lighttable] image expose took 0,0000 sec 4,743323 [sql] C:/msys64/home/Graham/darktable/src/libs/tools/filmstrip.c:682, function _lib_filmstrip_draw_callback(): prepare "SELECT imgid FROM memory.collected_images ORDER BY rowid LIMIT ?1, ?2" 4,743323 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 1" 4,743323 [lighttable] image expose took 0,0000 sec 4,743323 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 4" 4,743323 [lighttable] image expose took 0,0000 sec 4,743323 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1434, function dt_history_hash_get_status(): prepare "SELECT CASE WHEN basic_hash == current_hash THEN 1 WHEN auto_hash == current_hash THEN 2 WHEN (basic_hash IS NULL OR current_hash != basic_hash) AND (auto_hash IS NULL OR current_hash != auto_hash) THEN 4 ELSE 1 END AS status FROM main.history_hash WHERE imgid = 2" 4,743323 [lighttable] image expose took 0,0000 sec 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/image_cache.c:39, function dt_image_cache_allocate(): prepare "SELECT id, group_id, film_id, width, height, filename, maker, model, lens, exposure, aperture, iso, focal_length, datetime_taken, flags, crop, orientation, focus_distance, raw_parameters, longitude, latitude, altitude, color_matrix, colorspace, version, raw_black, raw_maximum, aspect_ratio FROM main.images WHERE id = ?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:39, function dt_history_snapshot_undo_create(): prepare "SELECT history_end FROM main.images WHERE id=?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:50, function dt_history_snapshot_undo_create(): prepare "SELECT MAX(id) FROM memory.undo_history WHERE imgid=?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:62, function dt_history_snapshot_undo_create(): prepare "INSERT INTO memory.undo_history SELECT ?1, imgid, num, module, operation, op_params, enabled, blendop_params, blendop_version, multi_priority, multi_name FROM main.history WHERE imgid=?2" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:75, function dt_history_snapshot_undo_create(): prepare "INSERT INTO memory.undo_masks_history SELECT ?1, imgid, num, formid, form, name, version, points, points_count, source FROM main.masks_history WHERE imgid=?2" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/metadata.c:241, function dt_metadata_get_list_id(): prepare "SELECT key, value FROM main.meta_data WHERE id=?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/libs/metadata.c:142, function update(): prepare "SELECT COUNT() FROM main.selected_images" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/libs/metadata.c:145, function update(): prepare "SELECT key, value, COUNT(id) AS ct FROM main.meta_data WHERE id IN (SELECT imgid FROM main.selected_images) GROUP BY key, value ORDER BY value" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/libs/metadata_view.c:232, function _metadata_view_update_values(): prepare "SELECT imgid FROM main.selected_images LIMIT 1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:2703, function dt_exif_xmp_read(): prepare "DELETE FROM main.masks_history WHERE imgid = ?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:2759, function dt_exif_xmp_read(): prepare "DELETE FROM main.history WHERE imgid = ?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:2771, function dt_exif_xmp_read(): prepare "INSERT INTO main.history (imgid, num, module, operation, op_params, enabled, blendop_params, blendop_version, multi_priority, multi_name) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:2925, function dt_exif_xmp_read(): prepare "UPDATE main.images SET history_end = ?1 WHERE id = ?2" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/iop_order.c:516, function dt_ioppr_write_iop_order(): prepare "INSERT OR REPLACE INTO main.module_order VALUES (?1, 0, NULL)" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/iop_order.c:539, function dt_ioppr_write_iop_order(): prepare "UPDATE main.module_order SET version = ?2, iop_list = NULL WHERE imgid = ?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1360, function dt_history_hash_write(): prepare "INSERT INTO main.history_hash (imgid, basic_hash, auto_hash, current_hash) VALUES (?1, ?2, ?3, ?4) ON CONFLICT (imgid) DO UPDATE SET basic_hash = ?2, auto_hash = ?3, current_hash = ?4" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:39, function dt_history_snapshot_undo_create(): prepare "SELECT history_end FROM main.images WHERE id=?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:50, function dt_history_snapshot_undo_create(): prepare "SELECT MAX(id) FROM memory.undo_history WHERE imgid=?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:62, function dt_history_snapshot_undo_create(): prepare "INSERT INTO memory.undo_history SELECT ?1, imgid, num, module, operation, op_params, enabled, blendop_params, blendop_version, multi_priority, multi_name FROM main.history WHERE imgid=?2" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/history_snapshot.c:75, function dt_history_snapshot_undo_create(): prepare "INSERT INTO memory.undo_masks_history SELECT ?1, imgid, num, formid, form, name, version, points, points_count, source FROM main.masks_history WHERE imgid=?2" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/image_cache.c:235, function dt_image_cache_write_release(): prepare "UPDATE main.images SET width = ?1, height = ?2, filename = ?3, maker = ?4, model = ?5, lens = ?6, exposure = ?7, aperture = ?8, iso = ?9, focal_length = ?10, focus_distance = ?11, film_id = ?12, datetime_taken = ?13, flags = ?14, crop = ?15, orientation = ?16, raw_parameters = ?17, group_id = ?18, longitude = ?19, latitude = ?20, altitude = ?21, color_matrix = ?22, colorspace = ?23, raw_black = ?24, raw_maximum = ?25, aspect_ratio = ROUND(?26,1) WHERE id = ?27" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:220, function dt_image_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id and i.id = ?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:305, function dt_image_path_append_version(): prepare "SELECT version FROM main.images WHERE id = ?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:220, function dt_image_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id and i.id = ?1" 23,095465 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:248, function _image_local_copy_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id AND i.id = ?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:3269, function dt_exif_xmp_read_data(): prepare "SELECT filename, flags, raw_parameters, longitude, latitude, altitude, history_end, datetime_taken FROM main.images WHERE id = ?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/iop_order.c:314, function dt_ioppr_get_iop_order_version(): prepare "SELECT version FROM main.module_order WHERE imgid = ?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/iop_order.c:605, function dt_ioppr_get_iop_order_list(): prepare "SELECT version, iop_list FROM main.module_order WHERE imgid=?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:3220, function dt_set_xmp_dt_metadata(): prepare "SELECT key, value FROM main.meta_data WHERE id = ?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:3245, function dt_set_xmp_dt_metadata(): prepare "SELECT color FROM main.color_labels WHERE imgid=?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:633, function dt_set_darktable_tags(): prepare "SELECT COUNT() FROM memory.darktable_tags" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:665, function dt_tag_get_attached(): prepare "SELECT DISTINCT T.id, T.name, T.flags, T.synonyms, 1 AS inb FROM main.tagged_images AS I JOIN data.tags T on T.id = I.tagid WHERE I.imgid = 109 AND T.id NOT IN memory.darktable_tags ORDER BY T.name" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:1223, function dt_selected_images_count(): prepare "SELECT count() FROM main.selected_images" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:633, function dt_set_darktable_tags(): prepare "SELECT COUNT() FROM memory.darktable_tags" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:665, function dt_tag_get_attached(): prepare "SELECT DISTINCT T.id, T.name, T.flags, T.synonyms, 1 AS inb FROM main.tagged_images AS I JOIN data.tags T on T.id = I.tagid WHERE I.imgid = 109 AND T.id NOT IN memory.darktable_tags ORDER BY T.name" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/tags.c:1223, function dt_selected_images_count(): prepare "SELECT count(*) FROM main.selected_images" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:3042, function dt_set_xmp_dt_history(): prepare "SELECT imgid, formid, form, name, version, points, points_count, source, num FROM main.masks_history WHERE imgid = ?1 ORDER BY num" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/exif.cc:3095, function dt_set_xmp_dt_history(): prepare "SELECT module, operation, op_params, enabled, blendop_params, blendop_version, multi_priority, multi_name, num FROM main.history WHERE imgid = ?1 ORDER BY num" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/history.c:1385, function dt_history_hash_read(): prepare "SELECT basic_hash, auto_hash, current_hash FROM main.history_hash WHERE imgid = ?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:2067, function dt_image_write_sidecar_file(): prepare "UPDATE main.images SET write_timestamp = STRFTIME('%s', 'now') WHERE id = ?1" 23,111111 [sql] C:/msys64/home/Graham/darktable/src/common/image_cache.c:235, function dt_image_cache_write_release(): prepare "UPDATE main.images SET width = ?1, height = ?2, filename = ?3, maker = ?4, model = ?5, lens = ?6, exposure = ?7, aperture = ?8, iso = ?9, focal_length = ?10, focus_distance = ?11, film_id = ?12, datetime_taken = ?13, flags = ?14, crop = ?15, orientation = ?16, raw_parameters = ?17, group_id = ?18, longitude = ?19, latitude = ?20, altitude = ?21, color_matrix = ?22, colorspace = ?23, raw_black = ?24, raw_maximum = ?25, aspect_ratio = ROUND(?26,1) WHERE id = ?27"

johnny-bit commented 4 years ago

Can you attach logs? The pasted ones are a bit unreadable due to github design not spanning whole window :/

GrahamByrnes commented 4 years ago

Hmmm, they should be in users/[user]/appdata/local/temp ? I see a bunch of almost empty .log files, nothing that appears to hold enough text. Something else I need to set to ensure the trace is saved?

I could cut & paste to a text file, but the lines are already folded by the msys2 window. (And most of it is now gone from the window buffer, should be able to re-produce though).

Good weather here for doing bisections, it sounds just the tool I need... but each build is kind of slow :-)

Le sam. 29 févr. 2020 à 19:22, Hubert Kowalski notifications@github.com a écrit :

Can you attach logs? The pasted ones are a bit unreadable due to github design not spanning whole window :/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=AN2X2RSMQNMFIGKPNDDP43TRFFI6XA5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENMBXKQ#issuecomment-592976810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2X2RVJBHKRQ3KXPLMI4HDRFFI6XANCNFSM4K6V5SVQ .

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

GrahamByrnes commented 4 years ago

Yesterday I reset to the current master, then applied git bisect to localise the source of the problem, going back roughly as far as my last good copy. The problem appears to be in the rather large commit by @phweyland which is unfortunate given its size: commit 126821631f5680461d94732f2af0bfa15750707a Author: phweyland philippe.weyland@laposte.net Date: Mon Feb 24 07:54:26 2020 -0300

fixes db migration

src/common/collection.c | 12 +++-- src/common/database.c | 118 +++++++++++++++++++++++++++++++++++++++--------- src/common/exif.cc | 21 +++++---- src/common/history.c | 106 +++++++++++++++++++++---------------------- src/common/history.h | 10 ++-- src/common/image.c | 3 ++ src/develop/develop.c | 5 +- src/libs/collect.c | 9 ++-- 8 files changed, 185 insertions(+), 99 deletions(-)

I've looked through it and it all looks fine to me... but then it would, since it runs on Linux...

GrahamByrnes commented 4 years ago

There appears to be an error writing the hash to the db: "no such column" perftrace1mars2020.txt

johnny-bit commented 4 years ago

add column basic_hash to the table. It seems like @phweyland might've missed it on the commit. Try testing with manually added column, so the sqlite problem is not causing your issue.

niedz., 1 mar 2020 o 11:50 GrahamByrnes notifications@github.com napisał(a):

There appears to be an error writing the hash to the db: "no such column" perftrace1mars2020.txt https://github.com/darktable-org/darktable/files/4271665/perftrace1mars2020.txt

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=ACRRKFJKV5I25MHTRD7QTIDRFI4WNA5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENM3SLY#issuecomment-593082671, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACRRKFLOJ2S7F7GYTPPI6ILRFI4WNANCNFSM4K6V5SVQ .

-- Pozdrawiam, Hubert Kowalski

GrahamByrnes commented 4 years ago

Sun is out, first I go drink coffee and make some new photos :-)

Le dim. 1 mars 2020 à 12:26, Hubert Kowalski notifications@github.com a écrit :

add column basic_hash to the table. It seems like @phweyland might've missed it on the commit. Try testing with manually added column, so the sqlite problem is not causing your issue.

niedz., 1 mar 2020 o 11:50 GrahamByrnes notifications@github.com napisał(a):

There appears to be an error writing the hash to the db: "no such column" perftrace1mars2020.txt < https://github.com/darktable-org/darktable/files/4271665/perftrace1mars2020.txt

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=ACRRKFJKV5I25MHTRD7QTIDRFI4WNA5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENM3SLY#issuecomment-593082671 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACRRKFLOJ2S7F7GYTPPI6ILRFI4WNANCNFSM4K6V5SVQ

.

-- Pozdrawiam, Hubert Kowalski

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=AN2X2RXYA65T4Z53LQYYE7TRFJA7TA5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENM4N4A#issuecomment-593086192, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2X2RUJAOWXWH4CAVKUTNTRFJA7TANCNFSM4K6V5SVQ .

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

GrahamByrnes commented 4 years ago

Yes! That seems to have fixed it :-) So how should it be fixed properly? Something in the db update procedure? I've no idea how that works...

johnny-bit commented 4 years ago

Wait... "fixed" what exactly? if adding basic_hash was enough to fix your problems then maybe your db has wrong version synced in it and is lacking proper columns? If it fixed only the commit you're bisecting, you still have whole bisecting to do ;)

TurboGit commented 4 years ago

As far as I see basic_hash is properly created in database.c so something really strange on your side.

GrahamByrnes commented 4 years ago

It allowed yesterday's current master to run without crashing. Prior to that, I was deleting data.db at each bisect. However, when it closed, there was the same "no column" error. Re-opening data.db it was as before: basic_hash was gone and replaced with "initial_hash". Replaced again, ran an almost-current version, which found the data.db, asked to update, did so, Again, on shutting down, no such column.

That is what I saw when I first opened the table history_hash: 4 columns, primary key and 3 hash values, first one being initial_hash. I naively added a 5th column. I can't find "initial_hash" anywhere in the code, so I'm supposing this is an SQL initialisation that has somehow come back at the end.

I'll try again with the current master...

johnny-bit commented 4 years ago

Clear darktable config would be simply no data.db and no library.db... I mean, last time I've checked data.db schema was ver 6 while library.db was 24?

phweyland commented 4 years ago

As commented some days ago, my commits about history_hash cannot be safely played one after the other. At least restart each time with a db which has not the table history_hash... If not this table is not aligned with the commit because the migration has already been executed (with different columns names).

GrahamByrnes commented 4 years ago

My bad... global sub "library.db" for "data.db" above.

I had deleted everything except darktablerc while bisecting, but my understanding of "add a row by hand" was to not delete, but modify it. Wrong move?

This is what I see under the DDL for history_hash: CREATE TABLE history_hash (imgid INTEGER PRIMARY KEY, initial_hash BLOB, auto_hash BLOB, current_hash BLOB);

GrahamByrnes commented 4 years ago

@phweyland Ok, I thought I had done that but I'll re-check more carefully. It's possible that at 1am I screwed up.

GrahamByrnes commented 4 years ago

Ok, built new version. Dir empty excepting darktablerc Opened a folder, LT -> DT, ok, click "apply style" -> no styles loaded (ok), click LT, crash.

Checked library.db, this time has correct column names. Closed, left in dir with darktablerc, removed the rest (including lock files)

Re-launched: segmentation fault in GLib.

Backtrace: darktable_bt_PJMXG0.txt

Now I should eat :-(

Tried one more time with a clean directory... again apparently random crash. Was running -d all, but no backtrace... the screen version offered no clues, last para:

86,957733 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:220, function dt_image_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id and i.id = ?1" 86,957733 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:305, function dt_image_path_append_version(): prepare "SELECT version FROM main.images WHERE id = ?1" 86,957733 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:220, function dt_image_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id and i.id = ?1" 86,957733 [sql] C:/msys64/home/Graham/darktable/src/common/image.c:248, function _image_local_copy_full_path(): prepare "SELECT folder || '\' || filename FROM main.images i, main.film_rolls f WHERE i.film_id = f.id AND i.id = ?1"

TurboGit commented 4 years ago

The backtrace makes sense to me and I have an idea of the possible issue. I'll try a fix.

johnny-bit commented 4 years ago

Umm... the backtrace makes it related to tagging module - could it be related to #4407?

GrahamByrnes commented 4 years ago

I specifically remember NOT touching the tagging module before any crash... but yep, there it is tagging.c (shrug).

GrahamByrnes commented 4 years ago

This morning I thought of looking at the tagging...; thinking the problem might be coming from the tags embedded in the xmp's. So in a bare directory I: a) opened yesterday's current master (no problem) and imported a bunch of styles; b) removed all the xmp's from a directory of 41 images; c) imported the above images.

dt crashed during the import, stopping early. No obvious sign in the trace. I then opened the library and looked at the tagging table: weird. rowid imgid tagid 1 1 1 2 1 2 3 1 3 4 2 1 5 2 2 6 2 3

and so on... 18 rows, 6 imgid, tags 1,2,3 for each. Is this normal, every image gets 3 default tags?

GrahamByrnes commented 4 years ago

Ohhhh... no xmp's, images still treated on import. So there would appear to be still the internal LR type storage of edits... even though these photos are recent enough to have never seen LR.

Other thing: opened first in a version preceding history_hash. Stable as expected, no dramas. Close, leave directory alone. Created library.db is ver 23. Re-open without deletes using a current version... asks about updating db's, yes. And it's stable! Checked after closing, library has moved to version 24.

Conjecture: editing info contained inside dng files even with no xmp's. If run first through an old version to create a v23 library, this triggers the library update when opening in a v24 version, and this converts both internal & external edit info. Results in stable db.

If this is not done, there is a conflict between v24 library created ex-nihilo and the internally stored edit data (which is not obviously visible... will need to use exiftool). Result is crash.

TurboGit commented 4 years ago

Can you try with the latest version ? I have pushed a possible fix this morning. Thanks.

GrahamByrnes commented 4 years ago

Thanks, will try tonight... at work for a day of stats programming...

Le lun. 2 mars 2020 à 09:45, Pascal Obry notifications@github.com a écrit :

Can you try with the latest version ? I have pushed a possible fix this morning. Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4400?email_source=notifications&email_token=AN2X2RTM3APAPU6VTUK45UTRFNW3PA5CNFSM4K6V5SV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENONRWA#issuecomment-593287384, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2X2RXPT76AFZH2RI6FJS3RFNW3PANCNFSM4K6V5SVQ .

-- Graham Byrnes Bron (Lyon), France Mes photos: https://500px.com/grahambyrnes

phweyland commented 4 years ago

and so on... 18 rows, 6 imgid, tags 1,2,3 for each. Is this normal, every image gets 3 default tags?

yes I think, there should be darktable tags