search

darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.77k stars 1.14k forks source link

darktable complies "history problem detected" at importing tiff files #7156

Closed BigSerpent closed 1 year ago

BigSerpent commented 3 years ago

At importing tiffs darktable complies "history problem detected". Any further action in darkroom with such a file comes along with the same complain.

Git master at 3eb2253b956aadfaad885919138174c28b4af3d0 . Linux Mint 20 Mate x64.

With respect, Alexander Rabtchevich

johnny-bit commented 3 years ago

Any tiff file? Currently importing tiff and it looks like it's ok...

Can you share specific tiff file causing problem? (and if generated - xmp file?)

chhil commented 3 years ago

I have seen this message when I was re importing older tif files with xmps into current master. Don't have images that I can import to reproduce this.

BigSerpent commented 3 years ago

Here is a sample image with xmp file. The tiff was produced by darktable itself. https://yadi.sk/d/Y42WMnxQC4mwaw

ptilopteri commented 3 years ago

there is an error in your xmp file, I guessing around "colorout"

remove the xmp file (or rename) and the image from your collection and reimport there then is no error notice. I cannot see that there is any problem editing the tiff image with or w/o removing the previous edits (xmp file and image from db).

BigSerpent commented 3 years ago

The file had been exported to tiff and imported into darktable 5 minutes before I sent it to the Internet file server. So If I repeat import with another tiff file I will obtain the same error.

ptilopteri commented 3 years ago

only if you import a tiff image which has an accompanying xmp file contain similar "colorout" .

exported from ???

BigSerpent commented 3 years ago

No. The included xmp was created for the first time by darktable at importing.

chhil commented 3 years ago

This tiff has embedded xmp info. It was created by a previous export of a raw file DSC05363.ARW to /media/sasha/WD5_4/Photos/2020.11.28_park_Minsk_birds/darktable_exported/DSC05363_1600.tif in darktable.

I am not 100% sure if the xmp info embedded is used on import.

Tiff metadata info extracted using exiftool.... DSC05363.txt

I imported it into the current master 3.3.0+1953~g3b9113709 and did not get the warnings. The xmp that got created... DSC05363.tif.xmp.txt It's different from the xmp that got created for you and thats probably because of some changes where demosaic should be off etc. were updated in master.

I suggest you try this with current master.

On a separate note, after the import history looks like image On compress history an additional orientation shows up image Maybe this needs to be looked into.

ptilopteri commented 3 years ago

@BigSerpent again: what did you export the tiff image from?

ptilopteri commented 3 years ago

How did you generate the tiff image? can you provide the RAW image, not the tiff?

BigSerpent commented 3 years ago

Here is the RAW. https://yadi.sk/d/H8npp-AZb1hWRQ

ptilopteri commented 3 years ago

@BigSerpent I just imported your awr into dt master 75b8e51e1 and adjusted exposure adjust blackpoint in filmicrgb exported 16bit tif imported the new tif into a new dt session with --library :memory: and did not get the history error message you mention

the tif and xmp file from the awr and the imported tif are available: http://wahoo.no-ip.org/~paka/DSC05363.ARW.xmp http://wahoo.no-ip.org/~paka/DSC05363a.tif http://wahoo.no-ip.org/~paka/DSC05363a.tif.xmp

BigSerpent commented 3 years ago

Here is the terminal output at consequent image export and importing: [export_job] exported to /media/sasha/WD5_4/Photos/2020.11.28_park_Minsk_birds/darktable_exported/DSC05363_01.tif' [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 18246 (0x4746) encountered [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 18249 (0x4749) encountered [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 36867 (0x9003) encountered [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 50341 (0xc4a5) encountered [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 18246 (0x4746) encountered [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 18249 (0x4749) encountered [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 36867 (0x9003) encountered [tiff_open] warning: TIFFReadDirectory: Unknown field with tag 50341 (0xc4a5) encountered [dt_dev_pixelpipe_synch] found enabled demosaic in history for non-rawDSC05363_01.tif, id: 494201 [dt_dev_pixelpipe_synch] found enabled demosaic in history for non-rawDSC05363_01.tif, id: 494201 [dt_ioppr_transform_image_colorspace_cl] invalid conversion from 2 to 0 [_transform_matrix] invalid conversion from 2 to 0 [dt_ioppr_transform_image_colorspace] invalid conversion from 2 to 0 [_transform_matrix] invalid conversion from 2 to 0 [dt_ioppr_transform_image_colorspace] invalid conversion from 2 to 0 [dt_dev_pixelpipe_synch] found enabled demosaic in history for non-rawDSC05363_01.tif`, id: 494201 [_transform_matrix] invalid conversion from 2 to 0 [dt_ioppr_transform_image_colorspace] invalid conversion from 2 to 0

ptilopteri commented 3 years ago

but no "history problem detected" and if you run dt from your window manager menu, you don't get any notice on dt screen as before.

which is the title of this issue, yes?

BigSerpent commented 3 years ago

Nothing has changed - the same actions, the same message: "history problem detected" appear. The above terminal output corresponds that actions.

ptilopteri commented 3 years ago

do the following and report results

move ~/.config/darktable ~/.config/XXdarktableXX

start darktable and import the tiff file I provided, DSC05363a.tif

do you still see the problem?

to clean up what you just changed, rm -rf ~/.config/darktable mv ~/.config/XXdarktableXX ~/.config/darktable

and your system should be the same as it was before

ptilopteri commented 3 years ago

ps: quit darktable before doing the "cleanup"

chhil commented 3 years ago

This tiff has embedded xmp info. It was created by a previous export of a raw file DSC05363.ARW to /media/sasha/WD5_4/Photos/2020.11.28_park_Minsk_birds/darktable_exported/DSC05363_1600.tif in darktable.

I am not 100% sure if the xmp info embedded is used on import.

Tiff metadata info extracted using exiftool.... DSC05363.txt

I imported it into the current master 3.3.0+1953~g3b9113709 and did not get the warnings. The xmp that got created... DSC05363.tif.xmp.txt It's different from the xmp that got created for you and thats probably because of some changes where demosaic should be off etc. were updated in master.

I suggest you try this with current master.

On a separate note, after the import history looks like image On compress history an additional orientation shows up image Maybe this needs to be looked into.

The double orientation issue is going to be looked at here... https://github.com/darktable-org/darktable/pull/7225#issuecomment-739841379

BigSerpent commented 3 years ago

I've tried importing after .config/darktable had been moved and had no problems. So what is the solution? I can upload my darktable folder archived.

chhil commented 3 years ago

Usually db probkems are fixed by starting with a clean config. Your problems are resolved, nothing needs to be done at your end .

The reference above for orientation was just made to indicate it's been reported here too.

ptilopteri commented 3 years ago

I've tried importing after .config/darktable had been moved and had no problems. So what is the solution? I can upload my darktable folder archived.

either you discard the "moved" config and reimport your images or you need to go thru and determine which item is causing the problem, the library.db, data.db (probably not) and/or darktablerc.

how man images do you have in your collections? how much effort do you want to expend?

first step: mv ~/.config/darktable/darktablerc ~/.config/darktable/XXdarktablerc cp ~/.config/darktable/XXdarktableXX/darktablerc ~/.config/darktable/darktablerc

test tif import

if it works, rm ~/.config/darktable/* mv ~/.config/XXdarktableXX ~/.config/darktable

if not, mv ~/.config/darktable/XXdarktablerc ~/.config/darktable/darktablerc

then try repeat above substituting "library.db" for XXdarktablerc

if the convoluted steps I descriped do not work, I would suggest removing/deleting ~/.config/darktable and ~/.config/XXdarktableXX and reimporting your images.

note: I am a user not a developer

BigSerpent commented 3 years ago

The situation is still the same - importing tiff images reacts with wrong history message. The darktable is compiled from git master.

ptilopteri commented 3 years ago

you will need to provide a better explanation of what and how you did .... as your answer doesn't provide any point where something worked or did not.

BigSerpent commented 3 years ago

The actions are:

  1. Export tiff (I've tried 16 bit, no compession).
  2. Import it with scene-referred workflow (color adaptation is on). I've noticed, that white balance module of the imported image is disabled, but the module has a message, that several copies of the white balance module is detected. Here is a link to the archive of my darktable config folder. https://yadi.sk/d/3uf0r_P3MzhHoQ
BigSerpent commented 3 years ago

Hm, the same message on several copies of WB module is written for previously imported RAWs with scene-referred workflow enabled.

BigSerpent commented 3 years ago

In English: "a white balance applied twice"

johnny-bit commented 3 years ago

Very similar (perhaps the same) issue in #7534

github-actions[bot] commented 3 years ago

This issue did not get any activity in the past 30 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

komodo666 commented 3 years ago

For me it happens for all non RAW images. Please fix this, it's annoyng and this whas not happening before.

github-actions[bot] commented 3 years ago

This issue did not get any activity in the past 60 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

github-actions[bot] commented 1 year ago

This issue did not get any activity in the past year and thus has been closed. Please check if the newest release or master branch has it fixed. Please, create a new issue if the issue is not fixed.