Missing lens model in EXIF data

[jcubic]

Describe the bug Exported JPG from NEF file is missing lens model information (in Lens Id Exif tag).

To Reproduce Export jpeg files from NEF files (Nikon)

kuba@jcubic:~/Zdjecia/Praga/darktable_exported$ exiftool DSC_6447.jpg | grep -i lens
Lens Type                       : E VR
Lens                            : 15-30mm f/2.8
Lens F Stops                    : 6.00
Lens Spec                       : 15-30mm f/2.8 E VR
kuba@jcubic:~/Zdjecia/Praga/darktable_exported$ exiftool ../DSC_6447.NEF | grep -i lens
Lens Type                       : E VR
Lens                            : 15-30mm f/2.8
Lens Data Version               : 0204
Lens ID Number                  : 202
Lens F Stops                    : 6.00
Lens ID                         : Tamron SP 15-30mm f/2.8 Di VC USD G2 (A041)
Lens Spec                       : 15-30mm f/2.8 E VR

Expected behavior Lens ID should be in JPG file EXIF data

Platform (please complete the following information):

• Darktable Version: darktable-2.6.3-2.fc30
• OS: Fedora 30 GNU/Linux

Additional context Because of this there is missing Lens information on Flickr https://www.flickr.com/photos/jcubic/49844418938/in/dateposted-public/ It was reported to Flickr but it seems that lens information is not in EXIF data that why it's missing on photos. I'm not sure if it's specific to Nikon, I have another compact camera with raw files I can check later and comment in this issue when Lens Id is also missing.

[phweyland]

Comparing on Flicker information from other contributors in the group Tamron 35.0-150.0 mm I see these fields (which are missing in your case):

Lens Info - 35-150mm f/2.8-4
Lens Model - 35.0-150.0 mm f/2.8-4.0 

So I suggest you make the test of setting Exif.Photo.Model Exif.Photo.LensModel and Exif.Image.Info Exif.Image.LensInfo with the manual text "Tamron 35-150mm f/2.8-4 Di VC OSD" or whatever else in exported metadata. Sending the image to flicker you could be able to see the corresponding information.

will send another email to Flickr support will see what they would say.

In any case it would be very interesting to know which actual standard exif tags Flicker does use.

[esq4]

@jcubic Compare:

a@debian9:/tmp$ exiftool ./badlands-moon.nef | grep Lens
Lens Type                       : G VR
Lens                            : 70-200mm f/2.8
Lens Data Version               : 0201
Lens ID Number                  : 119
Lens F Stops                    : 6.00
Lens ID                         : AF-S VR Zoom-Nikkor 70-200mm f/2.8G IF-ED
Lens Spec                       : 70-200mm f/2.8 G VR
a@debian9:/tmp$ exiftool -v ./badlands-moon.nef | grep Lens
  | | | 17) LensType = 14
  | | | 18) Lens = 70 200 2.8 2.8 (70/1 200/1 280/100 280/100)
  | | | 22) LensFStops = H..
  | | | 32) LensData0201 (SubDirectory) -->
  | | | | LensDataVersion = 0201
  | | | | LensIDNumber = 119
  | | | | LensFStops = 72

and from flickr with lens info:

a@debian9:/tmp$ exiftool ./50248947136_f3ec4f791b_o.jpg | grep Lens
Lens Type                       : E VR
Lens                            : 16-80mm f/2.8-4
Lens F Stops                    : 6.00
Lens Model                      : Nikon AF-S DX Nikkor 16-80mm f/2.8-4E ED VR
Lens Spec                       : 16-80mm f/2.8-4 E VR
a@debian9:/tmp$ exiftool -v ./50248947136_f3ec4f791b_o.jpg | grep Lens
  | | | 6)  LensDataUnknown (SubDirectory) -->
  | | | 40) LensType = 78
  | | | 41) Lens = 16 80 2.8 4 (160/10 800/10 28/10 40/10)
  | | | 45) LensFStops = H..
  | | 35) LensModel = Nikon AF-S DX Nikkor 16-80mm f/2.8-4E ED VR
  | LensID = 173

flickr displays a LensModel tag.

I made a setup like this:

It works ( https://www.flickr.com/photos/miroru/50248947136/in/pool-3171613@N22/ ) Unfortunately, this customization is not obvious. I hope that it will be included in the FAQ.

[phweyland]

@esq4, the customization you have made should mimic the standard dt behavior (with exif data checked).

If you save the xmp file corresponding to that image (with exif data checked but without your customization) can you see in the xmp file "Exif.Photo.LensModel: Nikon AF-S DX Nikkor 16-80mm f/2.8-4E ED VR" ?

Sorry I've confused exif and xmp data.

So to conclude: Some exif data, out of camera, are not compatible with flicker. And what you did overcome this, providing the missing value.

[phweyland]

I hope that it will be included in the FAQ.

The metadata exportation is already fully described in dt manual. And its goal is to allows the user to provide (add, remove) the metadata he wants.

The problem described above is related to flicker which expects some values which are missing. Some other applications may expect other missing information. It's up to the user to understand this and provide them if possible.

I don't see which kind of information / text we should add on the dt FAQ. Suggestion ?

[esq4]

The problem described above is related to flicker which expects some values which are missing. Some other applications may expect other missing information. It's up to the user to understand this and provide them if possible. I don't see which kind of information / text we should add on the dt FAQ. Suggestion ?

I thought about it and agree that this is not the best solution. It is more correct and perhaps easier to add this tag to the default export preset (and there may be some other tags in the future). In any case, this tag will act as an illustrative example using the exported metadata editor.

[ptilopteri]

@esq4 if this satisfies you, please consider closing the issue

[jcubic]

I'm not at home right now so I can't test it, I don't have my laptop with me. I will see when I got back from my trip and close the issue if it works, which looks that it will. sent from Android26.08.2020 6:08 PM ptilopteri notifications@github.com napisał(a): @esq4 if this satisfies you, please consider closing the issue

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

[esq4]

One more point in favor of the tag LensModel: it is perfectly displayed in the "image infomation" section of darktable. Checked on linux without exiv installed.

[phweyland]

it is perfectly displayed in the "image infomation" section of darktable.

Yes, dt, based on exiv2 API, does its best to decode properly the camera exif tags. This is displayed in image information. But there are failures as @jcubic has shown.

This value is usable through the variable $(LENS), including to translate the lens model id into clear text when necessary or correct it, if wrong, when exporting the image (=> Exif.Photo.LensModel).

[esq4]

Yes, dt, based on exiv2 API, does its best to decode properly the camera exif tags... But there are failures as @jcubic has shown.

The LensModel tag is not a modern Nikon cameras (and many other) tag. In my example, it's added manually. Why not export from DT automatically without additional user settings?

[phweyland]

Why not export from DT automatically without additional user settings?

I'm not an expert in that domain but I had understood that exif data were matter of camera and should not be modified. So I'm a bit doubtful about this. That's one thing to let the user fix some error or missing information. That's another topic to make a systematic change. At least that deserves input from other people, and about exif rules. But again, this problem is a flicker one, which is not able to read camera exif.

[jcubic]

@esq4 I've tested with my Tamron 35-150 and it don't work. No lens info on Flickr and exiftool show exact same tags.

kuba@jcubic:darktable_exported$ exiftool DSC_8050_01.jpg | grep -i lens
Lens Type                       : E VR
Lens                            : 35-150mm f/2.8-4
Lens F Stops                    : 5.00
Lens Spec                       : 35-150mm f/2.8-4 E VR
kuba@jcubic:darktable_exported$ exiftool DSC_8050.jpg | grep -i lens
Lens Type                       : E VR
Lens                            : 35-150mm f/2.8-4
Lens F Stops                    : 5.00
Lens Spec                       : 35-150mm f/2.8-4 E VR

Maybe the problem is that the lens is not recognized by exiv2.

This is my image, https://www.flickr.com/photos/jcubic/50290407593/in/dateposted-public/ I've exported the file then I've added the tag and exported it again.

[phweyland]

I've tested with my Tamron 35-150 and it don't work. No lens info on Flickr and exiftool show exact same tags.

Could you please share the screenshots of image information (lens model) and your setup of metadata export (Exif.Photo.LensModel ...) ? (as shown in #6074)

[jcubic]

Using pl_PL locale. No exiv2 info just number. Selected image is same lens as the one on flicker.

[phweyland]

ok, so could you test with the metadata setting:

Exif.Photo.LensModel $(LENS/201/Tamron 35-150mm f/2.8-4 E VR) The / in f/2.8 may interfere. You may want to try with f2.8 instead.

you can set also in the same way: Exif.Image.LensInfo $(LENS/201/Tamron 35-150mm f/2.8-4 E VR)

[jcubic]

It don't work, even with known lens. But I'm using Darktable 3.1.0 from Fedora distribution. I can try the one from git tomorrow.

[jcubic]

What I have from exiftool is that data is empty:

kuba@jcubic:darktable_exported$ exiftool -v DSC_7566_01.jpg | grep -i lens
  | | | 6)  LensData0204 (SubDirectory) -->
  | | | 40) LensType = 4
  | | | 41) Lens = 11 11 4 4 (110/10 110/10 40/10 40/10)
  | | | 45) LensFStops = <..
kuba@jcubic:darktable_exported$ exiftool -v ../DSC_7566.NEF | grep -i lens
  | | | 34) LensType = 4
  | | | 35) Lens = 11 11 4 4 (110/10 110/10 40/10 40/10)
  | | | 39) LensFStops = <..
  | | | 46) LensData0204 (SubDirectory) -->
  | | | | LensDataVersion = 0204
  | | | | LensIDNumber = 191
  | | | | LensFStops = 60

[phweyland]

I'm using Darktable 3.1.0

Was already in 3.0.0. Should not make difference.

What does give $ exiv2 -PE yourfile.jpg | grep lens with Exif.Photo.LensModel $(LENS) then with Exif.Photo.LensModel $(LENS/201/Tamron)

Exiftool may be misleading here. dt uses Exiv2.

| | | | LensIDNumber = 191

Is the id 201 or 191 ? ...

[jcubic]

191 was from different image, I was testing the one that had lens info from exiv2. Sorry about that.

Both give this output:

Exif.NikonLd3.LensIDNumber                   Byte        1  201
Exif.NikonLd3.LensFStops                     Byte        1  F5.0
Exif.Nikon3.LensType                         Byte        1  D G VR
Exif.Nikon3.Lens                             Rational    4  35-150mm F2.8-4
Exif.Nikon3.LensFStops                       Undefined   4  5

But the same output is when I deleted all the tags in that modal.

[phweyland]

Could you please share a raw image with this problematic tamron lens ? The nef shared above is with a nikon recognized lens. Thanks

[jcubic]

This is the one that I've uploaded to flickr http://jcubic.pl/DSC_8050.NEF I will report those two lenses to exiv2 that are missing (I wanted to link my two raw files but I will need to create two jpeg with lens cap on).

[phweyland]

Seems to work as expected.

$ exiv2 -PE DSC_8050.jpg | grep -i lens
Exif.NikonLd3.LensIDNumber                   Byte        1  201
Exif.NikonLd3.LensFStops                     Byte        1  F5.0
Exif.Nikon3.LensType                         Byte        1  D G VR
Exif.Nikon3.Lens                             Rational    4  35-150mm F2.8-4
Exif.Nikon3.LensFStops                       Undefined   4  5
Exif.Photo.LensModel                         Ascii      23  Tamron 35-150mm F2.8-4

EDIT with the same settings and the nikon lens:

$ exiv2 -PE DSC_3862.jpg | grep -i lens
Exif.NikonLd3.LensIDNumber                   Byte        1  Nikon AF-S Nikkor 28-300mm f/3.5-5.6G ED VR
Exif.NikonLd3.LensFStops                     Byte        1  F5.3
Exif.Nikon3.LensType                         Byte        1  D G VR
Exif.Nikon3.Lens                             Rational    4  28-300mm F3.5-5.6
Exif.Nikon3.LensFStops                       Undefined   4  5.33333
Exif.Photo.LensModel                         Ascii      44  Nikon AF-S Nikkor 28-300mm f/3.5-5.6G ED VR

[jcubic]

Do I need to do something extra, maybe some settings that is not set by default? I have the same tag in this modal, but no "Exif.Photo.LensModel" in output for known and unknown lenses.

Are you sure that DT 3.1.0 is enough for this to work?

[phweyland]

Are you sure that DT 3.1.0 is enough for this to work?

I've just tested with 3.1.0-1209 and it fails. It works only with fixed value (not for $(LENS) but yes for "tamron"). ... I don't know why, sorry. I've checked it works for 3.2.1.

[jcubic]

Will try latest version, I've tried to compile from source but had some issues with /usr/lib/gcc/x86_64-redhat-linux/9/libgomp.so but just found that there is repo for Fedora 32 (link in readme) will install and test. If it work in 3.2.1 then it will probably also work on my computer.

[jcubic]

Ok it works. This should be documented somewhere (maybe in FAQ) or the tag should be set by default. With message that it's in version 3.2.1.

[phweyland]

With message that it's in version 3.2.1.

This feature was already in 3.0.0 if I'm not mistaken and documented. A regression has probably been introduced meanwhile.

[jcubic]

I have a question I've submited the files from missing lenses to exiv2 but they closed the issue because you can add those lenses in ~/.exiv2 dot file. I've added those lenses but the info panel on the left still have number instead of Lens full name.

$ exiv2 --grep lens/i DSC_8050.jpg
Exif.NikonLd3.LensIDNumber                   Byte        1  Tamron 35-150mm f/2.8-4 Di VC OSD (A043)
Exif.NikonLd3.LensFStops                     Byte        1  F5.0
Exif.Nikon3.LensType                         Byte        1  D G VR
Exif.Nikon3.Lens                             Rational    4  35-150mm F2.8-4
Exif.Nikon3.LensFStops                       Undefined   4  5

But the exported file have proper lens exif tag. Maybe it's just xmp file that store old value. Can I refresh the files without deleting my edits?

[jcubic]

It seems that Darktable don't refresh the exiv2 string on old exported photos:

$ exiftool DSC_8050_08.jpg | grep -i lens
Lens Type                       : E VR
Lens                            : 35-150mm f/2.8-4
Lens F Stops                    : 5.00
Lens Model                      : 201
Lens Spec                       : 35-150mm f/2.8-4 E VR
$ exiv2 --grep lens/i DSC_8050_08.jpg 
Exif.NikonLd3.LensIDNumber                   Byte        1  Tamron 35-150mm f/2.8-4 Di VC OSD (A043)
Exif.NikonLd3.LensFStops                     Byte        1  F5.0
Exif.Nikon3.LensType                         Byte        1  D G VR
Exif.Nikon3.Lens                             Rational    4  35-150mm F2.8-4
Exif.Nikon3.LensFStops                       Undefined   4  5
Exif.Photo.LensModel                         Ascii       4  201

exiv2 identified the lens but Exif.Photo.LensModel is 201 instead of the proper string. Do you know any way to update the data?

[phweyland]

you can add those lenses in ~/.exiv2 dot file.

Sounds like a good news... I've set up ~/.exiv2 for your lens on my system.

[nikon]
201=Tamron 35-150mm f/2.8-4 Di VC OSD (A043)

But for me that doesn't change anything in dt. selected image[s] / refresh exif doesn't do anything. Even the command line does give the same result as before.

The following command doesn't give the config_path. Maybe that's why.

$ exiv2 --verbose --version --grep config_path
exiv2 0.25 001900 (64 bit build)

The current exiv2 version is 0.27. Which may be another reason.

Can I refresh the files without deleting my edits?

selected image[s] / refresh exif doesn't jeopardize your edits.

[phweyland]

Is there a away to force refresh of database the same

For the existing files, have you tried:

selected image[s] / refresh exif

[jcubic]

I have exiv2 0.27.3, Fedora keep that package up to date. After updating exif, everything seems to work fine. Export also works. So for me the issue is solved, if you think that everything is documented and nothing need to be changed, you can close the issue.

Maybe it would be good idea to link here the link to docs where this is explained, and maybe one sentence in docs about flickr.

PS: I've deleted the comment because I've found your comment after commenting.

[phweyland]

the link to docs where this is explained

https://darktable.gitlab.io/doc/en/export_selected.html, including metadata export settings, variables definition and syntax.

In dt 3.2.1 3.0.0 some albums storage have been removed because their interface moves too often. This includes flickr.

you can close the issue.

I have no rights to do that. Normally that's up to the caller to close it.

[phweyland]

I have exiv2 0.27.3, Fedora keep that package up to date. After updating exif, everything seems to work fine. Export also works.

I've upgraded exiv2 and I confirm this. :)