Tamron lens not being recongnized and recognized after changin exiv2 config (on windows)

[ariznaf]

This is not really a bug. But I could not find a better place. There is a place to ask for camera inclusion but not for lenses.

Tamron SP 70-200 f2.8 Di VC USD (first version) lens is not recognized with canon cameras, at least not with my canon 80D and under Windows 10 64bits.

So automatic lens correction does not work, it is detected just as "255" lens. But the lens is listed in lensfun database.

After some research I have been able to config exiv2 to get it working configuring exiv2.ini under C:\Users\USERNAME\exiv2.ini This is the entry I created to get it working [canon] 255=Tamron SP 70-200mm f/2.8 Di VC USD A009

This is the thread where we discussed about the solution https://discuss.pixls.us/t/my-tamron-lens-is-not-automatically-detected/17939

I report it in case it can be added to the exiv2 lens database distributed with darktable for other users.

I was told that exiv2 is not distributed with darktable, but at least under windows it is. I had not previously installed exiv2 and it is not obviously a windows system library.

If it can be included in next update it would be great, if not, may be it helps other windows users of darktable with this lens.

[Nilvus]

Did you search on Exiv2 github? If this lens is not added on Exiv2, it will not be added on darktable as I said and even on Windows package. And on canon and related lenses, it doesn't seems to be on exiv2. Check here: https://github.com/Exiv2/exiv2/blob/master/src/canonmn_int.cpp

And the place for workaround is on pixl.us not here. Think that this place is to help developers to improve darktable code and fix darktable issues not libraries related that have their own repository. Then, when library is updated, darktable could show what the updated library have.

The traditional 70-200 is on but not the SP one. So, sorry but again: not a darktable issue. Think of closing this one (@TurboGit) and propose the add on Exiv2. Thanks.

[ariznaf]

Thank you I will try to open an issue there, if it can help to fix it in other versions.

As long as I know pixls is not the official forum of darktable developers.

I have develope some software too, time ago, and I problems or bugs with libraries that I used in my software a problem with my software. I did not expect users to know each detail of what library I used. If there was a problem with some library we tried to get a solution or bypass for it.

If the issue is seen by somebody that can help to fix it, perfect. If not, it can help other people with the same issue to fix it in their machine.

I did not opened this issue to polemize or disturb anybody, but for help others with the same problem.

I have solved it already for me.

[ariznaf]

As expected they reply me that it is a pity I have been directed to ask them.

https://github.com/Exiv2/exiv2/issues/1202#event-3319071961

255 seems to be the number assigned to any unknows lens (-1) by Canon. As it is a tamron lens in a canon camera, I guess any non canon lens will be assigned this number.

Exiv2 then tried other ways to identify the lens but it fails. So any other lens not from canon and not identified will have this ID.

I has not been provided of a way to provide exif data in order to get this lens correctly identified, so the problem does not seem to get solved in the near future from exiv2.

[Thanatomanic]

Please take a look here: https://discuss.pixls.us/t/my-tamron-lens-is-not-automatically-detected/17939/28

The Exif.Photo.LensModel tag provides the correct name for the lens. This should be forwarded to lensfun, but instead the Exif.CanonCs.LensType tag is used (which is useless in this case).

[Nilvus]

As expected they reply me that it is a pity I have been directed to ask them.

As the "pity" is for me, just take in account that's anyway your workaround is provided by exiv2 nor darktable. Some similar issues like yours have been, by the past, solved by pull requests posted merged on Github exiv2 (just check them and see, for example, all Tamron lenses added by default on their database). So for this man, the pity is that I didn't know that he stop merging lenses on their database since exiv2 0.26 and added configuration to let users do that for themselves. Good to learn that and so the only pity I see here is this word used. Seems that missing a change on an external library is not accepted. Glad to know that.

[Nilvus]

Just an add: I didn't know that 255 number was for that (sorry to not know all exiv2 details) but that doesn't change that your problem is, at first, related to exiv2 (your solution is provided by exiv2 library as this man says). @clanmills answers show that he know quite better things related to this than me (or probably most of darktable developers. Quite normal: exiv2 and lens identification is, at first, the speciality of exiv2 team, not darktable one.

[Thanatomanic]

@Nilvus I am confused why you still seem to claim this is an issue in exiv2. The information I linked to on the pixls forum clearly shows that the lens model is correctly identified by exiv2. However, due to mechanics I don't know, darktable picks another field of metadata and therefore only gets the "255". I assume it could/should also check the Exit.Photo.LensModel tag and try to match it with lensfun. Can you please clarify?

[ariznaf]

Nilvus, your are killing the messenger.

I only have said what I was answered there. I do thank your efforts in this topic, just you have to understand that a user of darktable cannot know the details of every library you use. If something does not work and he wants to help solve it, he is going to come to darktable to post the issue.

I do not know whether this is a problem in exiv2 or darktable, I don't know what exiv2 is supposed to do exactly.

I just have pointed a problem I had using darktable.

I am happy using darktable and grateful to the developers both, darktable developers and exiv2, that have given us the time and effort to have this tools working. May be there are problems, there always are in opensource and commercial software.

What I understand from the explanation is that 255 just means that canon does not recognize the lens that is mount in the camera. It is not strange as it is not a canon lens and the field is extracted from makernotes (a propietary field written by Canon, but some other lenses from sigma an id different of 255 is provided).

Other exif fields identify the lens correctly.

In that sense exiv2 is doing its job: correctly providing the fields store in exif data.

It seems that exiv2 usually does some magic and tries to guess the lens when it gets an unkown descriptor.

But there are articles from the developer stating that they are not going to provide that any more and that the config file of exiv2 was introduced as a patch (just to match the unknown 255 to a lens name, the problem will come when you have more than one unknows lens).

It seems that there are going to be changes in that aspect in the future, and that will affect darktable funcitionality. He talks about some scripting language to help configure the lens matching problem.

In this case Exif.Canon.LensModel and Exif.Photo.LensModel provide the correct name to the lens and corresponds with lensfun database.

Mapping 255 to a lens name is a patch that may work for some time... as long as there are few lenses that are not identified by exiv2. Because you cannot map it to two lenses… I only have one canon lens.

[TurboGit]

Other exif fields identify the lens correctly.

In that sense exiv2 is doing its job: correctly providing the fields store in exif data.

Which field (Exif.Photo.LensModel ?) ? Can you provide a RAW ?

[Nilvus]

@Nilvus I am confused why you still seem to claim this is an issue in exiv2. The information I linked to on the pixls forum clearly shows that the lens model is correctly identified by exiv2. However, due to mechanics I don't know, darktable picks another field of metadata and therefore only gets the "255". I assume it could/should also check the Exit.Photo.LensModel tag and try to match it with lensfun. Can you please clarify?

Just because it was fixed with the exiv2 config file and because most of the time lens id number is related to exiv2. And I didn't follow pixl.us thread. That means there was on start 2 issues:

• first one to associate lens id to name with the exiv2 file introduced in exiv2 0.25
• second one you point. This one seems to be darktable one indeed. But I can't help here for that.

So maybe the real issue would be that exif field, not the number id.

[clanmills]

Folks

I've been working on Exiv2 since 2008. The project began about 2004, I believe. Lens Recognition was added to Exiv2 before I joined the project.

Lens recognition is very troubling because the Exif Spec doesn't define how to record this in the metadata.

Exiv2 has two ways to report metadata. By value (option -pv) and By Interpretation (option -pt). Sometimes the conversion from value to interpretation is precise and well defined. For example, the orientation of the image. However sometimes it's vague - and that's what's happened with Lens Recognition, as I will explain. Because the Exif Standard doesn't define how to store the lens, the Manufacturers have adopted a hotchpotch of ways to store this information in their MakerNotes. For example, Canon have a LensID (in this case 255). The Manufacturer's makernotes are undocumented and the exiv2 code to handle them is based on reverse engineering. What does LensID=255 mean? We don't know.

The decision to provide Interpretation of the LensID has led to that being used to drive downstream image processing. I'm not a photographer. I have no idea what LensFun does. However to say it's the responsibility of Exiv2 to get this right at all times is a severe burden on Exiv2.

It the job of Exiv2 to correctly read/write/modify metadata according to the Standards. And that's what it does.

As you have realised, there are multiple elements of metadata in the image and they may even contradict each other. And none of this is caused by Exiv2. It's the data that's in the image.

On the pixls discussion forum there has been a suggestion that we hold a birds-of-a-feather discussion at LGM next year in Rennes in May 2021 to discuss ways in which the community can cooperate to address this. I intend to make a proposal which I discussed today with a user. https://github.com/Exiv2/exiv2/issues/1202#issuecomment-626142229

[ariznaf]

Other exif fields identify the lens correctly. In that sense exiv2 is doing its job: correctly providing the fields store in exif data.

Which field (Exif.Photo.LensModel ?) ? Can you provide a RAW ?

Of course here you have it

https://discuss.pixls.us/uploads/short-url/9mkVwZYgcIuNSfdQTuvk9eFMfSs.CR2

This are lens fields provided by exiv2

Exif.CanonCs.LensType Short 1 255 Exif.CanonCs.Lens Short 3 70.0 - 200.0 mm Exif.Canon.LensModel Ascii 74 TAMRON SP 70-200mm F/2.8 Di VC USD A009 Exif.Photo.LensSpecification Rational 4 70/1 200/1 0/1 0/1 Exif.Photo.LensModel Ascii 40 TAMRON SP 70-200mm F/2.8 Di VC USD A009 Exif.Photo.LensSerialNumber Ascii 11 0000000000

The offending field is lenstype in canon makernotes

Exiftool provides same value

[phweyland]

If I'm not mistaken you have not mentioned you had updated the lensfun database. It should be done by with lensfun-update-data. Would that help ?

[Thanatomanic]

Just because it was fixed with the exiv2 config file and because most of the time lens id number is related to exiv2. And I didn't follow pixl.us thread. That means there was on start 2 issues:

• first one to associate lens id to name with the exiv2 file introduced in exiv2 0.25
• second one you point. This one seems to be darktable one indeed. But I can't help here for that.

So maybe the real issue would be that exif field, not the number id.

@Nilvus @clanmills Unfortunately, the premise of this issue was wrong, causing a lot of confusion. A short investigation on Pixls showed that no modification to the exiv2 config file was actually ever needed in the first place. All the useful metadata is right there. So the title of this thread is unfortunately misleading for what the issue actually is. I'll repeat myself: there is nothing wrong with how exiv2 reads the metadata from this particular raw file.

@ariznaf shows exactly what @butcherg found as well, and that is:

Exif.CanonCs.LensType Short 1 255
Exif.CanonCs.Lens Short 3 70.0 - 200.0 mm
Exif.Canon.LensModel Ascii 74 TAMRON SP 70-200mm F/2.8 Di VC USD A009
Exif.Photo.LensSpecification Rational 4 70/1 200/1 0/1 0/1
Exif.Photo.LensModel Ascii 40 TAMRON SP 70-200mm F/2.8 Di VC USD A009
Exif.Photo.LensSerialNumber Ascii 11 0000000000

There seems to be a problem with which metadata field darktable uses to find the correct lens in lensfun. It seems to use the Exif.CanonCs.LensType and return '255', but should use the Exif.Photo.LensModel or Exif.Canon.LensModel tags.

@phweyland Updating lensfun makes zero difference here. The right lens is available in slr-tamron.xml:

<model>Tamron SP 70-200mm f/2.8 Di VC USD A009</model>
<model lang="en">Tamron SP 70-200mm f/2.8 Di VC USD</model>

[clanmills]

It sounds as though Exiv2 (and therefore me) are no long suspects "helping the police with their enquiries" I'm glad about that.

About 50% of the issues raised on Exiv2 concern Lens recognition. I wish Exiv2 had never become involved with in Lens recognition. I hope we will have a BOF discussion about this at LGM in May 2021.

[TurboGit]

Should be fixed in master.

[Nilvus]

@Nilvus @clanmills Unfortunately, the premise of this issue was wrong, causing a lot of confusion.

Indeed, most of the times is wrong premises causing lot of misunderstanding. The more important thing is now it's clear and the issue is finally fixed.

A short investigation on Pixls showed that no modification to the exiv2 config file was actually ever needed in the first place.

My mistake!

[TurboGit]

@Nilvus : Not really your fault, the way the lens exif data is read by dt (and probably due to difference in way maker are recording this info) is a mess if you ask me. The code I put in place certainly does not clean-up things. I suppose we would want to have a safe default if no proper match works.

[Nilvus]

@Nilvus : Not really your fault, the way the lens exif data is read by dt (and probably due to difference in way maker are recording this info) is a mess if you ask me. The code I put in place certainly does not clean-up things. I suppose we would want to have a safe default if no proper match works.

Good to read that. Thanks @TurboGit

[ariznaf]

Not my goal to look for cultripts. As you will have deduced, I am not native english speaker and it is difficult for me to express correctly.

Again, I am grateful for this software, I think developers have done a great job, that is why I am using it. And just wanted to contribute to solve a problem with a lens.

As an end user it seemed quite simple contribute the exif data for the lens to be added to a database. That is what I expected, no blaming intention. I apologize, Nilvus, if I gave you other impression.

Surprisingly, correctly identifiying a lens has come out to be a quite difficult task.

For this lens, usng Exif.Photo.LensModel if Exif.Canons.LensType is 255 would be the solution, but probably is not going to be a general solution.

May be having a config file that lets user add some rules for lens identification that overload the general rules would be a good solution.

Something that lets you specify a rule like: if Exif.Canons.LensType = 255 then use Exif.Photo.LensModel.

That would let final users fix the identifications of evasive lenses or where the general rules don't identify them correctly.

But I understand now it may be a task for exiv2, as it seems it has been doing the task of guessing the correct lens identification with some Heuristics.

For what @clanmills has told us it seems he is proposing something in that direction, with a simple scripting language. But it will take more than a year just to propose it in public.

Whatever you decide would be OK for me, as I have mentioned the patch of assigning 255 to the tamron lens works for me as it seems I don't have other unknown lens.

Thank you all for your time and great work.

[Thanatomanic]

For this lens, usng Exif.Photo.LensModel if Exif.Canons.LensType is 255 would be the solution, but probably is not going to be a general solution.

@ariznaf I'm sorry, but you miss the point entirely. There is no need for a general solution, because there is no general problem. exiv2 already does its job perfectly, without any need for a special config file. It was the interpretation of the metadata by darktable that needed a little modification.

Edit: also, please let's stop discussing here. That is not the purpose of this issue tracker. Any further discussion is better done on the thread in the pixls forum.

[ariznaf]

If I'm not mistaken you have not mentioned you had updated the lensfun database. It should be done by with lensfun-update-data. Would that help ?

I have not modified lensfun database. I just looked in lensfun database to see which was the correct name for the lens.

I am not an expert in lensfun either (I have read about it just right now that I am using darktable) but it seems it has a short name display or coloquial name for the lens and a list of detailed names.

The detailed name (model) matches exactly the Exif.Canons.LensModel and Exif.Photo.LensModel in the exif data.

[ariznaf]

For this lens, usng Exif.Photo.LensModel if Exif.Canons.LensType is 255 would be the solution, but probably is not going to be a general solution.

@ariznaf I'm sorry, but you miss the point entirely. There is no need for a general solution, because there is no general problem. exiv2 already does its job perfectly, without any need for a special config file. It was the interpretation of the metadata that needed a little modification.

I did not pretend to know how darktable or exiv2 make their job.

If I mentioned about the LensType field and the 255 being unknown is because I was addressed in Piexls.us to an article that explained that you had to map it to you lens in exiv2 and that was the solution.

I have provided what my situation was and the exif data of my raw.

It is great to read that the solution is simply changing the exif field.