Exiv2 / exiv2

Image metadata library and tools
http://www.exiv2.org/
Other
931 stars 281 forks source link

Add a decoder for Exif.Nikon3.FlashInfo #1941

Closed clanmills closed 2 years ago

clanmills commented 3 years ago

I received the following email:

Hi Mr. Mills,

Some time ago, I contacted a developer of your cousin project exiftool, because I had managed to decode some private Nikon data when using a radio flash trigger, showing some settings for each group. This person completed my findings, asking for images with certain combinations of settings, and added the result to their project.

In the meanwhile, I'm using only darktable which is based on exiv2, where this information is not available.

Is there any possibility to get Exif.Nikon3.FlashInfo also decoded in exiv2?

Thanks

Christoph


Here is my reply:

Christoph

Thank You for your email. Exiv2 has a good and friendly working relationship with Phil Harvey. An excellent engineer and very nice person.

In June, I retired from Exiv2 and will leave it to others to support and develop Exiv2 in future. I volunteered to be the release engineer for v0.27.5 which is scheduled for release on 2021-10-22. I released Exiv2 v0.27.5 RC3 today. So, I’m almost out of the project.

I recommend that you open a new issue on https://github.com/exiv2/exiv2 requesting these features. Please attach your sample files. Any information you have concerning Nikon3.FlashInfo will be appreciated. Phil has probably documented this on his web-site.

The tag is known to Exiv2:

580 rmills@rmillsmm-local:~/gnu/exiv2/team $ taglist ALL | grep Exif.Nikon3.FlashInfo
Nikon3.FlashInfo,168,0x00a8,Nikon3,Exif.Nikon3.FlashInfo,Undefined,"Flash info"
581 rmills@rmillsmm-local:~/gnu/exiv2/team $ 

We don’t decode it. Here’s an example with my D5300:

584 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiv2 -g FlashInfo ~/Stonehenge.jpg 
Exif.Nikon3.FlashInfo  Undefined  49  48 49 48 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
585 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ 

So there are 49 bytes which are "0106\0......\0". I know the flash was not used on that photo. I see there are notes relating to this tag here: https://exiftool.org/TagNames/Nikon.html#FlashInfo0106

I hope the people working on Exiv2 will help you. Please remember they are all volunteers with jobs, family and other demands on their time. In anticipation of retirement, I spent 6 months last year writing a book: Image Metadata and Exiv2 Architecture https://exiv2.org/book/index.html So how to deal with this is documented and I am willing to mentor/assist anybody who volunteers.

If you have good C++ skills, you may enjoy the challenge to bring this feature to life by joining Team Exiv2. I assure you of my support and assistance.

Robin


Christoph replied and declined my invitation to undertake this work. He didn't want to open an issue as he knows nothing about git. So, I offered to open this issue.

postscript-dev commented 3 years ago

I had a look at Exiftool's website and the Nikon tags are listed here and the pretty printing here (both found under FlashInfo). There are 7 FlashInfo groups listed, each applying to a different set of camera models. Adding every group would take a long time, however only adding the one Christoph's needs, would be much quicker.

As we don't know Christoph's camera model and don't having a sample image, it is unlikely that we can proceed.

clanmills commented 3 years ago

Thank you @postscript-dev for looking at this. For sure, we need a test image. This is unlikely to arrive following Christoph's private emails about his privacy.

clanmills commented 3 years ago

The architecture of tag decoding is discussed in my book in 7.7 Tag Decoder https://exiv2.org/book/#7-7

clanmills commented 3 years ago

@postscript-dev I'm going to reopen this as the user is now offering test files. He want to discuss this by email and he doesn't want his email address published. I don't know your email address, so I cannot forward you the correspondence. Can you shoot me your email address (via email).

cris46 commented 3 years ago

@postscript-dev, @clanmills: my cameras are Nikon D7100 and Nikon D850. The work done with Mr. Harvey from exiftool is a while back. In the meanwhile, Nikon updated the CLS (the remote triggering framework in Nikon), basically to use radio rather than IR, allowing for more channels and groups. Specially the groups should extend the codes we found back then. Nikon flash trigger gear is very expensive, so there are many people using third party hardware. As long as that hardware tries to emulate Nikon's TTL, they follow Nikons framework very closely. I use that from Phottix, which is recognized as a Nikon compact flash (SB-900 IIRC), in master mode.

clanmills commented 3 years ago

@cris46 Thanks for this information. I'm not a photographer, so I've never heard of CLS/TLS/TTL and those other terms. However, I know about metadata and 100% confident of success. Can you share some files from your cameras with the flash in use. And can you list the exiftool presentation of the FlashInfo metadata for your test files.

It's probably sufficient to extract the metadata from your image with the command:

$ exiv2 -ea --force --verbose image-file

The option -ea extracts all metadata (Exif, XMP, Iptc and ICC).
The option --verbose reports the location of the .exv file.
The option --force means it's OK to overwrite the .exv file.

@postscript-dev Are you willing to undertake writing the decoder for Christoph's cameras? I'm happy to mentor and steer you through this project. I have documented tag decoders in my book. The architecture is quite complex and sophisticated. However tag decoders are quite easy to code and test. You'll get a lot of satisfaction from getting this to work.

cris46 commented 3 years ago

The interesting point of FlashInfo is when using more than one flash, off camera, recording mainly the power settings of those flashes. CLS is a Nikon specific acronym which is related to the management of off camera flashes which are controlled remotely from the camera. TTL means ''through the lens''. There are fast test flashes fired and the camera measures the light through lens to figure out a "correct" power setting for the lights.

I've created a .exv-file and tried to append it here, but I get the message "we don't support that file type". So I packed it into a tar.gz (this is a MS platform after all).

It's not easy to copy here the output of FlashInfo, as this is one Exif-Tag which yields several output lines. So, I'll copy only the 6 lines I consider the most important:

Flash Group A Control Mode : Manual Flash Group B Control Mode : Off Flash Group C Control Mode : Off Flash Group A Output : 4% Flash Group B Compensation : 0 Flash Group C Compensation : 0

If you want the full output of exiftool, I can do that as well, but that's long. Just say so.

Today, I'll try to find the notes about at least part of the meaning of these 49 bytes.

_DSC6115.exv.tar.gz

kmilos commented 3 years ago

As Robin hinted, adding the decoder should be relatively easy, once one manages to do the hard part of reverse mapping what the bytes in the structure mean...

See the recently added Nikon3.Ld4 (Nikon3LensData ver 0800) code for a decoder skeleton...

clanmills commented 3 years ago

@kmilos Thanks for your helpful comments. The reversal engineering of the binary tag is painful. Thanks to Phil, he's both done that and documented many of those tags.

@cris46 You file has arrived safely and I see:

1006 rmills@rmillsm1:~/GoogleDrive/Exiv2 $ exiv2 -g FlashInfo/i ~/Downloads/_DSC6115.exv
Exif.Nikon3.FlashInfo         Undefined 163  48 49 48 56 1 48 5 1 1 128 0 255 0 255 0 0 0 6 0 28 30 42 96 0 0 0 0 0 0 0 0 0 1 1 16 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 85 85 85 85 80 0 0 0 0 .  .  .  .
#                                             0  1  0  8 . . .
1007 rmills@rmillsm1:~/GoogleDrive/Exiv2 $ 

So, you're using FlashInfo v0108 which Phil hasn't explicitly documented, however it's probably an extended version of 0107 which is documented: https://exiftool.org/TagNames/Nikon.html#FlashInfo0107

I hope @postscript-dev accepts the challenge of implementing Nikon3.FlashInfo.

cris46 commented 3 years ago

Oh. In 2018 I used the D7100, and today it was the D850 (because I had a file at hand). I'll look for one from the D7100.

postscript-dev commented 3 years ago

Sorry for the delay.

Looking at your uploaded file, ExifTool already processes the FlashInfo for your Nikon D850, and outputs it as the Nikon FlashInfo0107 Tags group.

C:\temp\>exiftool -D -*flash* _DSC6115.exv
37385 Flash                           : On, Return not detected
    8 Flash Setting                   : Normal
    9 Flash Type                      : Optional,Comdr.
   23 External Flash Exposure Comp    : 0
   24 Flash Exposure Bracket Value    : 0.0
   72 Flash Shutter Speed             : 1/60 s
   73 Flash Exposure Comp Area        : Entire Frame
   74 Auto Flash ISO Sensitivity      : Subject Only
  156 Modeling Flash                  : On
  135 Flash Mode                      : Fired, External
    0 Flash Info Version              : 0108
    4 Flash Source                    : External
    6 External Flash Firmware         : 5.01 (SB-900)
  8.1 External Flash Zoom Override    : No
  8.2 External Flash Status           : Flash Attached
  9.1 External Flash Ready State      : n/a
   10 Flash Compensation              : 0
   15 Flash GN Distance               : 0
 17.1 Flash Group A Control Mode      : Manual
 18.1 Flash Group B Control Mode      : Off
 18.2 Flash Group C Control Mode      : Off
   40 Flash Group A Output            : 4%
   41 Flash Group B Compensation      : 0
   42 Flash Group C Compensation      : 0

Your Nikon 7100 is included in the FlashInfo0106 group.

Are you asking for new tags that ExifTool does not support? or are you only interested in the ExifTool tags being transcribed into exiv2?

cris46 commented 3 years ago

I was reviewing the emails I exchanged in 2018 with Phil Harvey, when I realized, that my D850 is older than I remembered. So I was mistaken earlier. There I also saw, that the D850 had 163 Bytes for this tag. I didn't remember that.

Actually both. As a darktable user, it would be helpful to see this information when reviewing the images. But a remote flash setup can get pretty complex, so additional information is potentially always helpful. There are many number involved, so it's hard to remember even if I shot the images just 10 minutes earlier.

But back in 2018 I did not have hardware to make use of the newer version (D850), which in the meanwhile I do have. The thing is, the new version is not backward compatible. The lower channels force the older system, higher channels the newer one. As soon as at least one device has only 4 channels, nothing of the newer system can be used at the same time. And as long as the channel is low, only 3 groups can be used.

Reviewing the messages from 2018, I also recalled that there is a Windows program from Nikon (NXView, I believe) which of course knows to decode the new tag. I don't own Windows, but I know someone how does and has this NXView installed. So I plan to shoot an image using the new version with several groups and try to see what Information I can get from NXView. Knowing what information to look for was also helfpul in 2018.

postscript-dev commented 3 years ago

I will try and work on adding the new tags, but it wont be quick. I have less time on the run up to Christmas.

First, it is best to transcribe ExifTool's FlashInfo107 group into exiv2. Once this is completed, then we can look at FlashInfo106 and then any new tags after that.

clanmills commented 3 years ago

@postscript-dev Thank You, Peter. Don't bother with v0106, as that appears to be used by my 7 year old D5300. @cris46's image is using v0108. Usually with the Nikon structures, they get longer and retain backwards compatibility. So fields don't move about, new fields are added.

I don't think it's a big job to convert the definition on Phil's web-site into code for Exiv2 and write a python test that uses DSC6115.exv. Could be hours of works or a day or two (not weeks or months)_. I believe both @kmilos and @hassec have implemented similar things, so you're not on your own.

exiftool has a debugging feature -v5 which is very useful for understanding how metadata is identified and decoded:

$ exiftool -v5 _DSC6115.exv
....
  | | | 63) FlashInfo0107 (SubDirectory) -->
  | | |     - Tag 0x00a8 (163 bytes, undef[163]):
  | | |         2095: 30 31 30 38 01 30 05 01 01 80 00 ff 00 ff 00 00 [0108.0..........]
  | | |         20a5: 00 06 00 1c 1e 2a 60 00 00 00 00 00 00 00 00 00 [.....*`.........]
  | | |         20b5: 01 01 10 00 00 00 00 00 1c 00 00 00 00 00 00 00 [................]
  | | |         20c5: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
  | | |         20d5: 00 00 00 01 55 55 55 55 50 00 00 00 00 00 00 00 [....UUUUP.......]
  | | |         20e5: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
  | | |         20f5: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
  | | |         2105: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
  | | |         2115: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
  | | |         2125: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
  | | |         2135: 00 00 00                                        [...]
  | | | + [BinaryData directory, 163 bytes]
  | | | | FlashInfoVersion = 0108
  | | | | - Tag 0x0000 (4 bytes, string[4]):
  | | | |     2095: 30 31 30 38                                     [0108]
...

I don't want to get sucked into this FlashInfo issue. I'm working with David on the cr3/preview request this week #1893. I really haven't studied the preview code in the past, so I'm on a steep learning curve. I want to update the discussion of the preview code in my book. On Friday, I'm away for another one week vacation and when I return, I will release v0.27.5 on 2021-10-22. After that, I'm believe I'm retired.

cris46 commented 3 years ago

@postscript-dev um. of course, I'm happy with whatever support we can get in exiv2, whenever that might be. But Nikon did produce some rather good cameras, and I think there are still many cameras out there being used actively which are even older than 7 years. My D7100 also is from nov 2013, and I don't consider it obsolete. So, maybe these cameras shouldn't be discarded right away.

cris46 commented 3 years ago

Here is a file from the D7100. _DSC4962.exv.tar.gz

postscript-dev commented 3 years ago

Sorry for the delay.

I have completed adding the existing tags for the Nikon D850 (0108). When I started the work, I found that we already had NikonFl1 (FlashInfo0100), NikonFl2 (FlashInfo0102) and NikonFl3 (FlashInfo0103) groups defined. I have added a NikonFl7 group (FlashInfo0107).

Now when using the exiv2 tool with your test file, you will see:

$ exiv2 --grep NikonFl7 _DSC6115.exv
Exif.NikonFl7.Version                      Undefined   4  1.08
Exif.NikonFl7.FlashSource                  Byte        1  External
Exif.NikonFl7.ExternalFlashFirmware        Short       1  5.01 (SB-900)
Exif.NikonFl7.ExternalFlashData1           Byte        1  No external flash zoom override, external flash attached
Exif.NikonFl7.ExternalFlashData2           Byte        1  n/a
Exif.NikonFl7.FlashCompensation            SByte       1  0
Exif.NikonFl7.FlashFocalLength             Byte        1  n/a
Exif.NikonFl7.RepeatingFlashRate           Byte        1  n/a
Exif.NikonFl7.RepeatingFlashCount          Byte        1  n/a
Exif.NikonFl7.FlashGNDistance              Byte        1  None
Exif.NikonFl7.FlashGroupAControlData       Byte        1  Manual
Exif.NikonFl7.FlashGroupBCControlData      Byte        1  Off, Off
Exif.NikonFl7.FlashGroupAData              Byte        1  4%
Exif.NikonFl7.FlashGroupBData              Byte        1  0
Exif.NikonFl7.FlashGroupCData              Byte        1  0

You can find the ExifTool equivalent output in my post above.

@cris46: I will take @clanmills advice and leave working on the tags for the Nikon D7100, but will still help discover the new tags for your Nikon D850. Please start gathering together the sample images that demonstrate the new tags and then I can start to investigate.

cris46 commented 3 years ago

This looks good. thanks. There is just one little thing. The 4% are most certainly correct, but it's a way a photographer can't deal with easily. We use full stops and thirds between. It would be really helpful, if this could be translated. I guess a simple array should do it. With the files I attached, and the numers in this post, it should be easy to get the table right.

ver1.0.tar.gz

BTW. I believe that Nikon also allows to use one half-steps in between, but I don't know how to set it. It should be an option somewhere, but I can't find it. And I don't know anybody using it.

I didn't look for images having different codes, but shot a few, trying to be more systematic. As I mentioned before, there are two versions which can not be used together. The files I've attached are all the older version. I'm not sure how Nikon calls them, so I'll call them version 1 and 2, and this tarball has only version 1.

Image 6268 uses channel 1 and has the AF-Ill active (this means that while focussing the camera, the flash system is instructed to illuminate the scene to help the focussing system). There are two modes, one with three groups, and one with two. This file starts with 3 groups, called A, B and C. Each group can be off, in manual or in TTL mode. This starts in manual mode. Next is the power setting, which is currently output in %, but which is normally used as a fraction. This image starts at the lowest power setting (1/128). We could consider that as 1/128 +0/3, but usually it's just written as 1/128. And finally, there is a zoom setting which represents mm of focal length. I've shot enough images so you get all number of these series. This image (still 6268) has: A manual 1/128 zoom 17mm B manual 1/128 +1/3 zoom 18mm C manual 1/128 + 2/3 zoom 20mm

(actually 1/64 -1/3 and 1/128 +2/3 is the same and both are used. I suggest you stick with + values)

Image 6289 uses channel 2 and has AF-Ill off. The power settings are 1/64, 1/64 +1/3 and 1/64 +2/3, and the zoom settings are 24, 28 and 35mm (there are no values in between).

Image 6270 uses channel 3. AF-Ill was turned on for this and all further images. The power settings are 1/32, 1/32 +1/3 and 1/32 +2/3. Zoom was 50, 70 and 85mm

Image 6271 uses channel 4, the highest possible in version 1. The powers are 1/16 plus 0, plus 1/3 and plus 2/3. Zoom was 105, 120 and 135mm.

Image 6272 and all further uses channel 1 again. Power is 1/8 plus 0, plus 1/3 and plus 2/3. Zoom was 180, 200 and 200mm. 200mm is the largest focal length available. For the rest, I left zoom always at 50mm.

Image 6274 has power 1/4 plus 0, plus 1/3 and plus 2/3

Image 6275 has group A at power 1/1. Of course, there is not more than that. Group B was turned off (then there is no power and no zoom). And group C was switched to TTL mode which specifies the power in terms of plus or minus n EV, and I started with +0.0 EV.

Image 6276 has all three groups in TTL, with +0.3, +0.7 and +1.0 EV.

Image 6277 is the same, but power levels are +1.3, +1.7 and +2.0 EV

Image 6278 is the same, power: +2.3, +2.7 and +2.0 EV

Images 6279, 6280 and 6281 are the same, but power levels are -0.3, -0.7 and -1.0; -1.3, -1.7 and -2.0; and -2.3, -2.7 and -3.0 EV.

The second mode is a balancing mode. There are only two groups, which are supposed to be left and right, and power can be shifted from one side to another. The display says it's the ratio of groups A:B as 1:1, 1.5:1, 2:1, 3:1, 4:1, 6:1 and 8:1 shifting left (to group A), and 1:1, 1:1.5, 1:2 etc. shifting right to group B. At the same time the whole power can be weighted by an EV setting which can be one of -3.0, -2.7, -2.3, -2.0, -1.7, -1.3, -1.0, -0.7, -0.3, -0.0, +0.3, +0.7, +1.0, +1.3, +1.7, +2.0, +2.3, +2.7 and +3.0EV.

Image 6282 is the neutral setting (AB = 1:1 and +0.0EV). Image 6283 is A:B=1.5 and +0.3EV.

If you need more images with these combinations, let me know.

When the batteries are charged, I'll do something similar with version 2 (unless you have a better suggestion). The way to identify the protocol version is the channel. Channels 1 to 4 are necessarily the older version (1), and channels 5 to 32 are the newer (2) version. The new system can set channles 1-4 as well, but then they'll operate in a compatibility mode. Channels exist to allow two or more photographers using the same system coexist in the same room, without one triggering the flashes of another. Of course, there can be only one channel at the same time.

cris46 commented 3 years ago

ver2.0.tar.gz

Regarding the older version ("v1"), I forgot about two things. The first an automatic zoom setting according to the focal length of the lens as reported by the camera to the trigger device. To do this, there is a "mode" for the zoom which can be set to TTL (through the lens, meaning, whatever the camera chooses to report). In this setting, of course there is no manual adjustment for the zoom. The second thing is more of a hack, which possibly is not reported to Nikon at all. It's called ODS (by Phottix) and it is a delay of the flash trigger expressed in miliseconds (0.0 to 50.0 in steps of 0.1) Normally, when using flash, there is a sync time: the shutter must not be faster than that. If it is, banding or a black image is the result. However, with slow flashes, this can work nevertheless, but some delay would be needed. And that is where ODS comes in.

In image 6284 I've used channel 1, AF-Ill on, group A at manual power 1/8 and the zoom on automatic ("TTL"). Group B is also manual 1/2, the zoom at 50mm. And group C is on TTL with -3EV and a manual zoom for a 50mm lens. Additionally, I've set ODS to 0.1, hoping that this shows up somewhere. All other images had ODS at 0.0 (off).

The newer trigger can be in a mode for version 1 or 2, depending on the channel. In compatibility mode (version 1), things are almost identical to the trigger supporting only the older version. However, that automatic zoom setting (zoom mode TTL) is not supported, it must always be manual. Also, there is a global EV weighting which is set to +1.0EV.

I suspect that this is similar enough to the other pictures, so I shot only one. Should there be some difference, I can shoot more combinations.

Image 6285 has: channel 1, AF-Ill on, Group A manual 1/8 zoom 50. Group B manual 1/2, zoom 50. Group C on TTL with -3.0 EV, zoom 50. ODS was set to 0.2, and that global EV weighting was on +1.0 EV.

Images 6286 and 6287 are in version 2. There are channels 1 to 32 (but 1-4 is version 1 only) This is to avoid interference with several photographers/cameras in the same place. Additionally there is an ID (just an integer) which severs the same purpose. This has 5 groups (A to E) and allows the AF-Ill (auto focus illumination) to be set individually per group with a power level from 0 (off) to 9.

6286: channel 5, ID: 1234, AF-Ill is on (globally). All groups but C are manual, C is on TTL. Group A is 1/8, B is 1/2, D is 1/4 and E is 1/64. Group C has -3.0EV. Additionally, ODS was set to 0.2 (if that appears in Exif at all), and the global EV weighting was set to +1.0 All per group AF-Ill settings where 9.

Image 6287 is in balancing mode, i.e., only two groups shifting the power to one side or another.: channel 5, ID 1234, AF-Ill is on, A:B is 3:1, the global EV weight is +2.0 EV, zoom is 50mm for each group. ODS was on 0.2 and FEC (flash exposure compoensation) on +2.0

The last image, 6288, is also version 2. The two previous flash triggers where from Phottix and are called Odin and Odin II. The one for image 6288 is from Godox (X2T-N) and should behave like version 2. There are missing some functions, and there are some additional, and some functions are the same but have different names. As this has to play with the Nikon camera, internally, there shouldn't be so much of a difference. After all, it's the Nikon camera generating the Exif data. To get a feeling for this, I've made a test shot with this device too.

So 6288 has channel 5, ID 15, all groups on manual: A M 1/4 +0.5 B M 1/8 C M 1/2 +0.5 D M 1/32 E M 1/16 +0.5

(BTW, the association of percentages to fraction isn't linear but logarithmic; this makes it hard for the photographer to understand the percentages).

And here are two differences: This device allows to change the subdivision to 1/10 rather than 1/3, so it is possible to set it to +0.5. (besides, there is also a decimal system between 2 and 10 or 3 and 10, but I didn't use that) Also, this device does not allow different zoom settings for each group. There is just one for all, and that was set to 50mm. ODS is called ''high speed delay" and was on 0ms (off). BTW, while the Godox trigger doesn't distinguish groups regarding the zoom, it does have the TTL variant (which is in version 2), and it is called ''auto zoom''. So the lack of TTL-Zoom in Phottix v2 is not a protocol limitation.

I know the first trigger (Phottix Odin) best. The second I have not used very much, so I know it less. And the third I've never used in a shooting, so I know little about it. The Phottix and Godox triggers are incompatible with each other; and, of course, only one can be mounted on the camera at a time.

If you understood the working of a flash trigger just by this, without having used it (or seen someone using it), you are so much more intelligent than me... It took me quite a while to wrap my head around this. So if you have any question, just ask, and I'll try to explain.

clanmills commented 3 years ago

Let's give a HUGE round of applause to Peter @postscript-dev for working on this issue. Well Done, Peter. So pleased to see you dig in and make the effort to bring this to life.

And a little round of applause to @cris46 for working with Peter on this. I'm on vacation and haven't followed the details of this. @cris46 - pleased don't over burden Peter with this task. Remember, he's an unpaid volunteer.

postscript-dev commented 3 years ago

@clanmills: Thanks for the kind words.

@cris46:

This looks good. thanks. There is just one little thing. The 4% are most certainly correct, but it's a way a photographer can't deal with easily. We use full stops and thirds between. It would be really helpful, if this could be translated. I guess a simple array should do it. With the files I attached, and the numbers in this post, it should be easy to get the table right.

I will take a look at the files that you have submitted and if they match up, then we have understood the tag enough to make this change. Are you thinking 4% would be 1/25, 10% would be 1/10 and 25% would be 1/4 ?

I have some small jobs that need finishing, so for the next week I plan to work on those. After that I will study your messages and start investigating the files. Thank you for explaining the different flash features, this looks very helpful.

cris46 commented 3 years ago

As power of light is dealt with on a logarithmic scale, the association between percentage and fraction isn't that straight forward. Actually, when the values get small and there is some rounding, it's not obvious at all. That's why I included each possible power level in those files. Together with my description, it should be easy to solve this using a look up table. If you need more explanations or more sample files, just let me know.

postscript-dev commented 3 years ago

There's another job that I am trying to finish, but I have started to look into this.

In the example files that have been posted, there are 5 tags in the NikonFl7 group that we don't know anything about (the ones that have hexadecimal in the key). A full list of the NikonFl7 tags is:

$ exiv2 --unknown --grep NikonFl7 version1/_DSC6269.exv
Exif.NikonFl7.Version                        Undefined   4  1.08
Exif.NikonFl7.FlashSource                    Byte        1  External
Exif.NikonFl7.0x0005                         Byte        1  48
Exif.NikonFl7.ExternalFlashFirmware          Short       1  5.01 (SB-900)
Exif.NikonFl7.ExternalFlashData1             Byte        1  No external flash zoom override, external flash attached
Exif.NikonFl7.ExternalFlashData2             Byte        1  n/a
Exif.NikonFl7.FlashCompensation              SByte       1  0
Exif.NikonFl7.0x000b                         Byte        1  255
Exif.NikonFl7.FlashFocalLength               Byte        1  n/a
Exif.NikonFl7.RepeatingFlashRate             Byte        1  n/a
Exif.NikonFl7.RepeatingFlashCount            Byte        1  n/a
Exif.NikonFl7.FlashGNDistance                Byte        1  None
Exif.NikonFl7.0x0010                         Byte        1  0
Exif.NikonFl7.FlashGroupAControlData         Byte        1  Manual
Exif.NikonFl7.FlashGroupBCControlData        Byte        1  (96), Manual
Exif.NikonFl7.0x0013                         Byte       21  36 34 32 96 0 0 0 0 0 0 0 0 0 1 1 16 0 0 0 0 0
Exif.NikonFl7.FlashGroupAData                Byte        1  2%
Exif.NikonFl7.FlashGroupBData                Byte        1  2%
Exif.NikonFl7.FlashGroupCData                Byte        1  2%
Exif.NikonFl7.0x002b                         Byte      120  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 85 85 85 85 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Unfortunately in all the files supplied, of those 5 tags, the following ones always have the same value.

Exif.NikonFl7.0x0005                         Byte        1  48
Exif.NikonFl7.0x000b                         Byte        1  255
Exif.NikonFl7.0x0010                         Byte        1  0

This makes deciphering the purpose difficult but perhaps over time things may become clearer.

I had an idea that I could take the supplied raw metadata files, insert them into an empty JPEG and use the free Nikon software (NX Studio) to decipher the metadata. There is an insert bug in the exiv2 sample program (#1934) but working around that, the NX Studio output for version1/_DSC6269.exv (after reformatting) is:

+ File Info 1
File:                       Nikon_test2.jpg
Date Created:               23/10/2021 15:11:17
Date Modified:              23/10/2021 15:16:38
File Size:                  15.7 KB
Image Size:                 L (150 x 91), FX

+ File Info 2
Date Shot:                  15/10/2021 04:37:03.53
Time Zone and Date:         UTC+1, DST:OFF
Image Quality:              Lossless Compressed RAW (12-bit)
Artist:                     Cris
Copyright:                  Cris

+ Camera Info
Device:                     Nikon D850
Lens:                       50mm f/1.4G
Focal Length:               50mm
Focus Mode:                 AF-S
AF-Area Mode:               Single
VR:
AF Fine Tune:               OFF

+ Exposure
Aperture:                   f/8
Shutter Speed:              1/8s
Exposure Mode:              Manual
Exposure Comp.:             0EV
Exposure Tuning:
Metering:                   Spot
ISO Sensitivity:            ISO 100

+ Flash
Device:                     SB-900
Flash Mode:                 Front Curtain
Wireless flash options:     Optical AWL
Master:                     ---
Group A:                    M x 1/8
Group B:                    M x 1/2
Group C:                    TTL, -3.0EV (Camera: 0EV, Speedlight: -3.0EV)
Group D:                    ---
Group E:                    ---
Group F:                    ---

+ Image Settings
White Balance:              Natural light auto, 0, 0
Color Space:                Adobe RGB
High ISO NR:                OFF
Long Exposure NR:           OFF
Active D-Lighting:          OFF
Vignette Control:           OFF
Auto Distortion Control:    OFF

+ Picture Control
Picture Control:            [SD] STANDARD
Base:                       [SD] STANDARD
Quick Adjust:               0.00
Sharpening:                 +3.00
Clarity:                    +1.00
Contrast:                   0.00
Brightness:                 0.00
Saturation:                 0.00
Hue:                        0.00
Filter Effects:
Toning:
Location Info
Latitude:
Longitude:
Altitude:
Altitude Reference:
Heading:
UTC:
Map Datum:

I have only started looking at this and will post updates if I make some progress or have questions.

cris46 commented 3 years ago

First, I don't know the meaning of these 5 tags. Then I have some thoughts, the usefulness of which I also don't know. But here they go:

There may be some bytes reserved for a certain context, not being used in other contexts. And I'm not sure if those (then meaningless) bytes would be initialized to anything. I vaguely recall that I hit on such a suspicion last time.

I've used all combinations of settings withing the trigger at least once in those files. But:

TTL is an older idea which has been replaced by iTTL (Nikon) or eTTL (Canon). While TTL was unidirectional [camera->trigger-receiver(s)->flash(es)], iTTL is said to be bidirectional, i.e. also the same path backwards. I've never come across how this works in detail, but there should be such information around, as there are third party triggers around using it.

The trigger works no matter if it is actually mounted on the camera, or if there is actually a flash connected somewhere (although it is aware of both facts). When I generated those files I submitted in my previous messages, the trigger was indeed mounted on the camera, but there was no receiver or flash actually connected.

As said, I know very little about the backward flow of information. As a matter of fact, I have non-Nikon flashes which seem to behave just like the Nikon flashes, so I can't really see a difference in effect. But then, I also don't know if the flashes communicate with the camera, maybe even if the camera has no need for information (e.g., because everything is set to manual). If this would be the case, we might have a purpose for the bytes in those 5 tags.

The menu of the SB-910 flashes (the Nikon-branded flashes I have) is quite extensive, allowing for many exotic settings, even for triggering a stroboscopic light. So any menu setting of the flash potentially could be recorded in those bytes. Besides that, the SB-910 flashes are aware of some mechanical settings like the rotation or tilt of the flash heads, but also if a Nikon-branded color filter or diffuser is mounted on the flash head.

OTOH, the triggering configures only the power of the flashes and the zoom setting (position of the reflector within the flash to focus or broaden the light cone). So only these settings need to be the same for all flashes belonging to the same group (unless one is set to manual (*)). Everything else can be set differently in each flash, even if they are in the same group. So there might be three flashes configured for group A, but only one of them having a diffuser put on. I can't see a way that Nikon records so much detail in Exif, as this would require potentially a very large amount of data coming potentially from a hundred flashes. At the same time, I do remember that Mr. Harvey did discover some bytes regarding the diffuser and color filter settings (but I don't recall if that required to mount the flash directly on camera, without a trigger).

(*) The distinction between Manual and TTL is not only within the trigger (i.e, the information coming from the camera or a setting of the trigger). There is a similar but independent Manual or TTL setting in each flash: If the flash itself is set to manual, I can adjust the flash to anything I like. The receiver will only cause the flash to be triggered at a certain moment, but the power/zoom settings will be ignored by the flash. This is less comfortable, as I have to go to that flash (maybe climbing up to reach the flash), so I won't use such a setting unless I need more than 3 (or newer: 5) groups of power/zoom settings.

Trying to generate files with every thinkable combination of settings would generate a very large amount of sample files. For this reason, I think it would be more efficient, if you (@peter) could make a first guess, I'll shoot one or more files with those settings and we'll see where that leads us.

postscript-dev commented 3 years ago

For anyone following this, there are copies of the instruction manuals online which may help. See Nikon D850 and Nikon Flash SB-900.

postscript-dev commented 3 years ago

@cris46: I am sorry for not working on this more, I have been busy.

If anyone is interested, I have inserted the .exv files into JPEGs, so that they can be used with NX Studio. You can find the version1 files here and version 2 files here.

There is just one little thing. The 4% are most certainly correct, but it's a way a photographer can't deal with easily. We use full stops and thirds between. It would be really helpful, if this could be translated.

Today I looked at the Exif.NikonFl7.FlashGroupAData tag. It seems that there is not one universal measure, but each flash type (e.g. manual) has their own set of values (I only have manual and iTTL-BL sample files). I have made changes so that the output is the same as the NX Studio program. For example, now the version 1 files output:

$ exiv2 --Print kyvt --key Exif.NikonFl7.FlashGroupAData version1/*.exv
version1/_DSC6268.exv  Exif.NikonFl7.FlashGroupAData                Byte       42  1/128
version1/_DSC6269.exv  Exif.NikonFl7.FlashGroupAData                Byte       36  1/64
version1/_DSC6270.exv  Exif.NikonFl7.FlashGroupAData                Byte       30  1/32
version1/_DSC6271.exv  Exif.NikonFl7.FlashGroupAData                Byte       24  1/16
version1/_DSC6272.exv  Exif.NikonFl7.FlashGroupAData                Byte       18  1/8
version1/_DSC6273.exv  Exif.NikonFl7.FlashGroupAData                Byte       12  1/4
version1/_DSC6274.exv  Exif.NikonFl7.FlashGroupAData                Byte       6  1/2
version1/_DSC6275.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  1/1
version1/_DSC6276.exv  Exif.NikonFl7.FlashGroupAData                Byte       254  +2.3EV
version1/_DSC6277.exv  Exif.NikonFl7.FlashGroupAData                Byte       248  +1.3EV
version1/_DSC6278.exv  Exif.NikonFl7.FlashGroupAData                Byte       242  +0.3EV
version1/_DSC6279.exv  Exif.NikonFl7.FlashGroupAData                Byte       2  -0.3EV
version1/_DSC6280.exv  Exif.NikonFl7.FlashGroupAData                Byte       8  -1.3EV
version1/_DSC6281.exv  Exif.NikonFl7.FlashGroupAData                Byte       14  -2.3EV
version1/_DSC6282.exv  Exif.NikonFl7.FlashGroupAData                Byte       14  -2.3EV
version1/_DSC6283.exv  Exif.NikonFl7.FlashGroupAData                Byte       14  -2.3EV

and version 2 files output:

$ exiv2 --Print kyvt --key Exif.NikonFl7.FlashGroupAData version2/*.exv
version2/_DSC6284.exv  Exif.NikonFl7.FlashGroupAData                Byte       18  1/8
version2/_DSC6285.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  0EV
version2/_DSC6286.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  0EV
version2/_DSC6287.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  0EV
version2/_DSC6288.exv  Exif.NikonFl7.FlashGroupAData                Byte       15  1/4 (-1/2EV)

Note: Generally, the fractions are for manual and the EV for iTTL-BL.

There is one oddity though. NX Studio reports that the version1/_DSC6273.jpg value is 1/4 (exiv2 vanilla value 12) and that version2/_DSC6288.jpg is 1/4 (-1/2EV) (exiv2 vanilla value 15) - both using manual settings. This doesn't follow the +/-6 difference pattern used by the other sample files. Above, you did mention that the _DSC6288.exv file was different, although I am a little unclear how.

So 6288 has channel 5, ID 15, all groups on manual: A M 1/4 +0.5 B M 1/8 C M 1/2 +0.5 D M 1/32 E M 1/16 +0.5

And here are two differences: This device allows to change the subdivision to 1/10 rather than 1/3, so it is possible to set it to +0.5. (besides, there is also a decimal system between 2 and 10 or 3 and 10, but I didn't use that).

What is the difference between 1/4 and 1/4 (-1/2EV)? Is the notation A M 1/4 +0.5 the same as 1/4 (-1/2EV)?

I will look at the Exif.NikonFl7.FlashGroupBData and Exif.NikonFl7.FlashGroupCData tags which are hopefully very similar to the Exif.NikonFl7.FlashGroupAData tag.

cris46 commented 3 years ago

In manual mode, you set the absolute value of the power a flash device is able to emit. If a flash can emit at most 500Ws (Watt seconds), 1/1 means that. And 1/4 would mean 125Ws. And if I buy a second flash with 2000Ws, at max it's just 2 stops (2 EV) brighter.

Then there is the EV (https://en.wikipedia.org/wiki/Exposure_value). This is a measure for differences: If one of two power settings produces the double or half of the physical energy of light regarding the other, the difference is called "one stop" or 1.0 EV (exposure values).

So the difference between 1/2 and 1/1 is perceived as the same change in brightness as the difference between 1/64 and 1/32. This is a logarithmic relation.

There is one amount of light which makes the sensors particularly happy (it's related to the middle gray). It's a physical amount of light, the "luminous exposure" measured in lx.s (lux seconds). This is proportional to a particular amount of photons arriving at the sensor during the time the shutter is open or the flash is emitting light. This happy amount is often called 0.0 EV, such that +12EV would be 12 stops brighter requiring 12 times doubling that lux seconds of 0.0 EV arriving at the sensor. The scale based on those 0EV is represented by table 1 in the wikipedia page. Each setting can be produced by a host of combinations of ISO, aperture and shutter. The units are the same as for the EVs of flash power differences. So if there is iTTL +2EV you can read this as 0.0EV + 2EV and compare that to the table in wikipedia. My guess is that "vanilla value 12" is on such a scale.

Note that flash power does not make any statement of how much light will reach the sensor. So one difference between manual and TTL-mode is, that manual refers to light as emitted by the flash, and TTL as light arriving at the sensor. There is much too little information even in the camera to convert one into the other.

So in 1/4 -1/2EV (or simple 1/4 -1/2), the first fraction is absolute and regarding physical power emitted by the flash. The second fraction is a relative measure in EV, i.e., related to the double (if positive) or half (if negative), each of which would represent the difference of 1EV (doubling or halfing) of the first fraction. 1/4 power plus 1 EV is 1/2 power, and 1/4 power minus 1 EV is 1/8th power. In these cases, of course, we'd use directly 1/2 or 1/8; the notation is only useful if it's a fraction of that.

Cameras can be adjusted to subdivide 1.0 EV in thirds or halves. This controls the sequence of settings which can be made turning the camera's dials. But more values can be produced by the automatic modes. Some triggers allow for subdivisions as fine as 1/10th of an EV. But this is somewhat virtual: First, there is no guarantee that the flash device does support such a setting (it might be "rounding"). And the same for the camera, which might not be able to register that or there might not be an encoding for Exif. So I can't know--if a flash trigger allows me to set 1/4 +4/10--what value would arrive at the flash or at the Exif file. I do know that some fine subdivisions of my flash triggers are not supported by my Metz flashes, which has a somewhat undefined behavior (sometimes "rounding", but not always).

Now I can answer your questions:

1/4 is the absolute light emitted by the flash device. You can express that in Ws as soon as you know that device.

1/4 -1/2EV means that there is a power setting causing an emission of light of that flash device which is between 1/4 and 1/8th of the maximum power that device is able to emit (because of the minus sign). The 1/2EV means, that it is perceived to be right in the middle between 1/8 and 1/4. As this is a logarithmic concept, it is not equal to 1/8 + 1/16 (0.1875).

"A M 1/4 +0.5" means group A in manual mode at a power setting which is perceived as right in the middle of 1/4 and 1/2 of the full power that flash device is able to emit.

1/4 -0.5 is not the same. It's between 1/8 and 1/4, or, it's one stop darker than 1/4 +0.5

"1/4 +0.5" and "1/2 -0.5" however are synonyms. This is like in music: d sharp and e flat is the same (although in some cases, there is a subtle difference in music, but never in light measures)

Finally: Yes, Group[ABCDEF]Data must be analogue to each other. These groups can be used freely, When triggering just one single light, I can use any group I want, as long as the Group settings of the transmitter and receiver match. It is just a lower hierarchy of the channel/ID settings (which would distinguish all groups and flashes for one camera/transmitter from another).

postscript-dev commented 3 years ago

@cris46: Thanks for the help. Using your explanation, I have added intermediate values to the translated output for the Exif.NikonFl7.FlashGroupAData tag.

Manual mode:

Vanilla value Translated value
0 "1/1"
3 "1/1 (-1/2EV)"
6 "1/2"
9 "1/2 (-1/2EV)"
12 "1/4"
15 "1/4 (-1/2EV)"
18 "1/8"
21 "1/8 (-1/2EV)"
24 "1/16"
27 "1/16 (-1/2EV)"
30 "1/32"
33 "1/32 (-1/2EV)"
36 "1/64"
39 "1/64 (-1/2EV)"
42 "1/128"
45 "1/128 (-1/2EV)"

iTTL-BL mode:

Vanilla value Translated value
0 "0EV"
2 "-0.3EV"
4 "-0.6EV"
6 "-1EV"
8 "-1.3EV"
10 "-1.6EV"
12 "-2.0EV"
14 "-2.3EV"
242 "+0.3EV"
244 "+0.6EV"
246 "+1.0EV"
248 "+1.3EV"
250 "+1.6EV"
252 "+2.0EV"
254 "+2.3EV"

I think that the 2 tables cover all the available values but if there are translated values missing, let me know.

ExifTool also listed modes such as iTTL, Auto Aperture, Automatic, GN (distance priority) and Repeating Flash, but I don't have examples for those, so can only output percentages. It this is something that you want me to add, then I will need more example files (.exv). For a mode, I only require an example of the highest, lowest and, if available, zero'th value - also let me know what values and increments are used for that mode. I think that I can work out the rest if I have that information.

cris46 commented 3 years ago

In my previous explanation I tried to clarify things using the terms "absolute" and "relative". But also the "absolute" setting is actually relative, because 100% power doesn't say how much power is indeed emitted. So strictly, both are relative but refer to a different thing:

What I called "absolute" is actually relative to the power the device is able to emit. As the flash can be closer of further away from the subject, this doesn't tell how much light will be reflected from the subject and how much would reach the camera's sensor.

What I called "relative" is relative to an optimal lighting the camera has computed by measuring some pre-flashes through the lens (TTL). A setting perfectly aligned with that computation would be 0EV, a setting of -n EV would make that darker than "ideal" and +n EV lighter. This is "more" relative, because it depends on a varying value computed by the camera, while the previous refers to a maximum power which is not variable.

One thing to remember is, that doubling or halving any amount of light (be it emitted by a light source or measured as reaching the sensor) is called "one stop" or 1.0 EV, depending on context. This is a very important concept, as such stops also exist in the context of ISO, aperture and shutter timing. They are interchangeable, as I can open the aperture one stop and reduce flash power by one stop and would get the same final brightness. For all of them, Nikon internally uses a subdivision of 1/6th of "one stop" (or of one EV):

n-stop, 1/6, 1/3, 1/2, 2/3, 5/6 next-stop...

[I am sure this is accurate for the older version, but I guess it's also accurate for the newer one. If the other flash is able to adjust in subdivisions of 1/10th of a stop, it's a thing limited to the communication between trigger and compatible flash, but the camera and Exif-file probably would get just a rounded value.]

So you can make a scale and simply number each possible setting. Nikon starts numbering with zero (like an offset from a starting point rather than a count). In order to leave integers for subdividing settings, there need to be 5 integers reserved between one stop and the next. And this is how Nikon encodes these values: 0 is the highest power value, 6 is one stop below, 12 is two stops below, etc. As you can measure a distance of 10 inches and 1/2 or 11 inches minus 1/2 (referring to the same absolute distance), 2 stops and 1/3 (of a stop) is the same setting than 3 stops minus 2/3 (of a stop). You chose to use the minus variant, which is perfectly fine. The devices used the plus or minus variants depending on the last change.

The table for the manual mode is correct, but all integers exist. It depends on the UI settings of the device if I can select thirds or halves, but that's just the UI. I can always change that to get another subdivision, and that wouldn't change the encoding (which can deal with both as it is based on 1/6ths). There is no way to dial in 1/6 or 5/6 between two stops however. The difference of 1/6th of a stop is very little. So if I usually work in 1/3 subdivisions, I would accept 1/3 even if 1/2 would be (very slightly) better. OTOH, if the setting wasn't dialed in but chosen by the camera in some automatic mode, any 1/6th subdivision is possible to show up in Exif.

This is the reason, why the "vanilla value" of 0 is actually to be expected.

The values in the TTL table also look correct, but I would assume that the high nibble is just the sign. If these values come from the sample files, that sign is inverted: 0xf0 would mean positive and 0x00 negative (are you sure?). Then, you get the same scale as before with the only difference that the reference is in the middle of - and +: While the manual mode represents a fraction of the maximum power, where the maximum power is 100% (1/1), and there can't be more than that, such that the full stops are always +0, +6, +12…, in TTL mode 0 means the mathematically perfect setting as computed by the camera. This setting can be adjusted increasing or decreasing the brightness in the scene, such that you get full stops at …, -12, -6, 0, +6, +12…. But the intermediate values can again be added or subtracted: Logically, there could be a "+1EV +1/2" which is the same as "+2EV -1/2". The first number would tell how far this setting defers from the camera's computation, and the second how far that is between two full stops. Device displays (specially the viewfinder of the camera) have little space, so they often replace the second fraction by .3 for 1/3 and .7 for 2/3, and of course .5 for 1/2.

Regarding the additional modes: I'm not really sure how these work. These modes can only be set on the flash itself, not on the trigger nor the camera. But even if there are only 3 (or 5) groups, there can be many flashes belonging to each of those groups. Potentially there could be hundreds of flashes fireing at the same time. But Exif doesn't provide the tags for so many flashes. Maybe this is only when (exactly) one flash is mounted on the camera. Would this make sense?

Normally I use the flashes off camera (i.e., fired by a remote trigger), and normally, they are set to TTL mode, meaning that they take the setting from the electronic contacts on it's foot, believing it is mounted on the camera, while in fact it is mounted on a receiver. In the (rare) cases I need more different setting than available groups (as all members of a group receive the same adjustments), I can set two (or more) flashes to the same group to manual, in order to adjust the power for them individually.

Caveat: There is a difference between manual in the trigger and manual in the flash: Manual in the flash means, that the flash will fire when receiving the signal, but ignoring all external settings. Manual in the trigger means, that the setting sent to the receiver comes either from the user or from the camera. But if the flash is set to manual, the manual or TTL setting in the trigger won't have any effect on the flash. While the manual/TTL setting of the trigger is per group, the manual/TTL (etc) settings for the flash is per flash device, of which there can be many in each group.

Setting the flash to TTL can mean iTTL or "iTTL BL" likewise; it's the camera which considers the information received by the lens (iTTL) and the intensity of ambient light (BL=balanced) picking those sub-modes always automatically. I've never used the other modes (besides manual and repeating flash), but I would guess that Automatic and Auto Aperture are just variations of TTL, while GN is a variation of manual (setting a distance rather than a power level). Maybe you can tell me some context where these values appear in the Exif file (single on-camera flash? group related?), then I could make some tests to see if I find out how to use it.

What complicates understanding the different TTL modes is that between Nikon products, there is also a two-way communication between the flashes and the camera. There are pre-flashes emitting light which the camera can measure, but there are also codes transmitting binary data, as each flash also has sensors and a lot of logic to find out circumstances in it's environment. But I don't know the protocols this is based on. This is also a reason why I avoid certain modes, as they just yield a surprising result, which changes with small changes in the scene without me noticing.

The last mode is the "repeating flash". This is stroboscopic light, defined by a number of flash lights emitted and a frequency (time between one light and the next). This is the only other mode I've used (very rarely). There are some severe limitations, as flashes require time to recharge the capacitor, thus, the more flashes are fired, the less power each can emit. I can make a few test shots (exv files) this weekend.

postscript-dev commented 2 years ago

I was planning on working on the tags last week, but I had problems with my computer which took several days to fix.

@cris46: The existing tags that ExifTool defined have now been merged into Exiv2. When we release the next version of the library, those tags will be available for darktable to use.

You chose to use the minus variant, which is perfectly fine. The devices used the plus or minus variants depending on the last change.

I used the output from NX Studio. If you provide a sample using a positive variant, then I can check if a different vanilla value is stored. If the two settings are equivalent though, the same vanilla value may be used and it is up to the software to decide whether to display the positive or negative version.

The values in the TTL table also look correct, but I would assume that the high nibble is just the sign. If these values come from the sample files, that sign is inverted: 0xf0 would mean positive and 0x00 negative (are you sure?)

Using the samples you provided, NX Studio outputs: Filename Vanilla Exif.NikonFl7.FlashGroupAData NX Studio output
version1/_DSC6278.exv 242 "+0.3EV"
version1/_DSC6277.exv 248 "+1.3EV"
version1/_DSC6276.exv 254 "+2.3EV"

The signed nibble looks correct. I noticed that the n.6 translated entries are better rounded to n.7, so I will update the iTTL-BL mode table accordingly.

postscript-dev commented 2 years ago

@cris46: I forgot to mention that as the existing NikonFl7 tags have been merged, they are now available in Exiv2's nightly builds. The builds are not for a production environment as they could still contain bugs, but will allow you to try them out on files if you want to.

You chose to use the minus variant, which is perfectly fine. The devices used the plus or minus variants depending on the last change.

I think that if the negative and positive values are equivalent then it is likely that they will have the same vanilla value, but I am happy to test this. BTW, If you think that the positive value make more sense to the average photographer, then I will make the change. I chose the negative version as this was used by NX Studio.

There is no way to dial in 1/6 or 5/6 between two stops however.

The version2/_DSC6288.exv file used a manual value of "1/1 (-5/6EV)" in the Exif.NikonFl7.FlashGroupCData tag (see example below). I went back and added them all in the Manual mode table.

I have looked again at your posts and think that I understand a little more. I went back and have updated the manual and iTTL-BL tables to include the n.7 change used by NX Studio and the intermediate values that you suggested. Also the B/C data added a couple of extra values in the iTTL-BL table. The "+2.7EV" and "+3.0EV" are output in NX Studio, so I expect them to be correct, but they look out of place - Nikon ran out of space when allocating values. Let me know what you think and if there are still values missing.

Manual mode: Vanilla Value Translated Value
0 "1/1"
1 "1/1 (-1/6EV)"
2 "1/1 (-1/3EV)"
3 "1/1 (-1/2EV)"
4 "1/1 (-2/3EV)"
5 "1/1 (-5/6EV)"
6 "1/2"
7 "1/2 (-1/6EV)"
8 "1/2 (-1/3EV)"
9 "1/2 (-1/2EV)"
10 "1/2 (-2/3EV)"
11 "1/2 (-5/6EV)"
12 "1/4"
13 "1/4 (-1/6EV)"
14 "1/4 (-1/3EV)"
15 "1/4 (-1/2EV)"
16 "1/4 (-2/3EV)"
17 "1/4 (-5/6EV)"
18 "1/8"
19 "1/8 (-1/6V)"
20 "1/8 (-1/3EV)"
21 "1/8 (-1/2EV)"
22 "1/8 (-2/3EV)"
23 "1/8 (-5/6V)"
24 "1/16"
25 "1/16 (-1/6EV)"
26 "1/16 (-1/3EV)"
27 "1/16 (-1/2EV)"
28 "1/16 (-2/3EV)"
29 "1/16 (-5/6EV)"
30 "1/32"
31 "1/32 (-1/6EV)"
32 "1/32 (-1/3EV)"
33 "1/32 (-1/2EV)"
34 "1/32 (-2/3EV)"
35 "1/32 (-5/6EV)"
36 "1/64"
37 "1/64 (-1/6EV)"
38 "1/64 (-1/3EV)"
39 "1/64 (-1/2EV)"
40 "1/64 (-2/3EV)"
41 "1/64 (-5/6EV)"
42 "1/128"
43 "1/128 (-1/6EV)"
44 "1/128 (-1/3EV)"
45 "1/128 (-1/2EV)"
46 "1/128 (-2/3EV)"
47 "1/128 (-5/6EV)"
iTTL-BL: Vanilla Value Translated Value
0 "0EV"
2 "-0.3EV"
3 "-0.5EV"
4 "-0.7EV"
6 "-1.0EV"
8 "-1.3EV"
9 "-1.5EV"
10 "-1.7EV"
12 "-2.0EV"
14 "-2.3EV"
16 "-2.7EV"
18 "-3.0EV"
238 "+3.0EV"
240 "+2.7EV"
242 "+0.3EV"
242 "+0.3EV"
243 "+0.5EV"
244 "+0.7EV"
246 "+1.0EV"
248 "+1.3EV"
249 "+1.5EV"
250 "+1.7EV"
252 "+2.0EV"
254 "+2.3EV"
255 "+2.5EV"

The new iTTL-BL values revise the nibble theory.

I have also added this output to the B and C groups. Using the sample files, the FlashGroup[ABC]Data tags with the vanilla and translated values are:

$ exiv2 -Pkyvt --key Exif.NikonFl7.FlashGroupAData --key Exif.NikonFl7.FlashGroupBData --key Exif.NikonFl7.FlashGroupCData version[12]/*.exvversion1/_DSC6268.exv  Exif.NikonFl7.FlashGroupAData                Byte       42  1/128
version1/_DSC6268.exv  Exif.NikonFl7.FlashGroupBData                Byte       40  1/64 (-2/3EV)
version1/_DSC6268.exv  Exif.NikonFl7.FlashGroupCData                Byte       38  1/64 (-1/3EV)
version1/_DSC6269.exv  Exif.NikonFl7.FlashGroupAData                Byte       36  1/64
version1/_DSC6269.exv  Exif.NikonFl7.FlashGroupBData                Byte       34  1/32 (-2/3EV)
version1/_DSC6269.exv  Exif.NikonFl7.FlashGroupCData                Byte       32  1/32 (-1/3EV)
version1/_DSC6270.exv  Exif.NikonFl7.FlashGroupAData                Byte       30  1/32
version1/_DSC6270.exv  Exif.NikonFl7.FlashGroupBData                Byte       28  1/16 (-2/3EV)
version1/_DSC6270.exv  Exif.NikonFl7.FlashGroupCData                Byte       26  1/16 (-1/3EV)
version1/_DSC6271.exv  Exif.NikonFl7.FlashGroupAData                Byte       24  1/16
version1/_DSC6271.exv  Exif.NikonFl7.FlashGroupBData                Byte       22  1/8 (-2/3EV)
version1/_DSC6271.exv  Exif.NikonFl7.FlashGroupCData                Byte       20  1/8 (-1/3EV)
version1/_DSC6272.exv  Exif.NikonFl7.FlashGroupAData                Byte       18  1/8
version1/_DSC6272.exv  Exif.NikonFl7.FlashGroupBData                Byte       16  1/4 (-2/3EV)
version1/_DSC6272.exv  Exif.NikonFl7.FlashGroupCData                Byte       14  1/4 (-1/3EV)
version1/_DSC6273.exv  Exif.NikonFl7.FlashGroupAData                Byte       12  1/4
version1/_DSC6273.exv  Exif.NikonFl7.FlashGroupBData                Byte       10  1/2 (-2/3EV)
version1/_DSC6273.exv  Exif.NikonFl7.FlashGroupCData                Byte       8  1/2 (-1/3EV)
version1/_DSC6274.exv  Exif.NikonFl7.FlashGroupAData                Byte       6  1/2
version1/_DSC6274.exv  Exif.NikonFl7.FlashGroupBData                Byte       4  1/1 (-2/3EV)
version1/_DSC6274.exv  Exif.NikonFl7.FlashGroupCData                Byte       2  1/1 (-1/3EV)
version1/_DSC6275.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  1/1
version1/_DSC6275.exv  Exif.NikonFl7.FlashGroupBData                Byte       0  n/a
version1/_DSC6275.exv  Exif.NikonFl7.FlashGroupCData                Byte       0  0EV
version1/_DSC6276.exv  Exif.NikonFl7.FlashGroupAData                Byte       254  +2.3EV
version1/_DSC6276.exv  Exif.NikonFl7.FlashGroupBData                Byte       252  +2.0EV
version1/_DSC6276.exv  Exif.NikonFl7.FlashGroupCData                Byte       250  +1.7EV
version1/_DSC6277.exv  Exif.NikonFl7.FlashGroupAData                Byte       248  +1.3EV
version1/_DSC6277.exv  Exif.NikonFl7.FlashGroupBData                Byte       246  +1.0EV
version1/_DSC6277.exv  Exif.NikonFl7.FlashGroupCData                Byte       244  +0.7EV
version1/_DSC6278.exv  Exif.NikonFl7.FlashGroupAData                Byte       242  +0.3EV
version1/_DSC6278.exv  Exif.NikonFl7.FlashGroupBData                Byte       240  +2.7EV
version1/_DSC6278.exv  Exif.NikonFl7.FlashGroupCData                Byte       238  +3.0EV
version1/_DSC6279.exv  Exif.NikonFl7.FlashGroupAData                Byte       2  -0.3EV
version1/_DSC6279.exv  Exif.NikonFl7.FlashGroupBData                Byte       4  -0.7EV
version1/_DSC6279.exv  Exif.NikonFl7.FlashGroupCData                Byte       6  -1.0EV
version1/_DSC6280.exv  Exif.NikonFl7.FlashGroupAData                Byte       8  -1.3EV
version1/_DSC6280.exv  Exif.NikonFl7.FlashGroupBData                Byte       10  -1.7EV
version1/_DSC6280.exv  Exif.NikonFl7.FlashGroupCData                Byte       12  -2.0EV
version1/_DSC6281.exv  Exif.NikonFl7.FlashGroupAData                Byte       14  -2.3EV
version1/_DSC6281.exv  Exif.NikonFl7.FlashGroupBData                Byte       16  -2.7EV
version1/_DSC6281.exv  Exif.NikonFl7.FlashGroupCData                Byte       18  -3.0EV
version1/_DSC6282.exv  Exif.NikonFl7.FlashGroupAData                Byte       14  -2.3EV
version1/_DSC6282.exv  Exif.NikonFl7.FlashGroupBData                Byte       16  -2.7EV
version1/_DSC6282.exv  Exif.NikonFl7.FlashGroupCData                Byte       0  n/a
version1/_DSC6283.exv  Exif.NikonFl7.FlashGroupAData                Byte       14  -2.3EV
version1/_DSC6283.exv  Exif.NikonFl7.FlashGroupBData                Byte       16  -2.7EV
version1/_DSC6283.exv  Exif.NikonFl7.FlashGroupCData                Byte       0  n/a
version2/_DSC6284.exv  Exif.NikonFl7.FlashGroupAData                Byte       18  1/8
version2/_DSC6284.exv  Exif.NikonFl7.FlashGroupBData                Byte       6  1/2
version2/_DSC6284.exv  Exif.NikonFl7.FlashGroupCData                Byte       18  -3.0EV
version2/_DSC6285.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  0EV
version2/_DSC6285.exv  Exif.NikonFl7.FlashGroupBData                Byte       0  n/a
version2/_DSC6285.exv  Exif.NikonFl7.FlashGroupCData                Byte       0  n/a
version2/_DSC6286.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  0EV
version2/_DSC6286.exv  Exif.NikonFl7.FlashGroupBData                Byte       0  n/a
version2/_DSC6286.exv  Exif.NikonFl7.FlashGroupCData                Byte       0  n/a
version2/_DSC6287.exv  Exif.NikonFl7.FlashGroupAData                Byte       0  0EV
version2/_DSC6287.exv  Exif.NikonFl7.FlashGroupBData                Byte       0  0EV
version2/_DSC6287.exv  Exif.NikonFl7.FlashGroupCData                Byte       0  n/a
version2/_DSC6288.exv  Exif.NikonFl7.FlashGroupAData                Byte       15  1/4 (-1/2EV)
version2/_DSC6288.exv  Exif.NikonFl7.FlashGroupBData                Byte       30  1/32
version2/_DSC6288.exv  Exif.NikonFl7.FlashGroupCData                Byte       5  1/1 (-5/6EV)

I think that these are now the same as NX Studio. If you are happy with this then I will move on to other tags.

cris46 commented 2 years ago

First, sorry for the delay; I also still owe you a few tests I planned for the past weekend, but wasn't able to make yet.

Also, I've downloaded the executable following your link, and picking exiv2-1.0.0.9-Linux64.tar.gz. The I executed "./exiv2 -pa file.nef | grep NikonFl", but I got group data in percents. Probably I'm using it wrong. Can you post the correct command line? Or did I pick the wrong file?

Regarding positive and negative values: It really doesn't matter: We don't dial in numbers, we change them in steps. When increasing the value, my trigger uses positive, when decreasing, negative values. So I'm used to see both likewise.

Image 6288 was using a different flash trigger (I mentioned that this is an additional device mounted on the camera, that can be from Nikon--very expensive and not better--or third party manufacturers). This one was from Godox. I've less experience with it. But being compatible with Nikon, and as the Nikon camera actually generates the Exif data, we can expect some common ground. This Godox device allows for finer subdivisions (in 1/10ths) and now I know that it's translated to the 1/6th subdivision in the Exif file. And this should explain that you saw 5/6th in manual mode.

Using NX Studio was a brilliant idea; the notation used is perfectly representative for common usage.

I think, you understood the thing very fast, and so far as it's possible without putting this into action yourself with a camera and a couple of flashes. It took me much longer when I started.

To see your code in action, I still need your command line. But the tables you copied here look perfect. Just don't count on the limits of my devices (1/128 for power reduction, and ±3EV for TTL compensation). Just extrapolate the decoding as far as the values can be encoded by the bits reserved for each value.

But I didn't understand, why you think that the values "+2.7EV" and "+3.0EV" look out of space.

postscript-dev commented 2 years ago

Just don't count on the limits of my devices (1/128 for power reduction, and ±3EV for TTL compensation). Just extrapolate the decoding as far as the values can be encoded by the bits reserved for each value.

I wont write the whole table out again, but here is a summary of the changes. I have added the n.2 (e.g. 1/6 = 0.166) and n.8 (e.g. 5/6 = .833) values to the iTTL-BL table. I can't find examples of these but this fills in the table and completes the increment-by-sixth rule that Nikon use. I can't find any examples of values beyond the ranges that you use, so I would prefer to leave this for now. If you come across other values, let me know.

Using NX Studio was a brilliant idea; the notation used is perfectly representative for common usage.

It was your idea to use NX View. Unfortunately NX Studio does not display interpretations of all of the NikonFl7 metadata so some guesswork will still be required.

But I didn't understand, why you think that the values "+2.7EV" and "+3.0EV" look out of space.

This is because the other vanilla values were sequential and "+2.7EV" and "+3.0EV" aren't. This isn't important, only an observation.

Also, I've downloaded the executable following your link, and picking exiv2-1.0.0.9-Linux64.tar.gz. The I executed "./exiv2 -pa file.nef | grep NikonFl", but I got group data in percent. Probably I'm using it wrong. Can you post the correct command line? Or did I pick the wrong file?

No, this is working as I expected, as the only code merged is the existing ExifTool tags. Our development process is:

  1. Make changes to the library on the local computer.
  2. Submit changes online (called a Push Request or PR) and request a review from another user.
  3. Make any requested changes until the review is approved.
  4. The code is then merged into the library (usually by a third person) and will be included in a future release.

The changes that we are discussing now are only on my computer, but when we have finished discovered the new tags, I will create a PR. I usually develop using MSYS2/MinGW64 on Windows, but I can make a Linux build. As this is an extra step, I don't want to do this for every change but will if we need to test something or hit a milestone (e.g. complete a tag).

cris46 commented 2 years ago

You can't find an example in the files, because the n/6th values are only generated in automatic mode, and I use virtually always manual. For this reason, I wouldn't find an example even scanning all my collection of images.

But as far as I can see, the table looks perfect to me. I guess the zoom values are in a different tag.

postscript-dev commented 2 years ago

@cris46:I have some questions about the zoom settings. Why does each flash group have it's own zoom value? If there is only one lens, which value or combination of values is used?

cris46 commented 2 years ago

Actually, it would even make sense to record a zoom setting for each single flash even within each group, but that is obviously impossible.

Like the zoom of a lens changing the angle of view with the focal length, the flash zoom changes the opening angle of the light beam. This is done by moving the flash tube more or less into the focal point of the parabolic reflector, in contrast to many light modifiers that simply restrict (block) light to a narrower angle. This is specially important for the less powerful, battery driven compact flashes, as blocking part of the light means to loose energy, specially at longer distances.

Studio lights normally don't have the means to focus the light beam under the control of the remote trigger, but the remote trigger still requires some setting to be dialed in. After all, the remote trigger usually doesn't know what kind of flashes each group is composed of (unless everything comes from the same manufacturer, but as far as I know, Nikon doesn't sell studio lights). Thus not all settings are really meaningful, but some can help explain stray light which can be a problem specially when there are many reflective surfaces around the scene.

postscript-dev commented 2 years ago

@cris46:

Actually, it would even make sense to record a zoom setting for each single flash even within each group, but that is obviously impossible.

So far, I haven't seen the individual flash zoom values in NikonFl7. It is possible that the NikonFl groups only record the values set on the master flash unit.

ExifTool also listed modes such as iTTL, Auto Aperture, Automatic, GN (distance priority) and Repeating Flash, but I don't have examples for those, so can only output percentages. It this is something that you want me to add, then I will need more example files (.exv). For a mode, I only require an example of the highest, lowest and, if available, zero'th value - also let me know what values and increments are used for that mode. I think that I can work out the rest if I have that information.

If you have time, could you also include examples when the master flash unit is used and the A/B/C groups are switched off (this will help narrow down the master flash values). I would like a couple of examples using Manual mode and a couple using iTTL-BL. Let me know what the other Speedlight values that are set to, as this will help me fill in more tags in the group. I think that it is likely that Exif.NikonFl7.FlashCompensation uses the manual and iTTL-BL tables that we have been working on. If the values are the same then I can update the tag.

cris46 commented 2 years ago

It can't be expected that Exif records the settings of every single flash. There is no upper bound to the number of flashes. There are artists (Joe McNally comes to mind) using dozens of flashes, and somewhere I read, that it's possible to trigger hundreds of them. And as soon as studio lights (which are not produced by Nikon) are among them, compatibility would become a major issue. Thus, Exif only records the settings of the central controling device.

I've never used the master/slave game which is proprietary to Nikon and works only with Nikon flashes (excluding all studio flashes). Normally it was thought to be used with a special device emitting IR signals (e.g. SU-4), but that was very expensive and of limited use. I don't have this, but I think my SB910 should be able to assume that role too. I'll have to experiment.

But I'm not sure, if this is what you mean. The role of Nikon's master consists in communicating the settings to the slave flashes. But if there are no groups, there is nothing to communicate. But maybe it's possible to configure that anyway. I'll have to see how to make that work. Once it's working it should be easier to choose about the variations of settings.

BTW, "BL" in "TTL-BL" stands for "balanced fill-flash", meaning that there is a significant amount of ambient light and the flash is used only to brighten some parts in the shadows. The setting is only available if Nikon thinks the ambient conditions are adequate. I'll try to set up such conditions to trigger that mode.

Flash compensation, like EV compensation, is a common thing in photography, when it's the camera computing the settings and the photographer still want's to have a word in it. So you are most likely to be right assuming that this works the same everywhere.

Tomorrow I'll try to play this master/slave game and see what I can come up with.

postscript-dev commented 2 years ago

While reading the Speedlight instruction manual, I found:

Flash output level compensation: -3.0 to +3.0 EV in increments of 1/3 steps in the i-TTL auto flash, Auto Aperture flash, Non-TTL auto flash and Distance-priority manual flash

The sample files I requested (Auto Aperture, Non-TTL auto flash and GN (distance priority) would have been used to discover this, so I don't need them. Elsewhere in the manual, Repeating Flash is explained to use the manual mode compensations which we also already understand. I will update the tags.

There is no upper bound to the number of flashes. There are artists (Joe McNally comes to mind) using dozens of flashes, and somewhere I read, that it's possible to trigger hundreds of them.

I didn't know that many flashes could be used - the Speedlight documentation recommends 3 per A/B/C group. There is not room in NikonFl7 to record all that data.

I've never used the master/slave game which is proprietary to Nikon and works only with Nikon flashes (excluding all studio flashes).

I looked at the documentation and may have used the wrong terminology. For the samples, I was hoping for the Speedlight flash to be fired and the A/B/C groups to be switched off. If you could include 3 Repeating Flash examples, with values of 1Hz, 10Hz and 100Hz this would be helpful.

BTW, "BL" in "TTL-BL" stands for "balanced fill-flash", meaning that there is a significant amount of ambient light and the flash is used only to brighten some parts in the shadows. The setting is only available if Nikon thinks the ambient conditions are adequate. I'll try to set up such conditions to trigger that mode.

This is interesting. I identifed the flash mode as iTTL-BL using the definitions from ExifTool. They have values for:

Vanilla value Translated value
0 Off
1 iTTL-BL
2 iTTL
3 Auto Aperture
4 Automatic
5 GN (distance priority)
6 Manual
7 Repeating Flash

NX Studio identifies the sample files as TTL (the Speedlight documentation seems to call this i-TTL but the diagrams of the camera use TTL). It looks as though iTTL is an automatic mode using TTL and Automatic is the non-TTL automatic mode (the other modes have different levels of automation). If I can have 2 sample files the same, one where BL is switched on and another when off, I will look at the difference.

cris46 commented 2 years ago

I never knew exactly the difference between CLS (creative lighting system) and AWL (advanced wireless lighting). Today I got a bit smarter for reading the manual: AWL is the part of CLS referring to Nikon's way of remote triggering. This is an optical way, but not just the traditional optical slave which fires when it sees another flash; there is a lot of other information being exchanged. So, for me, today was an AWL-day. All files attached use this way of remote triggering.

According to the manual (table on p. D-4), The flash modes you requested are only possible with a SU-4 type device (which also might be an SU-800). But I don't have such a thing. I thought the 910, being able to act as a master (the controlling device) would replace the need of that, but this isn't the case. So I'll be unable to provide sample files for that. I'm not even sure if it's possible to buy such a device, as since the advent of the SB-5000, Nikon switched to radio transmissions, leaving all other devices discontinued. However the R1C1 kit (macro flashes) still seem to be available (waiting 3 months and about 700USD), which still uses the optical triggering only.

BTW, there seem to be also a way to use a menu in the D850 to make the settings of the master, but I haven't tried to do so. I think I read somewhere that these are completely equivalent options.

Regarding the number of flashes. Yes, today I found that place where Nikon considers 3 flashes per group as a maximum. But I do recall having seen (when the SB-910 was brand new) a video with Joe McNally lighting a boat with a man in the cabin using 20 or 30 flashes. I think it took him a whole day to set that up. But that's a long time ago, so don't shoot me, if I'm wrong.

There is a difference between TTL and i-TTL (the Nikon analogue to e-TTL from Canon), which is just a more complete exchange of information between the flash devices and the camera. And one thing required to make that work is to have a computerized lens. This is the case with most modern lenses, but there are some exceptions, and older lenses often are not. TTL mode is used automatically if i-TTL isn't available.

Now, regarding the sample files: awl.tar.gz

For these I used 4 SB-910 flashes: one, the master, mounted on the camera (D850), and three in a distances, each one assigned to a different group (A, B and C). Following the manual, I set these three to "remote", adjusted the channel and assigned the individual group. Additionally I adjusted the zoom position to 200mm. It was then only the master which was being changed.

File 6292 has set M, A, B and C to TTL, the exposure compensation set to ±0EV. The image before got direct light from two flashes in the image, so it is quite possible that this wasn't just "i-TTL" but "i-TTL-BL" (balanced fill).

In 6293 I varied the exposure compensation: M ±0, and A, B, C -1.0EV. In 6294 I additionally changed groups A and B to -2.0EV.

File 6295 should be the image you requested: Master on TTL, and all three groups off. Actually, this isn't such a useless setting, as often we turn off flashes in order to better see and control the effect of a single one. However, unexpectedly, the flashes fired even turrned off, and I believe with more energy as before. I don't have an explanation for that.

The settings available as described in the manual are context dependent: If some mode doesn't make sense to the camera, it's not available for selection: not grayed out but just missing. This makes it more difficult to see what might be available if I change the conditions. In any case, I could set each group (including the master) to TTL, auto-aperture, manual and off. File 6296 has the Master in TTL and groups A, B, C in auto-aperture. Who knows, if I used a different type of lens, I might have got different options.

File 6307 uses manual mode. The power settings were: M=1/6, A=1/8, B=1/128, and C=1/128. At least the zoom settings for the master did change. I've set it to 200mm initially, but at least for 6307 I saw that it was at 17mm. I'm unsure if I changed it accidentally, or if that comes from the camera.

Files 6308 to 6312 have the settings for "repeating flash". This is essentially a stroboscope. The possible settings are the frequency (flashes fired per second) and the number of flashes. I used a number of 4 in all these files. The frequency was set to 100Hz in most of them but at least in one I've reduced that. There I also played with the shutter time, as it took one second to see all 4 firings. BTW, using the stroboscope, all groups fired, but I had no choice regarding the power settings. This is probably due to the fact, that the scroboscope's frequency reduces the maximum power of each single flash, as there is no time to charge the capacitor between firing and fireing.

For file 6313, I tried to use those other modes, I now know are only available with an SU-4 type device. But I did set the group A flash on the master to manual. That flash was changed from the "remote" position simply to "on". Then I could pick the option "A" (auto, not auto aperture). I didn't expect it to fire, as for that the "remote" position should be required. However, it did fire. I don't have an explanation for this one either. I couldn't find the GN priority mode there. But it should have been. Probably, I simply didn't find it in some menu.

As I have never used AWL this way before, this was mainly trying to make it work at all. If you need some variations of a particular mode, just tell me. I think, now I know how to do it.

postscript-dev commented 2 years ago

Thanks for providing the sample files. As previously, I have inserted the .exv files into JPEGS which are available in awl-jpegs1.zip.

I noticed that your new awl samples are identified as using a slightly different Speedlight model (SB-910 instead of SB-900). This still works with the NikonFl7 group and is probably similar to the other model. The SB-910 instruction manual can be found here.

I have looked at the files and have identified that the master flash mode is actually saved in the lower nibble of Exif.NikonFl7.ExternalFlashData2. This changes the use of the tag, but all the examples consistently work so I have updated my code to display this. At the moment, I don't know what the upper nibble does.

Files 6308 to 6312 have the settings for "repeating flash". This is essentially a stroboscope.

The Repeating Flash examples confirm that the vanilla value for the mode is correct and that Exif.NikonFl7.RepeatingFlashRate and Exif.NikonFl7.RepeatingFlashCount worked correctly.

File 6296 has the Master in TTL and groups A, B, C in auto-aperture.

The Auto aperture vanilla mode value was correctly identified.

For file 6313, I tried to use those other modes, I now know are only available with an SU-4 type device. But I did set the group A flash on the master to manual. That flash was changed from the "remote" position simply to "on". Then I could pick the option "A" (auto, not auto aperture).

NX Studio only identifies this file as using Manual modes. Perhaps the files were mixed up? Interestingly, NX Studio displays some extra information when the master flash is fired. For example, for _DSC6313.exv:

Device:                       SB-910
Flash Mode:                   Front Curtain
Wireless flash options:       Optical AWL
Master:                       M x 1/16
Group A:                      M x 1/8
Group B:                      M x 1/128
Group C:                      M x 1/128
Group D:                      ---
Group E:                      ---
Group F:                      ---
Zoom Position:                Manual Zoom, FX 12mm
Illumination Pattern:         CW
Advanced Operations:          Wide-flash Adapter

This will help uncover the rest of the group. If I have an example where only the illumination pattern changes then I can work out where it is set. The setting requires the master, but the remotes can be on or off - whatever is convenient. The 3 illumination patterns are standard, even and center-weighted.

There is a difference between TTL and i-TTL (the Nikon analogue to e-TTL from Canon), which is just a more complete exchange of information between the flash devices and the camera. And one thing required to make that work is to have a computerized lens. This is the case with most modern lenses, but there are some exceptions, and older lenses often are not. TTL mode is used automatically if i-TTL isn't available.

If you have the hardware, a couple of examples where TTL is used (not i-TTL) would help uncover the mode table. Currently we are identifying the files as iTTL-BL but this needs adjusting. When I looked at _DSC6292, NX Studio didn't identify it as using BL (but it may not do this). If I had a couple of examples where BL was definitely enabled, this would help. As always, provide all these file only if you have spare time.

I have built the tags using Ubuntu 21.10, you can find them in exiv2-1.0.0.9-Linux64-new-NikonFl7-0.5.tar.gz. The new manpage for the exiv2 sample program is not released yet, but can be found here. If you find any problems then let me know.

cris46 commented 2 years ago

Yes. The flashes I used actually are SB-910, but the third party remote triggers seem to lead the camera to believe it's an SB-900, but I'm not sure if this is true for all of them. And by our understanding, the camera assumes this SB-900 to be in master mode.

Now that you mention the thing with the nibble for the master mode, I vaguely recall something with exiftool. Didn't Mr. Harvey assign the upper nibble some meaning? Does the value of the upper nibble vary? Maybe I can try to recall differences in the shots for the exv files and reproduce them. But if it's constant, it'll be more difficult, as it might be simply four bits reserved for future use. Do you suspect some meaning which I could try to confirm?

Regarding the thing with 6313, just recalling the context: The file 6307 was set with the master and the three groups to manual. For images 6308 to 6312, I changed to stroboscope. This is a change in the menu of the flash. For 6313 I restored the previous menu setting and such, I got the previous values back. These where identical to 6307. This explains why they are all manual. I hope, that you can confirm that Exif of 6307 and 6313 in fact do match. Other than that: The trigger basically doesn't care how many remote devices will receive the commands for a certain channel and group, it just sends them. Nor does the camera. Hence the identical foot print. The fact the remote device fired inspite of not being on "remote" is another story, which (basically) shouldn't concern neither the trigger nor the camera. However, the i- in i-TTL seems to extend also to other modes, so I can't know which messages are sent or received by the camera or the devices. The bottom line seems to be, that as far as the Exif is concerned, things are coherent.

Your output of 6313 seems correct. "front curtain" is default; it could be changed to "rear curtain" in the camera only, so this is something the camera obviously "knows" (its about the moment of firing the flashes. As flashes are supposed to have a shorter duration than the time the shutter is open, firing can happen right after opening or just before closing, which can have a different effect on the image). The mention that it's "optical AWL" indicates, that this is the older version (there are only three groups available). The newer version uses radio frequency.

The zoom position may be correct for the master flash; initially, all where set to 200mm. I believe that the camera did change that for the master, but unfortunately I didn't pay too much attention to that. The structure of our output suggests, that in Nikon AWL, there is only one zoom setting which is valid for all flashes, no matter which group. However, the third party transmitters allow to set this individually, by group.

Nikon distinguishes FX and DX, which refers to the size of the camera's sensor (DX is APS-C, FX is full frame). Given a constant optical imaging of a lens, the larger the sensor, the larger the field of view, so this does change the meaning of the focal length (be it a lens or the flash zoom).

And again, I have to confess, that I've never really understood the "illumination patterns". I know so much, that it is related to the zoom setting, the use of the wide-flash adapter (a diffuser screen built into the flash which can be pulled out to cover the flash head), and maybe the angle of the flash head regarding the flash body. "CW" (center-weighted) would suggest a long zoom setting, which is true for the remote flashes, as I set them all to 200mm (maximum). As you know, there are many more options for the zoom setting (values in mm) than just the three "standard", "even", and "center-weighted", so I usually pay more attention to this view. BTW, Nikon changes the range of the possible values according to the lens mounted, but the third party triggers don't.

The advanced operations do have a few more possible settings I know of. Besides the wide-flash adapter, there is a green and orange cap which have a magnet telling the flash it's on, and there is a diffusion dome, sort of a cup put on the flash, and this is sent to the camera (but I don't know what the camera would do, if some have, some don't or if this information is restricted to the master flash, or if it's an SU-800).

I do have a manual lens which should not be able to activate i-TTL. I'll make a test shot with that. If it happens to be as expected, you can tell me the variations you'd like to get.

Regarding the i-TTL-BL (balanced fill light), I did expect that most of the files where not BL, but I was unsure on those having the two backlights within the frame, because they where small and contained. I guess I'll be able to make a BL shot, just waiting for enough ambient light. This will have to wait untill tomorrow as it's already dark outside.

I think we are doing pretty well, each one working at a rythm that is possible for him, so don't worry, just tell me the files you still need.

Later this evening, I'll play with the test version you uploaded.

cris46 commented 2 years ago

Here are three more files:

ittlbl.tar.gz

Shooting indoors in front of a window is a classic situation in which fill light is required. However, here the weather is very foggy these days. It's so dark that I have to turn on the lights even during the day. But the window is still much brighter than a face in front of it having the lights turned off, so I tried it.

6314 uses the master on camera to trigger one flash remotely, but itself being turned off. This should be a fill flash (i-TTL-BL). But neither of the two flashes showed the TTL-BL string on their monitors.

So I tried another one, 6315, using just one on-camera flash (i.e., without AWL), but also that didn't show the TTL-BL string. I do remember, when these flashes still were new, that I saw the TTL-BL string, so again, inspite of the situation requiring fill flash, I'm not sure, if that is actually a TTL-BL setup.

Finally, 6316, I tried to get a TTL-file (without i-) using a special lens. I did remember that this lens is manual only, but rechecking the manual, this refers to focussing only. There is no mention in the manual if this lens delivers enough information to the camera to allow for i-TTL. And I'm afraid it did not work: One information the camera needs for i-TTL is the focus distance, and that appears in the Exif data. Check it out anyway, but if it's i-TTL, I'm out of options, because I know that all my other lenses have a CPU providing all necessary infos to the camera.

I did download your precompiled version of exiv2, but the version of libc doesn't match, so it can't be executed on my computer.

Maybe you could have a look, if files 6314 and 6315 differ in some way from the others such that this could be the difference between i-TTL and i-TTL-BL. If not, we'll have to wait for better weather.

postscript-dev commented 2 years ago

If anyone wants to use the new files with NX Studio, JPEG versions can be found in ittlbl_JPEG-versions.zip.

Maybe you could have a look, if files 6314 and 6315 differ in some way from the others such that this could be the difference between i-TTL and i-TTL-BL. If not, we'll have to wait for better weather.

The files you provided have been helpful, if a little puzzling. Up until now, when Exiv2 listed a file as iTTL-BL then NX Studio output TTL. With the new files, we identify _DSC6315.jpg and _DSC6316.jpg as iTTL and NX Studio outputs iTTL (without mentioning BL). Is it possible that the previous files were somehow only ttl? For _DSC6315.jpg, NX Studio outputs something different, with the A/B/C missing:

Device:                 SB-910
Flash Mode:             Front Curtain
Flash Control Mode:     i-TTL, 0EV (Camera: 0EV, Speedlight: 0EV)
Zoom Position:          Manual Zoom, FX 12mm
Illumination Pattern:   CW
Advanced Operations:    Wide-flash Adapter

To experiment, I created a Python script to generate 256 JPEGs each with the Exif.NikonFl7.ExternalFlashData2 tag set to a different value. I then looked at each image in NX Studio and they use the following values:

Vanilla Value Mode
0 Off
1 TTL
2 iTTL
3 AA
4 A
5 GN
6 M
7 RPT

Apart from the values for 1 and 2, this confirms that we have correct values for the rest of the table.

I looked further at the files. In NX Studio, the _DSC6314.jpg file contains an "Advanced Operations" value of Bounced - which was new to me. I looked in the tags to try and work out where this was set and ended up re-evaluating the Exif.NikonFl7.ExternalFlashData1 tag. I reused my Python script to generate 256 images. I then looked at each image in NX Studio and made some discoveries. If the following bits are set, you can find out when certain features are switched on/off (the bits can be combined).

Bit to set Feature
0000 0100 Bounce Flash adaptor
0001 0000 Wide Flash adaptor
0010 0000 Nikon Diffusion Dome

ExifTool already used the bit 1000 0000 to described whether "External flash zoom override" was set. NX Studio prefers "Automatic zoom" if the bit is 0 and "Manual zoom" if 1. The Python trick will only work for values that I can change in NX Studio.

I applied the same Python trick to Exif.NikonFl7.0x0010 and found that NX Studio outputs an additional field - the Colored filter Bit to set Feature
0000 0001 FL-G1 (for fluorescent light)
0000 0010 FL-G2 (for fluorescent light)
0000 1001 TN-A1 (for incandescent light)
0000 1010 TN-A2 (for incandescent light)
0100 0001 RED
0100 0010 BLUE
0100 0011 YELLOW
0100 0100 AMBER
0100 1111 OTHER

While looking for the Bounced value, I found one of the other values NX Studio values. The number in "FX 12mm" is found in the 20th byte of Exif.NikonFl7.0x0013 (vanilla value==translated value). I don't understand what the value is though.

Note: I would treat the NikonFl7 values as read-only as we still don't understand all of the tags. The values could affect/depend on other data in the group and changing individual values could cause problems with the metadata.

I did download your precompiled version of exiv2, but the version of libc doesn't match, so it can't be executed on my computer.

I don't build very much on Linux, so will have to look into this.

I will try more Python experiments another time. I need to work on other things, so will come back to this at the weekend.

clanmills commented 2 years ago

@postscript-dev and @cris46 You've both put in an incredible effort on this matter. I'm sure this is the most complicated metadata record in Exiv2.

cris46 commented 2 years ago

Nikon never prints "i-TTL" on the camera or flash screens, probably there isn't much space. As TTL is only used when i-TTL is not available, this is a factor depending only on the lens. Who ever has such a lens (typically an older one), and who actually uses TTL, certainly will know that attribute of the lens. So not distinguishing between TTL and "i-TTL" isn't critical.

The situation is a bit different with TTL and TTL-BL, because for BL a much lower level of power is required. There is no need to illuminate the whole scene, it's just to brighten some shadow. So a priori, this would be an interesting fact. However, as there is no button where I could adjust for BL, there is little the photographer can do if the camera chooses the wrong mode. He can only look at the camera's screen and decide to apply some flash compensation (EV or, "FV"). The bottom line is, that while this bit is more interesting than the TTL/i-TTL thing, it still isn't really critical.

Nikon isn't always clear defining what it is meaning. But the general understanding is, that a lens with a CPU provides enough information such that i-TTL can be used. So I tried to get some defining sample files.

6316 was the only image I shot with that manual lens (which is identified correctly by lensfun). All others are modern lenses having a CPU. So I can't see why this should be TTL only. Maybe someone else, actually using TTL, could share some thought about the priority decoding this properly.

Regarding 6315: Yes, this was non-AWL, just one flash mounted on the camera, no remote flashes. As the displays on the camera and the flash are pretty crowded in AWL mode, I wanted to try to see, if the TTL-BL string would appear on the flash's normal (non-AWL) screen. But it didn't. But for this, there can't be any group information.

Your table lists all modes that are theoretically possible, but they are not always available at the all times. The codes look pretty good.

"bounced" refers to yet another thing this flash has built in. If you have the manual, you can see a drawing of it, item 6 on page B-1. Normally, the card as well as the diffusor are slided in. When pulled out, the card stays like drawn in the manual such that all light having an upward-direction is bounced back down. This was the case for image 6314. Sorry, I forgot to mention it.

While the bounce card and the wide diffusor can be used at the same time, the bounce card can not be used when the diffusion dome is put on. But that does make sense to using individual bits rather than consecutive numbers.

BTW., the table with the filters is consequence of item 9, also on B-1 in the manual.

The meaning of "FX 12mm":

Imagine holding a round, naked lens in your hand and projecting some image upon a piece of paper. For the lens being round, the projection will be a circle. The camera's sensor is rectangular and needs to fit into that circle. If there is a margin between that rectangle and the circle, the effective field (angle) of view is smaller than that given by this lens and the distance between lens and paper. As this distance is fixed within the camera, the focal length of the lens, as well as the size of the sensor determines the angle and field of view. If the camera wants to make sure there is enough light to illuminate the scene properly, it just needs to make sure to illuminate the same field of view. So the 12mm in the above expression means, that the flash is adjusted to illuminate an area in the scene that would be visible through a lens with a focal length of 12mm. As the size of the sensor influences the effective field of view, it must be given as well. Of course, if there is more than one flash, only one can be on-camera; the rest will be remote and could be positioned at different distances. With TTL the camera has little change to balance this. With i-TTL (presumably) the camera is able to chat with each flash and negotiate a power setting. But I doubt that Nikon can do this comprehensively.

Could you send me a command line which will expose the remainging unknown bits? I could run that over my whole collection. Perhaps I can find cases where the value changes. Else we would have to wait until that happens by chance.

Read-only is perfectly fine, as I can't imagine a situation in which changing it would make any sense.

postscript-dev commented 2 years ago

Nikon isn't always clear defining what it is meaning. But the general understanding is, that a lens with a CPU provides enough information such that i-TTL can be used. So I tried to get some defining sample files.

This was my understanding too. I will do some research and see if something turns up.

Regarding 6315: Yes, this was non-AWL, just one flash mounted on the camera, no remote flashes

I found that both _DSC6315.exv and _DSC6316 have AWL off. This is a big advantage as up until now, I didn't know how NX Studio detected the status of AWL. This looks as though it is controlled by the 1000 0000 bit in Exif.NikonFl7.ExternalFlashData2 which, on all the other examples, was always set. I think that ExifTool's "external flash attached" in Exif.NikonFl7.ExternalFlashData1 was intended to detect the AWL but it doesn't seem to work. I removed the ExifTool output and added my own, but this will need to be tested with other files where AWL is switched off.

I have renamed ExifTool's "External flash zoom override" as "Manual zoom"/"Automatic zoom" as this is the same as NX Studio and I think is clearer.

Although you don't use it, I have also completed the table for Exif.NikonFl7.FlashGNDistance by adding the missing values in meters and all the feet/inches ones. The flash offers the option to choose units.

I have added the color filters to the code by renaming Exif.NikonFl7.0x0010 to Exif.NikonFl7.FlashColorFilter. This also is untested with actual files.

I also found that the manual compensation values extend to 1/256. They may go further, but I have a feeling that this is connected to the high-speed sync setting (in page E–24) and I can't trigger that.

I discovered that the Exif.NikonFl7.ExternalFlashFirmware tag has a value of 0x0801 for the "SB-N5" Speedlight flash and added this in.

Could you send me a command line which will expose the remainging unknown bits? I could run that over my whole collection. Perhaps I can find cases where the value changes. Else we would have to wait until that happens by chance.

I have created a build using Ubuntu 20.04 which I hope that you can use. You can find the files in exiv2-1.0.0.9-Linux64-new-NikonFl7-0.6.tar.gz. Try the files out and let me know if you have problems.

cris46 commented 2 years ago

While AWL is the correct term, I think a better term would be "commander" or "command mode". If that's too long, "cmd mode" should be acceptable as well. This term also appears in the menu of the camera, when making the adjustment from there (which I never do). The reason for this is, that even if the camera (which generates the Exif) thinks it's AWL (which is remote control by optical means), most of the time these are third party radio controlled devices, the things we hold in our hands and thus are more real than the technology used.

I've seen more often talking about TTL mode (no manual adjustment of the zoom) vs manual mode (manual adjustment of the zoom). But considering, "automatic zoom" is perfectly correct, as in TTL mode the zoom isn't left alone nor set by the user, but by automatically the camera.

colors-exv.tar.gz

I've uploaded a few further files. As you have many in AWL ("commander mode"), I used a single on camera flash without AWL:

6327: plain TTL 6328: wide diffusor screen 6329: bounce card 6330: orange filter 6331: green filter 6332: diffusion dome 6333: FP (HSS)

FP is short for "focal plane" and Nikon's jargon for HSS (high speed sync), which should be the generic term.

I didn't even know that there is an SB-N5 flash, and certainly I don't have one. It's the first time this name comes up. Might that be the ID Nikon is lead to believe is connected when I used some third-party controller?

The binary you prepared does work OK. Now I still would need an exiv2 command line which displays the bits you're interested in and which I can use with find(1) to extract that form all files for filtering. Unless you want them all (there are thousands of them).

postscript-dev commented 2 years ago

I have made the .exv files in colors-exv.tar.gz available as JPEGs for use in NX Studio. You can find them in colors-exv_JPEG-versions.zip.

While AWL is the correct term, I think a better term would be "commander" or "command mode". If that's too long, "cmd mode" should be acceptable as well. This term also appears in the menu of the camera, when making the adjustment from there (which I never do). The reason for this is, that even if the camera (which generates the Exif) thinks it's AWL (which is remote control by optical means), most of the time these are third party radio controlled devices, the things we hold in our hands and thus are more real than the technology used.

Exiv2 already has a tag called Exif.Nikon3.FlashDevice (vanilla==translated). e.g., When AWL is switched on,

$ exiv2 -u -K Exif.Nikon3.FlashDevice ittlbl/_DSC6314.exv
Exif.Nikon3.FlashDevice                      Ascii      20  Optional,Comdr.

I will change the NikonFl7 output to "Comdr" as this is consistent with the other tag.

I've uploaded a few further files. As you have many in AWL ("commander mode"),

NX Studio identifies the flash as:

Filename Description
_DSC6328.exv Uses Wide-Flash Adaptor
_DSC6329.exv The same as _DSC6327.exv - not using Bounce Flash adaptor
_DSC6330.exv Uses TN-A1 (for incandescent light)
_DSC6331.exv Uses FL-G1 (for fluorescent light)
_DSC6332.exv The same as _DSC6327.exv - not using Nikon Diffusion Dome
_DSC6333.exv The same as _DSC6327.exv

Exiv2 outputs the same as NX Studio, but I expected the bounce flash adaptor and the diffusion dome to appear. Are they Nikon items? Although the adaptors were used, the camera may have considered that they weren't a significant enough factor in the photos to record the effect. Perhaps some other setting or situation determines whether the photo is classified as using that adaptor. It would help if you know what the difference between ittlbl/_DSC6314.exv and colors-exv/_DSC6329.exv is, as ittlbl/_DSC6314.exv does register as Bounce Flash.

In _DSC6333.exv, for the first time Exif.NikonFl7.0x0005 changed values. This mystery tag could be connected to the FP (HSS) and it would help if I had some samples to investigate. Please include TTL and Manual examples.

I didn't even know that there is an SB-N5 flash, and certainly I don't have one. It's the first time this name comes up. Might that be the ID Nikon is lead to believe is connected when I used some third-party controller?

This value is found in Exif.NikonFl7.ExternalFlashFirmware and doesn't affect your photos. I am trying to document the things that I have found, in case others find them useful in the future.

cris46 commented 2 years ago

Also I expected the bounce card and the dome to be recognized. Yes, they are accessories to the SB-910 and came in the same box. Of course, it's always possible that the camera does something unexpected, but with these things, they should be recognized by hardware not by some measurement. The funny thing is, that one of the previous images, you did mention to have seen the bounce card bit.

hss.tar.gz

Here are 6 more files, again w/o AWL. The first three are TTL flash, the other three manual (1/64). Both showed the FP on their screen, when I set the camera to a shutter time of 1/8000. If you need more examples, let me know.