Precise time mark using camera trigger and RAW GPS module

[Naish88]

Any Ublox with RAW data capability can be used for precise time measurements with sub-microsecond resolution using the external interrupt on the GPS module. This will be really good to timestamp each picture taken for mapping purposes. A connection between the trigger of the camera and the GPS has to be made ( TTL levels ). This function can be well integrated together with the new RAW data log capability function.

For reference: look on page 12 of the UBlox datasheet, (1.15.3) https://www.u-blox.com/images/downloads/Product_Docs/LEA-6_DataSheet_(GPS.G6-HW-09004).pdf

[rmackay9]

We actually already set the Pixhawk's time from the GPS. https://github.com/diydrones/ardupilot/blob/master/ArduCopter/commands.pde#L132 So, if it's ok, I think I'll close this issue.

So I think the time that we record the CAM message in the dataflash is already pretty accurate. What is probably a much large time-delay is the delay in the camera itself. It can often take a second for the camera to actually take a picture after it receives a command.

[Naish88]

Hi Randy, I had a chat this morning with Tridge and we discussed this idea, he is really keen go getting this implemented in the new logging code for RAW data. So he asked me to open the Issue. I'm aware that we already log the time on the CAM message, but that's only the time of when the Pixhawk sent the trigger pulse. Using the External interrupt on the GPS will give you the ability to log the exact triggering time. ( a lot of cameras have a trigger output for external flash light ). Maybe is not really critical in copters, but when you are doing mapping with a plane flying at 15m/s, few ms offset on the time can make a big difference.

[rmackay9]

aha, ok then!

[taedla01]

@Naish88 @rmackay9 @tridge As I mentioned through the Gitter private chat, I have precise time mark and precise coordinates stamped on CAM messages on my plane and multicopters. The simple mode is to solder two wires to Camera Flash Hot Shoe and then input them into Pixhawk FMU digital input pins. With this you can supply precise images positioning and then postprocess RAW msg to reach 2cm picture precission level. The image time mark have now a precission of 20 ms instead of current 200ms medium error.

I have the code for publish if you are interested but firstly is necesary to merge #2232 #2231 . It is very, very simple, but it is working :)

Cheers, Dario.

[bchristal]

Brilliant feature. I've been trying to figure out a solution for this exact problem. Not when the CAM message occurred, rather exactly when (and if) the picture was taken. Sorry to derail, but have you seen any tutorials for accessing Canon Powershot flashes since there's no Hot Shoe.

-Brian

[iskess]

+1 I've been needed this functionality for a long time. I need to know when the camera actually shutters. Not only does this nearly eliminate the latency in the logs, it resolves the issue of the logs having s different number of triggers than photos in the camera. Here is a thread I started a long time ago dealing with this question. Please implement this! Thank you Dario!

[taedla01]

Hello everyone,

Solved with #2309 (pending to be merged).

Cheers, Dario.

[nicksargeant]

Brilliant! Thank you! I too have wanted this feature for a long time! This not only allows for more accurate georeferencing, but a minor extension would allow the autopilot to warn the operator if the camera is being told to trigger but isn't capturing photos. For this to work, perhaps we should keep (but rename) the old cam message so we can see both when a trigger is requested and when the photo is actually taken?

[taedla01]

Hi @nicksargeant,

You can see the answer to your question (and that I have the code for it) in my last post here #2309

I've gotten to know if I'm shooting pictures or not through a change in my OSD. You can see here #1626

With camera in standby OSD shows this:

When camera shoots photos you can see this:

Greetings, Dario.

[Tietziano90]

One question, I m using the Arducopter version 3.3.3 how can I implement just this feature of precise time triggering?? I already got my trigger signal from the hotshoe logged with a raspberry pi but if the pixhawk could do that i would love to get the raspi out of the copter!

Thank you so much for your help!!

Cheers Michael

[onurboy]

Hi Folks, I've just bought a Sony RX100 m2 camera with hotshoe connection but I have no idea about how to get the signal from the camera. Can anyone help me please? Thanks Onur.

[Tietziano90]

Hey Onurboy,

basically what the hot shoe does, is connecting the two contacts to each other when triggering. It closes the circuit. I cant help you how to implement the signal recording into the ardupilot code because i use it with a raspberry zero ;)

So just connenct one wire to each contact and add a low voltage and try to log an eventpoint whenever the circuit gets closed!

Hope that helped a little!

cheers

[onurboy]

Tietziano90, Actually just for right now, I do not need to feed the signal or triggering information to the ardupilot. I just need to find out how to get an information from hot shoe that the camera actually triggers. Thank you. VR,

Onur.

[bchristal]

This is a feature in APM plane, not copter yet. See http://ardupilot.org/plane/docs/common-camera-shutter-with-servo.html#enhanced-camera-trigger-logging

[onurboy]

@bchristal thank you, I've already read that info in wiki, that's why I bought the sony rx100 m2 with hot shoe but I could not find any universal hot shoe adapter that I can put on the camera hot shoe area in order to get a feed back signal. And also there is no information about how to get that feedback signal. I mean which pins will go into the pixhaw etc?

[bchristal]

You need to make a circuit like this(I use a 10k resistor): Take power from the power pin on the Pixhawk rail. I'm currently making a slide in circuit board. Here's a prototype. I just need a spring loaded contact for the middle hot shoe pin.

Will be on tuffwing.com

[mboland]

Any ideas when we might have this feature for copter?

[onurboy]

@bchristal, thank you so much for the detailed interest and information. I think my biggest problem is hot shoe of my camera is different as shown below. I measured the center pin and ground with multimeter while taking pictures but I could not see any signal or voltage from the hot shoe. These recent Sony's have multi interface hot shoes and I think they are different from yours. I hope I can find a solution :( Thanks

Regards,

Onur.

[bchristal]

It will work. They are all the same since 1940. There is no signal at the hot shoe. There is a completed circuit between the base and the center when the shutter opens. When the circuit above is resting the center pin is +5. When the shutter opens, there is continuity between the center pin and base. The center pin is behind the resistor, so it goes to -. That change from High to Low is the signal the Pixhawk will detect.

[onurboy]

@bchristal , Wooow, that's a huge information :) Thank you I will try it ASAP.

Best Regards.

Onur.

[onurboy]

@taedla01, @bchristal,

I have implemented the circuit above to my camera and I think it works. I connected the multimeter and when I push the camera shutter button, voltage on the multimeter drops from 5.10V to 4.9V and goes back to 5.10V immediately. I also changed the parameters in mission planner, make CAM_FEEDBACK_PIN=50(AUX1), CAM_FEEDBACK_POL=0 (voltage drops down) and conected the circuit to the AUX1. I take 7 photos from mission planner and then press the camera shutter button after each trigger camera now command on mission planner screen (some at the same time some at a few seconds after). I added the CAM and TRIG message lines below text file with the regarding photos taken. How can I know which one is actual camera shooting time? And how can I geotag my photos with these exact info on mission planner tagging section? I think these questions will help everyone. Thanks, VR, Onur.

log.txt

DSC02065.JPG 20.32.50 (GMT+3) DSC02066.JPG 20.32.57 DSC02067.JPG 20.33.09 DSC02068.JPG 20.33.15 DSC02069.JPG 20.33.21 DSC02070.JPG 20.33.26 DSC02071.JPG 20.33.33

[bchristal]

Use the geotagging utility in Mission Planner, "Control F". The CAM Messages are the actual time. TRIG are picture commands sent from your Pixhawk. It is good that you have 7 CAM messages and 7 photos. The Geotag utility uses CAM messages. It ignores TRIG.

[onurboy]

@bchristal Thank you very much...

[cczeets]

I see that this feature is in plane, but not copter. Any idea when it will be added to copter? It is very important when using any of the newer L1 high-accuracy GPS. Thanks!

[bchristal]

The parameters are in Copter 3.4b. I have installed 3.4b, but not flown it or tested hot shoe feedback. You can install it by choosing beta firmware in Mission Planner.

-Brian

[cczeets]

Thanks Brian! I'll check it out.

[cczeets]

I loaded the Beta Firmware 3.4-rc1 and tested out the Enhanced Camera Triggering. Everything worked great, but when I used Mission Planner to geotag the images it claims that I have "0 CAM Messages". Does anyone know if Mission Planner was updated to read the new Enhanced Camera Triggering messages in the Log File? Or, is there something else I can look for? Thanks! Steve

[cczeets]

One other thing I noticed, that is unrelated to the camera triggering, the mission that I uploaded and flew didn't show up on the "Flight Data" screen. Although, it flew the mission fine, told me the correct distance to waypoints and took all of the pictures.

[bchristal]

I'll try and test Copter 3.4b today. The Misson planner geotagging tool does not need a change. Cam messages are still created when a signal is received from the hot shoe. Check your log for TRIG messages. Review this documentation: http://www.tuffwing.com/support/Install_a_sony_hot_shoe_precision_geotag_cable.html I'll get back in a few.

[cczeets]

Hi Brian! Thanks for the quick response! I am using a sync cable plugged into the hot shoe. I'm guessing it's not receiving the signal for some reason. The TRIG messages are there, but no CAM messages. If I can't get it working, I'll order one of the Tuffwing hot shoe plugs later today.

[bchristal]

Make sure CAM_FEEDBACK_POL is 0. Also, make sure all of your RELAY pins are set to -1. I just tried Copter 3.4 and had issues. Will try a different hot shoe adapter. Also, you need 5 volts on the + rail just like the Canon cable: http://www.tuffwing.com/support/Do_I_Need_a_BEC_for_PixHawk_to_Canon_Trigger_Cable_to_Work.html

[bchristal]

Confirmed - Precise feedback works in Copter 3.4. I was having trouble because I was testing by walking in my driveway with the copter disarmed. It will trigger the camera disarmed, but won't log CAM events. Look close the data below. Notice in some cases there are multiple TRIG events NOT followed by a CAM. I noticed this walking down my driveway. I had CAM_TRIG_DIST set to 3m. Sometimes I would walk that distance and notice the camera didn't trigger. So the system worked beautifully by only recording CAM events for actual pictures. I think the LED is too close to the camera, or maybe the Hobbyking IR triggers are better than Flyton. I don't know. The great thing is there's always something causing a missed picture, but now you're just recording CAM events for confirmed pictures. Good stuff.

[cczeets]

Thanks for testing Brian! Do you know if the Tuffwing Hot Shoe cable works on the Sony A7? I can't seem to get my Sync Cable working...

[bchristal]

Should work as long as your hot shoe looks like this:

[magicrub]

@bchristal wow, great compatibility list! @jmachuca77 do you know a good place on the wiki to put that?

[bchristal]

Hot Shoe Compatibility list picture sourced from this page: https://www.indiegogo.com/projects/flashq-the-little-cube-for-flash-photography#/ Not my work. They're all ISO 518.

[cczeets]

Brian, I received the Hot Shoe cable the other day. It works perfect with my Sony A7. It also works perfect in Mission Planner. Many Thanks!!

[Tietziano90]

Hi everyone,

one question, is it possible to just use the Pixhawk, a GPS and the camera trigger function to record GPS and precise foto timestamps? Since @bchristal bchristal said pixhawk doesnt record the cam messages when it is disarmed! Is it possible to disable the disarming? I disabled already all the arming checks, as well as the safety switch.

My goal is to trigger the camera via a raspberry and use the pixhawk to record all the data!

Thanks in advance!

Cheers Michael

[WickedShell]

Logging while disarmed works just fine, dining ensure you have it enabled. LOG_DISARMED 1 if on new enough firmware, otherwise it's part of the log bitmask.

On Aug 23, 2016 5:14 AM, "Tietziano90" notifications@github.com wrote:

Hi everyone,

one question, is it possible to just use the Pixhawk, a GPS and the camera trigger function to record GPS and precise foto timestamps? Since @bchristal https://github.com/bchristal bchristal said pixhawk doesnt record the cam messages when it is disarmed! Is it possible to disable the disarming? I disabled already all the arming checks, as well as the safety switch.

My goal is to trigger the camera via a raspberry and use the pixhawk to record all the data!

Thanks in advance!

Cheers Michael

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ArduPilot/ardupilot/issues/2289#issuecomment-241673275, or mute the thread https://github.com/notifications/unsubscribe-auth/AAipiD7z1cYMn5Ifd78AtVofme6ljESBks5qirnjgaJpZM4EZGeJ .

[Tietziano90]

Thanks WickedShell,

does it also log the cam messages even though i do not trigger the camera via the Pixhawk? So far I can't get it working....

[WickedShell]

You have to try and trigger one via pixhawk to enable the pin to see the feedback. This should be opened as an issue really that this pin should be configured on start up rather then the first time we trigger a camera, as both hearing and others have seen problems with that first trigger command logging 2 CAM events

On Aug 23, 2016 7:23 AM, "Tietziano90" notifications@github.com wrote:

Thanks WickedShell,

does it also log the cam messages even though i do not trigger the camera via the Pixhawk? So far I can't get it working....

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ArduPilot/ardupilot/issues/2289#issuecomment-241701488, or mute the thread https://github.com/notifications/unsubscribe-auth/AAipiD7AiWmSdt4yb4QaWZhjrLX3vRQ-ks5qitgYgaJpZM4EZGeJ .

[Tietziano90]

But just to make sure, I m right connecting the (+) output via an 10K resistor to the middle spot of the hotshoe, (-) to the outside of the hotshoe and (s) to the middle without any resistor right?

Thanks for your help!

[anestetico]

Hi Before trying the log feature I Just wanted to know if the Pixhawk AUX pins accepts 5V without damage because are normally used as signal pins. Thank you

[WickedShell]

This has been implanted and supported on copter release for awhile as well. I'm not seeing anything else from this PR that is still outstanding.

[gChuNguyen]

hi. Anyone can help me the geotagging problem. So now I just want to test the Geo Ref Images by using MP. But I need the log file and the respective pictures to test. I need more help. Thank you so much. If anyone who tested the Geotagging. Please give me the log file and pictures I just to test the procedure which generates a geo picture.

[Naterater]

Due to this being called/logged in the CAM loop at 50Hz, the tags still are not that accurate. They have a 20ms resolution (40cm at 20m/s). The follwing PR should solve it, and I will close this issue once it is fixed.

https://github.com/ArduPilot/ardupilot/pull/8397

[proficnc]

I’m still unaware of exactly what is still being complained about here.

From my understanding, the hot shoe logging was brought to ~1ms a very long time ago.

[Naterater]

The hotshoe can be recorded even if the signal is under 1 ms, but the timestamp written to the log is only recorded at 50 hz (The CAM loop rate). The following data from the CAM messages (from @bchristal's hotshoe recorder) was recorded on plane 3.8.4 about 6 months ago. The mode was about 180ms from the previous GPS tag, and if it wasn't 180ms, it was 160 or 200. There was a 140ms and a few smaller ones at the beginning of 40, 60, and 80ms. That consistency of stepping 20ms is obviously systematic at 50Hz and not real.

Feedback Delays.BIN.xlsx

Most users are probably good with 20ms, but not me when I want to interpolate going 15-20 m/s. The documentation has been somewhat misleading and confused with the "Fast capture" discussed here: https://discuss.ardupilot.org/t/fast-camera-trigger-capture-in-master/8450/2. They are NOT the same. Just because Ardupilot can detect a microsecond pulse doesn't mean that it applied the correct timestamp to that pulse occurring.

Logs here: https://www.dropbox.com/sh/lhjq33uywtpgi19/AADlQVcrz_4Q1HW8k7TbXQTAa?dl=0

[gastgit]

Hi Excuse me for reviving this thread.. @Naterater thanks for the great clarification on the 50Hz log granularity. However - I am currently using Copter 3.5.5 and am able to get TRIG and then CAM according to the docs found here https://ardupilot.org/copter/docs/common-camera-shutter-with-servo.html#enhanced-camera-trigger-logging

But the best I get is 100msec resolution (specifically I get a constant 300msec b/w TRIG and CAM in my case). As noted above, this is not fine enough.

How do I get to the 20msec discussed above or better - where in the code is it "hard wired" to 20msec? Is it possible to write at 1msec granularity to the log file?

Tnx !