Photogrammetry use

[Teo47]

Hi, i'm actually student in topography master. For my last project, i have to improve a MMS (Mobile Mapping System) that you can see here: http://www.sitecoinf.it/en/solutions/road-scanner

The goal of my study is to synchronize 2 GoPro Hero+ LCD to colorize point clouds. For the moment, my camera configuration is in Time-Lapse 0.5s.

To synchronize the 2 cameras with the INS system, i already know how to make the internal and external orientation (Photogrammetric purpose). But i have no idea of how to know time of every photo and how export it on PC. To colorize the point cloud, the orientation and the position of the photo have to bel well-known on the international system. I wish a temporal precision about a millisecond.

I want to use your code and library to control the 2 GoPro via PC under Windows. Get the information of photo acquisition with Windows time (can be precise a millisecond ex: https://msdn.microsoft.com/en-us/library/ms724408(VS.85).aspx) on the configuration TimeLapse 0.5sec.

I am totally novice on http, i worked a bit on C++ and Java but i am not expert at all. Do you think my project is applicable with your work? And do you know some tutoriel website to improve me on programmation ?

Thank you

[KonradIT]

Hi,

Gonna reply with what I know from this repo and my other stuff: you could have a script that takes a photo and downloads it to your computer, and it writes the time in to the photo. As an example:

require '../GoPro/GoPro'
require '../GoPro/constants'

herocam = Camera.new
while sleep 5;
    herocam.shutter(Shutter::ON)
    sleep 5
    herocam.dl_media()
end

This was using my library gopro-rb-api, with goprohero lib it would be:

from goprohero import GoProHero
import time
herocam = GoProHero(password='yourpass')
while(True):
        herocam.command('record', 'on')
        time.sleep(2)
        herocam.image() #Do whatever with this

Let me know if it helps you.

[Teo47]

Thank you for your fast answer. I'm gonna tryied this later. Firstly i need to discover Python and connect my two GoPro on WIFI in the same time using your work @KonradIT and @joshvillbrandt work.

Do i need two Wifi cards for this ?

Thank you

[KonradIT]

@Teo47 you do need a computer able to connect to WiFi networks. I use a USB antenna when I test with my desktop computer.

Also, I don't think you can connect to two cameras at the same time, you'll need to change their IP addresses.

[laxnpander]

I am quite sure that synchronizing GoPros is close to impossible. It just hast not the functionality for an external software or hardware trigger. Yes you can make photos, yes you can add a time stamp, but you will never be able to trigger your cameras at the same time that would be necessary for stereo vision.

[Teo47]

My purpose is not to synchronize the release of the two GoPro to take photo exactly at the same moment. I only need to have a temporal information about the release of every GoPro for every photo taken in a precision about 1 millisecond. And create a little interface to start the gopro acquisition and stop it, and create a list of timestamp for each photo.

I'm actually discovering Python and I hope it will not be difficult to create my project.

[laxnpander]

Ah okay, but still: you definitely need to use gopros? There are many problems that might occure: 1) Image transfer: Option 1: Make high res image, save on sd and then grab to your system -> bandwith of wifi might limit the fps you can run + wifi is not the most stable connection (compared to usb for example) Option 2: You use livestream feature, snap image directly to your main hardware -> higher fps, lower resolution / quality 2) Camera settings: Gopro images are made to look good, not to be used in robotic scenarios -> you have very little options of changing settings like for example exposure time 3) Fisheye: I think in your scenario you might somehow need to correspond 3d -> 2d informations, fisheye cameras are really just the worst for this. Can be solved by buying non fisheye lens (-> expensive) 4) Timing: Because gopros are not meant for low level control, you have no informations about what is happening in the background -> yes he takes a picture, but at the moment you get it what has happened to it in the meantime? Did some algorithms manipulate the image? You need the way "Reading pixels -> memory -> timestamp -> your software" but it is very likely that the process here is "Reading pixels -> memory -> gopro magic -> timestamp -> your software". Does this fulfill your 1 ms requirement? Maybe yes, maybe no. Maybe only sometimes, you won't know. 5) Rolling shutter: For timing purposes rolling shutter is not really perfect. What is the time of your image reading? When the first row of the sensor is read? Or the last?

These are just some points I know as medium experienced vision guy. There might be more. Don't want to discourage your plan, but it if you have the option to use another camera, maybe some designed for robotic applications: It will most likely save you time, frustrations and money.

[Teo47]

No, not definitely need to use Gopro. Thank you for your attention

1. Image Transfer: I planned the first option for more speed information. Transfer each image by WIFI seams difficult for the time-lapse i need 0.5sec. I know WIFI is not the most stable but it is not possible to control the GoPro via USB. (I dont found any topic for it). Do you think WIFI isnt able to be precise just for release and time information ? I found some library to calculate the answer time of a function in Python. It can be usefull to have an idea.
2. Camera settings What info i found is: - for last GoPro (Hero5, Hero4) the exposure time is automatically adapt with sensors
   • but for old GoPro like Hero3 or Hero+ LCD (mine) it is always the same
3. Fisheye Fisheye is not a problem, I have calibration method for georeferencing interior and exterior orientation of the camera
4. Timing I agree with you it's the main problem. I really don't know how to program it. But I'm sure it is possible
5. Rolling suhtter I think that this time lapse is too fast so that I worry about it

I am specialized in photogrammetry an topography but i don't understand electronic clock and all those time problems. Thank you discussing with me, it is helpfull for me. I plan realised this project with GoPro because the company where i'm actually working already had 2 GoPro Hero+ LCD. But it is possible to buy new cameras if the project is not possible with GoPro. Do you know where i can find that kind of camera for lower price ? I find some cameras but it is really expensive. http://www.i2s-vision.fr/selecteur.asp?cat=1&sX_Menu_selectedID=leftV_CA1&Pk_type=7&Pk_interface=&submit=Recherche... http://www.edmundoptics.fr/cameras/gigabit-ethernet-cameras/allied-vision-manta-gige-cameras/3318/ http://www.stemmer-imaging.fr/fr/produits/serie/avt-mako/

[laxnpander]

1. Yep, the wifi only control is a big negative on the gopro side. I think there is no easy solution, except you find a firmware hack to deactivate USB mode. But I don't think this is realistic.
2. Hmm, okay. Point is you have no control about it. With an industrial camera you are free to adapt your exposure to your situation (fast movement, fast exposure -> no blurr for example). But jeah, might be okay for your usecase.
3. Problem is: the worse the quality of your lens (high distortion, abberations), the worse your camera model fits reality. So even though you calibrate your camera, it might not be enough for robust solutions (thats why industrial cameras are so expensive).
4. Did you try the example code of KonradIT ? Does it work? Then you might play around extracting the current time before snapping an image and afterwards and see how much time passed.
5. Hope for you, you are right 💃

Stemmer is definitely high quality camera. Whats your budget? Your are right, you won't find anything below ~300€. But it's worth the money. There is also a german company I have made good experiences with: https://de.ids-imaging.com/home.html (they are also ~300€ and upwards)

Best regards

[Teo47]

3. Look at this if you're interested in: http://www.mdpi.com/1424-8220/14/9/17471 or http://www.ee.oulu.fi/mvg/files/pdf/pdf_697.pdf . It's for wide-angle or fisheye camera calibration.
4. I'm working on ! But it's pretty hard ! I'm actually trying to install a ruby editor so i'm far far away !

My budget is not defined, i need to see with my boss but i think around 300euros because i need 2 cameras. But for really improve the system, I need a 10Mpixel camera and fps between 2-10 fps. Do you think it is possible to find that kind of camera for my budget ?

[laxnpander]

Hm, windows developer?

300€ is rough, I am currently in the same situation looking for a photogrammetry camera with slightly other requirements. But I think it is the lower limit you will spend. Tendency is more like 500+ at least for industrial cameras.

As soon as you can run python programs, it should not be a lot of work. The api and gopro itself does not offer so much functionality to get lost :D

[Teo47]

Yes i'm trying to make it work on Aptana Studio. I have also VirtualBox and Ubuntu os. Do you know some tuto to learn it ? What do ou recommand to use ?

Using Virtual Box seems to me pretty hard because of the Wifi. I have to use it to control the GoPro and it is not easy defined

[KonradIT]

@Teo47 @laxnpander you can use Python too, also, the GoPro HERO4 Black should be cheap now, I got one and it can record w/o fisheye, and set the shutter speed for video and jump up/down on exposure steps. I'd see if you can maybe get two of these: http://hackhd.com/

As per a proof-of-concept of when exactly the GoPro picture was taken (using my library because I know it has the proper methods): video, script