Ideal config and how to record rtk gps and pose for each picture taken

[robogrow]

We are building a citizen driven urban/transportation planning online multplayer GUI and are building a digital twin for that purpose. ie. SimCity with real world 3D maps.

I took 150 jpeg picture and ran opensfm. The placement of each camera position and pose is way off, so no useful mesh was produced.

We will mount an IMU and RTK GPS tracker on each cam. We like to take picture with dash, side and rear window cameras. I have heard that SFM algorithms like varied position and poses of camera to get better meshes. sice we are a robotics company we can alter the pose of the dash cam as we travel through scanning areas.

We are working on a cross walk study and want to capture high injury network road segments. These will be used as an augmented reality overlay on a closed course test site and a fully virtual test site for each road section to be simulated. see Red lines are highest risk to pedestrians.

I would assume there is someone who has already done this sine 2014 when it appears mapilary started the OpenSfM project.

We can developing AI (GA) and quantum annealer backends to help the public design optimal communities.

The system's digital twin will receive real-time smart iot sensor data feeds for real-time simulation of a wide variety of mobility systems, traffic conditions, zoning related data.

Let me know where I can find the best open source roadway mesh builder. Has anyone built and iterative reconstruction engine that runs continually to update changes and to improve precision?

Here is a programmatically generated personal rapid transit guideway in Forth Worth.

[fabianschenk]

Hi @robogrow ,

This sounds like a very ambitious goal. What do you mean with "each camera position and pose is way off"? Could you share at least a screenshot? Is the whole 3D reconstruction wrong, is just the scale not correct or are there multiple unconnected reconstructions?

• If the scale is wrong, then this likely because you didn't have GPS positions added to the image's EXIF data.
• If there are not enough matches between images, then the reconstruction is split into smaller parts that are not connected to each other. Thus they are not aligned and might appear off, even if each individual part is correct.
• SfM in general needs some kind of baseline (translation) between images, ideally you cover multiple angles and move the camera up and down. Instead of using a complicated camera setup, couldn't you just use a GoPro Max with 360 images?

Another thing you can consider is filtering problematic scene parts, e.g. sky, moving objects, etc. Some ML methods could help you.

Regarding your last question: If somebody has built such an "iterative reconstructions engine" with your desired behavior, I don't think they would share it as open-source code.

Good luck, Fabian

[robogrow]

I see the EXIF folder data for each image and there is no GPS data for the images. I enabled GPS location permission for camera and now get lat & log but 0 altitude and no DOP. about 11 images of my house produce no reconstructions. Does Opensfm need altitude and DOP to work?

Also I ran dense point clud version of Berlin and Lund data but they are no where near the resolution shown on the Opensfm main website. Any clues?

[fabianschenk]

Hi @robogrow ,

With a screenshot, I meant a screenshot of your built reconstruction to get a better impression what you mean by "each camera position and pose is way off" and not some Google image that doesn't tell me anything.

It should also work without altitude or DOP but a common issue is that the overlap between images is not enough, so not all images can be added to the reconstruction. I'd suggest to simply take more images.

What command and what config did you use to create the dense point cloud for lund and berlin?

Please provide more details such as the commands you ran, logs or screenshots of the actual reconstruction in your next posts, otherwise it's impossible to help.

Best, Fabian