meetings master thesis

[surberj]

20160108: kick-off, see https://www.evernote.com/shard/s259/nl/2147483647/90a256da-8a29-4aa7-8f09-83bea7c5f37f/

[surberj]

20160119: first regular meeting, Margarita, Lucas, me

• proposal
• goal
• timeplan
• semester thesis

[surberj]

20160126: Margarita, Lucas, me

• snapdragon:
  • I will spend some time into it, but priority first lies on data collection, literature and ideation of the localization algorithm
  • Margarita is interested in feedback/experience from me
• report:
  • hardware --> data collection
  • "creating map" --> literature, scan internet, ...
• discussion:
  • very hard environment for 3d reconstruction (little structure, some small moving objects)
  • important to have soon a dataset to play with different map creations and to see if ROVIO (and OKVIS?) are tracking or diverging on the set
  • Margarita: "lateral motion" easier
  • camera configuration? inputs of Lucas: down looking? side looking?
  • map on two layers --> a sparse point/feature layer and a dense layer
  • Lucas: "Which gap to fill?"
  • more accurate than without vision?
  • relocalization?
  • make ROVIO drift less ?
  • Inputs for dataset collection: 131deg FoV, best would be near a building, structures in environment?
• literature/other:
  • Piksi (DGPS)
  • Line3D (people from graz) https://aerial.icg.tugraz.at/
  • VisualSFM

[surberj]

20160407: Helen Olynekova, me Topics: http://helenol.github.io/publications/iros_2015_localization.pdf, Drift estimation, Reference Map creation (global BA), https://3234f89137bccf2ede29cc86e315c75116020d70.googledrive.com/host/0B64GJ60h3Ai1MVVwWTZwekhtcFU/publications/bib/lynen_3dv14.pdf (place recognition)

Discussion:

reference map

• her procedure:
  • pass all information (camera poses, landmark positions, landmark observations, IMU observations) from OKVIS over ROS to the multiagentmapping (closed source)
  • (only for some examples) replace OKVIS poses with external pose measurement (vicon, leica)
  • run BA within multiagentmapping
• she likes the idea of including it in the backend of OKVIS
• in her opinion we should get reasonable results also without adding external poses.

placeless place recognition

• also closed source (included into the multiagentmapping)
• 2D-3D probably more accurate as less dependent on errors in the image poses of the reference map, 2D-2D is the more classical approach
• have a look in the place recognition nodes available in the ROS framework

other points that have been discussed

• she was using an old OKVIS version (3-4 years old)
• in the old version the ceres solver was not included in the OKVIS backend
• one issue she was facing: trajectories used for map creation and trajectories on which you run online estimation have be very similar (placeless place recognition can not deal with high viewpoint changes).
• issue of inaccurate depth estimation with our setting: she referenced to the SVO, where they derived an interesting depth estimation model, which seems to work quite well

important people / further literature:

• simon lynen
• torsten sattler, e.g. http://link.springer.com/chapter/10.1007%2F978-3-642-33718-5_54?LI=true#page-1
• margin dymczyk, thomas schneider, multiagentmapping
• localization with downlooking camera: SVO, http://infoscience.epfl.ch/record/199740/files/ICRA14_Forster.pdf

[surberj]

20160411 Margarita, Lucas, me

Report:

• Meeting with Helen, see above
• I was working on setting up a Global Bundle Adjustment within OKVIS (second optimization map with same costfunction but without marginalizing frames outside the window)
  • working on asl maschinehall dataset
  • cpp only --> I will port it to the ros implementation as soon as possible
  • I only have an easy "live" visualization. I have to find a way to log the results of the normal OKVIS estimates and the globally bundle adjusted estimates.

Discussion:

• Challenge my thesis with respect to Helen's paper:
  • I am doing it on new, more challenging datasets (outdoors, large distance to objects)
  • We will probably use another way to use the reference map for correction than what they did. They use 3D landmarks with BRISK descriptors as reference and match every new keyframe to the whole database with Kd-Tree based nearest neighbors, covisibilty graph filtering and PnP ransac filtering. Our reference map could look different, either A) only keyframes (with poses and keypoints with descriptors) or B) the pointcloud with another higher layer (either 3D-line segments on top of the 3D-pointcloud or denser structure/texture in between the points).
• We have to decide soon between the following 2 options:
  • stick to the plan with the challenging Wingtra flights, which probably means we can not focus on innovate a lot in the map representation (densification, higher semantics)
  • work on easier data (near to structure / ev indoors) but try out other map representations on top of the pointcloud
• Achievements so far:
  • working but drifting versions of OKVIS and ROVIO on the ticino dataset
  • working post-flight global BA implemented within OKVIS to achieve an accurate 3D pointcloud (tested only on the easier asl indoor dataset, where they correct for small accumulated drift errors)
• midterm presentation on 3rd may, 2pm
• Open general question: How to use the reference map? Do we use frames of the reference map within the OKVIS online optimization (instead of the frames in the window we use in normal OKVIS) or do we run OKVIS as it is and use the reference to correct the OKVIS estimates (in an additional optimization).

Decisions:

• Next step: Try global BA on Unispital and ticino datasets.
• We will decide next week, which of the 2 options we will continue with.

[surberj]

20160418 margarita lucas me

Discussion:

• we agreed on the overall system I proposed (see image)
• Instead of using a loop closure we will use a prior from GPS (or Leica, or OKVIS with manual alignment). The reason is that Marlin and Fabiola are working on the loop closure but the system is not likely to be ready till the end of my thesis.
• goal till midterm: show up issues within the "descriptor matching (knn)" and "outlier rejection, PnP ransac" blocks (see image)
• decided not to work on reference map for now.
• I will have a look at Fabiolas data. It contains VI data from clausiusstrasse with GPS.

[surberj]

20160503 midterm presentation (margarita, lucas, max, me) ppt: https://drive.google.com/drive/folders/0BwgFbOX2gQlnR1ByazV3cWR2SVk

We agreed on:

• work with OKVIS map
• use GPS as prior for key point selection within the reference map (instead of conventional place recognition, which compares the new frame's descriptors with all descriptors in the reference map)
• for now, we make the assumption of a down-looking camera (which is reasonable for a quadcopter) to simplify the key point selection and reduce the number of key point candidates.
• we will collect a quadcopter dataset with downlooking VI-sensor and gps data soon. The distance to structure e.g. will probably heavily influence the performance of the algorithm I write. Therefore it will be useful to have data which is similar to what we want to show in the end.

[surberj]

20160530 margarita, lucas, me Report:

• demonstration (video) of OKVIS + GlobalMapAlignment
• Worked on gps loss
• Worked on visualization

Discussion:

• High priority are the experiments to get a feeling for the accuracy --> we want to demonstrate that the system is better than gps + imu and better than OKVIS alone
• Second priority is the visualization, either find some densification of the map to make it look nicer or include a google map like tile as a background.
• Other data could be collected perhaps along the Limmat, with Leica being placed on the other side of the river.

Decision:

• Do the data collection with Leica in Irchel and leave open if we need data on a larger workspace for now.
• Lucas will reserve the Leica, I will talk to Helen about mounting the prism.