search

MRPT / GSoC2016-discussions

See README for instructions
7 stars 2 forks source link

PnP Algorithms toolkit for MRPT (By: Chandra) #1

Closed jlblancoc closed 8 years ago

jlblancoc commented 8 years ago

Initial description The objective of this project is to develop a novel closed form Perspective n Point (PnP) pose estimation algorithm for a monocular camera. A variety of techniques have been developed to estimate the pose using a monocular camera. However, the algorithms developed do not meet the simultaneous requirement of computation time, precision and efficiency. The proposed algorithm, Linear Least Squares (LLS-PnP) would minimize the cost function (non-linear error norm) which can be explicitly expressed as a quadratic function of the rotation matrix. The position is eliminated using least squares formulation. This leads to the possibility to solve the estimation problem in a closed form w.r.t. the rotation matrix. The proposed algorithm will break new grounds in computer vision and can be extended for pose estimation using Stereo vision. The final product (toolkit) will comprise of the new algorithm and the state-of-the-art PnP algorithms, which will provide the end user the choice to choose any given algorithm as well as to enable performance comparison between the various algorithms.

Progress: All comments below reflect the interactions between the GSoC student and the mentor(s).

Final results:

jlblancoc commented 8 years ago

Student: Chandra Prakash @mangi020 Mentor: @jesusbriales Backup mentor: @famoreno

jlblancoc commented 8 years ago

You are welcome!! We're also hoping to see you enjoying the internship. :-)

The mentors will go on with this project, but I guess that one good general advice in this project is to start this month browsing mrpt-vision, thinking where and how your methods will fit there, and whether you may feel like changing / updating part of the API in that library. As you can see here, a mid-term goal of MRPT is to undergo a big clean up of that library, so anything you propose in that line will be more than welcome, related or not with this project.

Again, welcome on board!

jbriales commented 8 years ago

Hi Chandra, we appreciate your motivation and I'm sure this is going to be a great experience for all of us. To begin with, as already said by Jose, I think it will be interesting to familiarize yourself with the existing code and vision library in MRPT and how the deliverables of the project may fit in. I think it may be interesting as well to have a look into the implementation of PnP algorithms in other libraries, e.g. in [OpenCV](http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#bool solvePnP%28InputArray objectPoints, InputArray imagePoints, InputArray cameraMatrix, InputArray distCoeffs, OutputArray rvec, OutputArray tvec, bool useExtrinsicGuess, int flags%29) or OpenGV. Following that we will be able to establish more concrete goals according to the expected timetable. And, of course, do not hesitate to make any questions/suggestions.

Best,

Jesus

ghost commented 8 years ago

Hi @jlblancoc I have created a new github account dedicated solely for GSOC - 2016 to help me keep up to date with my updates as my current git account has too many other activities that I am concentrating on. It would be great if you can update my id to @mangi020.

I have forked mrpt to my current git account @mangi020, so that I can learn and work on it.

@jesusbriales, I apologize for the slight delay in replying back. I have started looking at the mrpt-vision source code to understand the basic syntax and the coding style.

Future work: My preferred approach would be to implement all the source code in C++ (preferred over C) and add wrappers to test the source code in python (matplotlib).

We will later on perform unit tests on each specific PnP algorithm and get them ready for benchmark testing.

Most of the papers do not have consistent benchmark tests in their results. We will need to make an independent benchmark test module (preferably in python) to test all the algorithms simultaneously.

I look forward to your comments and to proceed ahead with the same.

Best

Chandra

jlblancoc commented 8 years ago

:+1: to everything!

Take a look at the current Python wrappers. It would be great to follow the same or a very similar convention to define your new wrapper.

jbriales commented 8 years ago

+1 for the proposed approach @mangi020, I think it's a great way to proceed, so you can go ahead with it. Just keep us as up to date as possible on the advances so that we can keep better track of them and help better whenever it is necessary.

Best

Jesus

famoreno commented 8 years ago

Hi everyone! (been out for a few days with no access to the Internet, sorry for the delay). It's a pleasure to meet you, Chandra, and feel your enthusiasm about this! I also agree with your proposed approach and join @jesusbriales's advice of keeping us updated about your progress. Besides, having a (deep) look within the mrpt-vision library is definitely a great idea, so that you become confident with the code.

All the best, Paco (yeah, this is my shortcut for Francisco-Angel Moreno ;-))

ghost commented 8 years ago

Hi Mentors Thank you for your support in accepting my base approach towards the proposal.

I have set-up git on my local machine, completed a hello world code and pushed it to my forked repo - link https://github.com/mangi020/mrpt/blob/master/samples/pnp_algo_demo/test.cpp

I have a few questions

  1. CQuaternionDouble q(1,2,3,4) does not automatically normalize the quaternion. Is there a function to call or any other way to automatically normalize the quaternion.
  2. (For the final algorithm) Will we be using MRPT library (objects and data-structures like say CPoint3D) or will it be built in the more fundamental Eigen library (Matrices and Vectors like say Vector3d) ?
  3. I will complete a basic implementation of an existing PnP algo in a couple of days and create a pull request.
  4. As seen, my base code would show up in mrpt/samples/pnp_algo_demo -> CMakeLists.txt test.cpp . Let me know if any changes are required in the placement of the demo code. (The final algorithm can be placed in the vision library)
  5. Any further feedback and comments would be greatly appreciated.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Mon, Apr 25, 2016 at 3:31 AM, Francisco Moreno notifications@github.com wrote:

Hi everyone! (been out for a few days with no access to the Internet, sorry for the delay). It's a pleasure to meet you, Chandra, and feel your enthusiasm about this! I also agree with your proposed approach and join @jesusbriales https://github.com/jesusbriales's advice of keeping us updated about your progress. Besides, having a (deep) look within the mrpt-vision library is definitely a great idea, so that you become confident with the code.

All the best, Paco (yeah, this is my shortcut for Francisco-Angel Moreno ;-))

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-214205610

jlblancoc commented 8 years ago

Hi,

Some partial answers:

1) See mrpt::math::CQuaternion< T >::normalize()

2) MRPT matrices and vectors, are Eigen objects underneath... so it's fine to use Eigen. We don't do this now because of historical reasons... think that MRPT is older than Eigen!!

4) Ok for me.

jlblancoc commented 8 years ago

ah! For the examples to work, one must also add the directory name (and the expected dependencies) to cmakemodules/DefineExamples.cmake

famoreno commented 8 years ago

+1 to all what @jlblancoc said (I was replying when he showed up with help :P).

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc said, it's a personal choice (nostalgia maybe? :P)

All the best

ghost commented 8 years ago

Hi,

ah! For the examples to work, one must also add the directory name (and the expected dependencies) to cmakemodules/DefineExamples.cmake Already added it and made sure the hello world program compiled. Now everything is set up. So, I can simply do a git push and any changes I make on my local machine will be visible on github.

Started working on a sample PnP algo demo.

@Francisco

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P) Agreed. Will become thorough and use the MRPT data structures for the final algo design and for building modules as well.

@Mentors I also changed the .gitignore file to exclude all object files (.o, ,so, CMakeCache.txt etc. for CMake and c++ ) and during pull request(s) it may also pop up. Just a heads up to let you know.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Wed, Apr 27, 2016 at 12:12 PM, Francisco Moreno <notifications@github.com

wrote:

+1 to all what @jlblancoc https://github.com/jlblancoc said (I was replying when he showed up with help :P).

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P)

All the best

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-215152864

ghost commented 8 years ago

Dear Mentors

I have written a simple C++ program to demonstrate pose estimation using E-PnP (efficient PnP). It works fine. Attached (test1.tar). Read compile command for compiling.

I tried to use BOOST to port the module to python and am having link errors to OpenCV. It says "Import Error: Undefined symbol cvSolve". Please help me figuring out the linking in cmake. (Attached as mrpt_sample.tar).

At the moment I am using Eigen as it is common to both OpenCV, MRPT and is easily portable, to build a testing framework. I am still learning the MRPT data structures and will use it for our final algorithm.

Any further comments would be greatly appreciated.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Wed, Apr 27, 2016 at 5:17 PM, Chandra Prakash Mangipudi <mangi020@umn.edu

wrote:

Hi,

ah! For the examples to work, one must also add the directory name (and the expected dependencies) to cmakemodules/DefineExamples.cmake Already added it and made sure the hello world program compiled. Now everything is set up. So, I can simply do a git push and any changes I make on my local machine will be visible on github.

Started working on a sample PnP algo demo.

@Francisco

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P) Agreed. Will become thorough and use the MRPT data structures for the final algo design and for building modules as well.

@Mentors I also changed the .gitignore file to exclude all object files (.o, ,so, CMakeCache.txt etc. for CMake and c++ ) and during pull request(s) it may also pop up. Just a heads up to let you know.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Wed, Apr 27, 2016 at 12:12 PM, Francisco Moreno < notifications@github.com> wrote:

+1 to all what @jlblancoc https://github.com/jlblancoc said (I was replying when he showed up with help :P).

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P)

All the best

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-215152864

ghost commented 8 years ago

Dear Mentors I have fixed all the linking issues and it is working now. Please find attached a sample of the framework.

Any comments would be greatly appreciated.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sun, May 1, 2016 at 11:57 AM, Chandra Prakash Mangipudi <mangi020@umn.edu

wrote:

Dear Mentors

I have written a simple C++ program to demonstrate pose estimation using E-PnP (efficient PnP). It works fine. Attached (test1.tar). Read compile command for compiling.

I tried to use BOOST to port the module to python and am having link errors to OpenCV. It says "Import Error: Undefined symbol cvSolve". Please help me figuring out the linking in cmake. (Attached as mrpt_sample.tar).

At the moment I am using Eigen as it is common to both OpenCV, MRPT and is easily portable, to build a testing framework. I am still learning the MRPT data structures and will use it for our final algorithm.

Any further comments would be greatly appreciated.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Wed, Apr 27, 2016 at 5:17 PM, Chandra Prakash Mangipudi < mangi020@umn.edu> wrote:

Hi,

ah! For the examples to work, one must also add the directory name (and the expected dependencies) to cmakemodules/DefineExamples.cmake Already added it and made sure the hello world program compiled. Now everything is set up. So, I can simply do a git push and any changes I make on my local machine will be visible on github.

Started working on a sample PnP algo demo.

@Francisco

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P) Agreed. Will become thorough and use the MRPT data structures for the final algo design and for building modules as well.

@Mentors I also changed the .gitignore file to exclude all object files (.o, ,so, CMakeCache.txt etc. for CMake and c++ ) and during pull request(s) it may also pop up. Just a heads up to let you know.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Wed, Apr 27, 2016 at 12:12 PM, Francisco Moreno < notifications@github.com> wrote:

+1 to all what @jlblancoc https://github.com/jlblancoc said (I was replying when he showed up with help :P).

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P)

All the best

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-215152864

ghost commented 8 years ago

Dear Mentors I apologize for putting so many previous mails. I have a successful working model of the class I would like to implement. Please find attached the same. Any comments would be helpful.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sun, May 1, 2016 at 3:08 PM, Chandra Prakash Mangipudi mangi020@umn.edu wrote:

Dear Mentors I have fixed all the linking issues and it is working now. Please find attached a sample of the framework.

Any comments would be greatly appreciated.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sun, May 1, 2016 at 11:57 AM, Chandra Prakash Mangipudi < mangi020@umn.edu> wrote:

Dear Mentors

I have written a simple C++ program to demonstrate pose estimation using E-PnP (efficient PnP). It works fine. Attached (test1.tar). Read compile command for compiling.

I tried to use BOOST to port the module to python and am having link errors to OpenCV. It says "Import Error: Undefined symbol cvSolve". Please help me figuring out the linking in cmake. (Attached as mrpt_sample.tar).

At the moment I am using Eigen as it is common to both OpenCV, MRPT and is easily portable, to build a testing framework. I am still learning the MRPT data structures and will use it for our final algorithm.

Any further comments would be greatly appreciated.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Wed, Apr 27, 2016 at 5:17 PM, Chandra Prakash Mangipudi < mangi020@umn.edu> wrote:

Hi,

ah! For the examples to work, one must also add the directory name (and the expected dependencies) to cmakemodules/DefineExamples.cmake Already added it and made sure the hello world program compiled. Now everything is set up. So, I can simply do a git push and any changes I make on my local machine will be visible on github.

Started working on a sample PnP algo demo.

@Francisco

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P) Agreed. Will become thorough and use the MRPT data structures for the final algo design and for building modules as well.

@Mentors I also changed the .gitignore file to exclude all object files (.o, ,so, CMakeCache.txt etc. for CMake and c++ ) and during pull request(s) it may also pop up. Just a heads up to let you know.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Wed, Apr 27, 2016 at 12:12 PM, Francisco Moreno < notifications@github.com> wrote:

+1 to all what @jlblancoc https://github.com/jlblancoc said (I was replying when he showed up with help :P).

Just one small thing ... I prefer using CPoint3D or TPoint3D instead of Vector3d as it's more clear about the kind of data you are using, but, as @jlblancoc https://github.com/jlblancoc said, it's a personal choice (nostalgia maybe? :P)

All the best

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-215152864

jbriales commented 8 years ago

Hi @mangi020,

sorry about the delay in answering but I was disconnected these past days. I was going to look at your last contributions, but didn't find any commited changes in the repository https://github.com/mangi020/mrpt in the last 5 days. Did you push your changes?

Regards

Jesus

ghost commented 8 years ago

Hi @jesusbriales https://github.com/jesusbriales That was more like a stand-alone application. I have made commits in my local-repo and created a pull-request as well (Travis CI tests passed).

I do need some help understanding the process of linking the shared object file .so created from the C++ code with the pymprt python object. At the moment, I am just linking it with a PnPAlgos python object locally. It would be great if you can help me with some documentation so that I can add the source code in the src directory and put the example code in python and move with the standard pattern of MRPT.

Best

Chandra https://github.com/jesusbriales

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Mon, May 2, 2016 at 10:18 AM, Jesús Briales notifications@github.com wrote:

Hi @mangi020 https://github.com/mangi020,

sorry about the delay in answering but I was disconnected these past days. I was going to look at your last contributions, but didn't find any commited changes in the repository https://github.com/mangi020/mrpt in the last 5 days. Did you push your changes?

Regards

Jesus

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-216264718

jbriales commented 8 years ago

Hi @mangi020 as @jlblancoc explained elsewhere let's keep for now working in your fork only. Trying your changes in a test branch as for now is fine, otherwise I would suggest a pnp branch or similar (just for better description). Then with respect to the distribution of your new files through the MRPT directory and the connection with the pymrpt module I would suggest you follow the standard way in MRPT:

In general, in the current phase it is still time for you to get comfortable with the general MRPT pattern. To the best of my knowledge, there is not an extensive documentation in the layout (except for this maybe) but browsing the code should be enough to introduce yourself (it has worked for all of us in our beginning! :D). Try to refactor your current code following the indications above and let's see if everything goes fine! I would proceed as follows:

  1. Put .h and .cpp files with new functionalities for PnP inside mrpt/libs/vision.
  2. Complete the C++ test example in _mrpt/samples/pnp_algodemo/ (linking to the libmrpt-vision library) to check the PnP functionalities.
  3. Move the Python wrapper functionalities to _mrpt/python/src/pnpbindings.cpp following an approach consistent with the rest of pymrpt content.
  4. Then add your _pnptester.py test using pymrpt.

As always do not hesitate to share any particular question that arises in the process and any of the mentors will help you as soon as possible.

Best

Jesus

ghost commented 8 years ago

Hi @jlblancoc https://github.com/jlblancoc,@jesusbriales https://github.com/jesusbriales I apologize for rushing to a pull request. I will build locally and work on my fork till it is formally ready and verified by mentor(s) to be added to the MPRT master/ nav-devel branch.

I will put all the individual files in the respective directories and build a basic framework for my module as @jesusbriales https://github.com/jesusbriales suggested above.

Also, I will focus on learning about the rest of the MRPT library, try to focus on other issues specifically on code-clean up (refactoring https://github.com/MRPT/mrpt/issues/120) and to get to familiarize myself with mrpt-vision.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Tue, May 3, 2016 at 5:38 AM, Jesús Briales notifications@github.com wrote:

Hi @mangi020 https://github.com/mangi020 as @jlblancoc https://github.com/jlblancoc explained elsewhere let's keep for now working in your fork only. Trying your changes in a test branch as for now is fine, otherwise I would suggest a pnp branch or similar (just for better description). Then with respect to the distribution of your new files through the MRPT directory and the connection with the pymrpt module I would suggest you follow the standard way in MRPT:

-

vision is the library we are going to add the PnP functionality to. I would suggest adding a new directory inside vision for PnP only, so put your .h header files in vision/include/mrpt/pnp and the corresponding .cpp implementation files in vision/src/pnp so that the new PnP functionality is allocated into libmrpt-vision.so. I think having a thorough look into the structure and content of current include and src files in vision will give you insight enough on how to introduce your own content. As already pointed by @jlblancoc https://github.com/jlblancoc, a refactoring https://github.com/MRPT/mrpt/issues/120 is pending for the vision library, so feel free to rearrange things as you browse the code if you find it appropiate. Before jumping to the Python wrapper, I would first check that the new functionalities are correctly added to the vision library by using them in your test _mrpt/samples/pnp_algodemo/test.cpp. Remember to follow the indications of @jlblancoc https://github.com/jlblancoc to add new samples:

For the examples to work, one must also add the directory name (and the expected dependencies) to cmakemodules/DefineExamples.cmake

-

Now, as for the Python wrapper via pymrpt this is relatively new (but stable) and I'm afraid that I have no personal experience with it. Good news is I think the existing module is working quite well and would definitely go for using it. For that purpose you probably should add a new bindings file, e.g. _mrpt/python/src/pnpbindings.cpp and create a new Python module such as pymrpt.pnp inside pymrpt for your new methods. Your _pnptester.py could be allocated in mrpt/python/samples/. I know there is a wiki https://github.com/MRPT/mrpt/wiki/PythonBindings describing the main functionality of pymrpt, although I think it will be best to browse mrpt/python/ to better understand how to use the current wrapper (maybe looking at a relatively simple and intuitive module? E.g. _mrpt/python/src/mathbindings.cpp). Furthermore, I think it was @sem23 https://github.com/sem23 who did a great work here setting up everything from scratch, so if he's available to give some indications here it will be great too.

In general, in the current phase it is still time for you to get comfortable with the general MRPT pattern. To the best of my knowledge, there is not an extensive documentation in the layout (except for this http://www.mrpt.org/libs_tree_layout maybe) but browsing the code should be enough to introduce yourself (it has worked for all of us in our beginning! :D). Try to refactor your current code following the indications above and let's see if everything goes fine! I would proceed as follows:

  1. Put .h and .cpp files with new functionalities for PnP inside mrpt/libs/vision.
  2. Complete the C++ test example in _mrpt/samples/pnp_algodemo/ (linking to the libmrpt-vision library) to check the PnP functionalities.
  3. Move the Python wrapper functionalities to _mrpt/python/src/pnpbindings.cpp following an approach consistent with the rest of pymrpt content.
  4. Then add your _pnptester.py test using pymrpt.

As always do not hesitate to share any particular question that arises in the process and any of the mentors will help you as soon as possible.

Best

Jesus

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-216491978

jbriales commented 8 years ago

Hi @mangi020, how is that first immersion into mrpt-vision and the refactoring of your already coded classes going? As always do not hesitate to ask if you feel it necessary or if you need deeper detail in any of the current tasks.

Best

Jesus

ghost commented 8 years ago

Hi @jesusbriales https://github.com/jesusbriales

Current Progress: pnp_tester.py -> mrpt/samples/python pnp_algos.cpp epnp.cpp -> mrpt/python/src/ pnp_bindings.cpp , epnp.cpp epnp.h -> mrpt/python/include.

The code compiled and executed successfully.

Next steps: pnp_algos.h -> mrpt/vision/include/pnp (create new) pnp_algos.cpp -> mrpt/vision/src/pnp (create new) epnp.h -> mrpt/vision/include/pnp epnp.cpp -> mrpt/vision/src/pnp

Will keep you updated as soon as I am done with it.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Fri, May 6, 2016 at 4:29 AM, Jesús Briales notifications@github.com wrote:

Hi @mangi020 https://github.com/mangi020, how is that first immersion into mrpt-vision and the refactoring of your already coded classes going? As always do not hesitate to ask if you feel it necessary or if you need deeper detail in any of the current tasks.

Best

Jesus

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-217396287

ghost commented 8 years ago

Hi @jesusbriales https://github.com/jesusbriales I apologize for the long week of no mailing.

Current Progress: pnp_tester.py -> mrpt/samples/python libs/vision/src -> dls.cpp epnp.cpp libs/vision/include -> dls.h epnp.h pnp_algos.h

Next steps: Implement a basic test in numpy python and plot results Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sat, May 7, 2016 at 11:21 AM, Chandra Prakash Mangipudi <mangi020@umn.edu

wrote:

Hi @jesusbriales https://github.com/jesusbriales

Current Progress: pnp_tester.py -> mrpt/samples/python pnp_algos.cpp epnp.cpp -> mrpt/python/src/ pnp_bindings.cpp , epnp.cpp epnp.h -> mrpt/python/include.

The code compiled and executed successfully.

Next steps: pnp_algos.h -> mrpt/vision/include/pnp (create new) pnp_algos.cpp -> mrpt/vision/src/pnp (create new) epnp.h -> mrpt/vision/include/pnp epnp.cpp -> mrpt/vision/src/pnp

Will keep you updated as soon as I am done with it.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Fri, May 6, 2016 at 4:29 AM, Jesús Briales notifications@github.com wrote:

Hi @mangi020 https://github.com/mangi020, how is that first immersion into mrpt-vision and the refactoring of your already coded classes going? As always do not hesitate to ask if you feel it necessary or if you need deeper detail in any of the current tasks.

Best

Jesus

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-217396287

ghost commented 8 years ago

Hi @jlblancoc, https://github.com/jlblancoc@jesusbriales, https://github.com/jesusbriales@famoreno https://github.com/famoreno The community bonding period has been very useful for me to deepen my knowledge about MRPT.

In addition, I have successfully created a basic test framework and ran the following algos -Epnp and DLS-Pnp using random rotation matrices. A simple schematic of the translation norm error is attached. A python script calls the c++ source code and generates the plots.

Location of files in fork:

Source code : mrpt->vision->src/include

Python bindings: python->src/include

Script:* python->samples*

My current focus this week will be on the following topics:

  1. Complete the framework (include several additional test and plot functions)
  2. Add more PnP algorithms.

  3. Complete Doxygen documentation for the pnp class.

    I look forward to any suggestions/feedback regarding the same.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sun, May 15, 2016 at 10:54 PM, Chandra Prakash Mangipudi < mangi020@umn.edu> wrote:

Hi @jesusbriales https://github.com/jesusbriales I apologize for the long week of no mailing.

Current Progress: pnp_tester.py -> mrpt/samples/python libs/vision/src -> dls.cpp epnp.cpp libs/vision/include -> dls.h epnp.h pnp_algos.h

Next steps: Implement a basic test in numpy python and plot results Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sat, May 7, 2016 at 11:21 AM, Chandra Prakash Mangipudi < mangi020@umn.edu> wrote:

Hi @jesusbriales https://github.com/jesusbriales

Current Progress: pnp_tester.py -> mrpt/samples/python pnp_algos.cpp epnp.cpp -> mrpt/python/src/ pnp_bindings.cpp , epnp.cpp epnp.h -> mrpt/python/include.

The code compiled and executed successfully.

Next steps: pnp_algos.h -> mrpt/vision/include/pnp (create new) pnp_algos.cpp -> mrpt/vision/src/pnp (create new) epnp.h -> mrpt/vision/include/pnp epnp.cpp -> mrpt/vision/src/pnp

Will keep you updated as soon as I am done with it.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Fri, May 6, 2016 at 4:29 AM, Jesús Briales notifications@github.com wrote:

Hi @mangi020 https://github.com/mangi020, how is that first immersion into mrpt-vision and the refactoring of your already coded classes going? As always do not hesitate to ask if you feel it necessary or if you need deeper detail in any of the current tasks.

Best

Jesus

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-217396287

ghost commented 8 years ago

Hi @jesusbriales , @famoreno Hope you are doing great. I have not heard back from you for a while. I am not sure if you have received all my e-mails as some of them are not listed in the github website.

At mentioned previously, I am working on completing the test framework

My current focus this week will be on the following topics:

  1. Complete the framework (include several additional test and plot functions)
  2. Add more PnP algorithms.
  3. Complete Doxygen documentation for the pnp class.

Attached is a sample plot comparing translation error norm of e-pnp and dls-pnp.

Best Chandra t_error

jbriales commented 8 years ago

Hi @mangi020, sorry for the delay, I was quite busy the last couple of days. I received all the notifications (I think) as long as you always use this thread for communication. I see you're doing a great work and quite fast, so keep that pace :) Now, a tip that comes to my mind is:

For the rest I have compiled and executed the test correctly, so everything looks fine! I saw some remarkably high errors in the plot results as well as not rotation error so far, but I assume you're currently working on that :) Good work!

Regards,

Jesus

jlblancoc commented 8 years ago

(CC: @Logrus & @bergercookie , points 2 and 3 are relevant to all MRPT contributors)

Hi @mangi020, I noticed you have been doing a lot of progress in your branch, great! :-)

But I must warn you about a couple of details that are problematic regarding "MRPT policies" and which may prevent the integration of the code in its present state into the main repository. Nothing hard to fix, don't worry:

1) File & classes organization: Since it seems you are creating several classes, I would advice grouping all PnP classes together under a new nested namespace and subdirectory, for example:

Namespace: mrpt::vision::pnp::XXXXX Directories:

2) From the Contributing document (-> Code style section):

Never, never, put a using namespace XXX; in a header file,

Read the link, and Google's code style, for the very good reasons not to do it ;-)

3) This is more tricky: Since its beginning, MRPT have tried (hard!) to be as "standalone" as possible regarding the build-time dependencies of users. Linux users "just" have to install some packages (inconvenient, but easy), but it's a pain to do that for Windows users. Also, many issues and incompatibilities arise as the number of libraries one single C++ file has to #include grows.

So, the MRPT policy is: do not use other libraries types in public headers. For example, cv::Mat.

Why? Because that forces all MRPT users to have the same version of OpenCV installed in their systems, not only for compiling MRPT, but also if they just want to build their own code.

Imagine what this implies when we want to supply MRPT binaries for Windows: we would have include a entire copy of OpenCV and everything else MRPT uses!! (a waste of disk space, and a dependency nightmare).

In short: In your case, you should have to remove all #include's of OpenCV files in all MRPT public headers (they may, and should remain in .cpp files, of course). And replace cv::Mat (and any other OpenCV types) with either:

I hope I gave you enough insight into the reasons for this additional complication. It is worth for the final user experience.

I though all this was already explained in the Contributing document, but it is not, which explains that you were not aware of all this :-)

ghost commented 8 years ago

Hi @jesusbriales Yes, the newly added altgorithm UPnP is giving high error and not converging for many values. I am working on fixing it. I have started improving on the commits and pull-requests on forks.

@jlblancoc I would need some guidance on the sorting of files.

I have two branches (pnp_soruce_mod -> better suited)

warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

Also, both branches have contributed header files. i.e. each pnp algorithm has a pnp class (eg. class dls, class epnp) in their respective header files (#include "dls.h", "epnp.h") which explicitly depend on opencv header files. Please let me know if it is okay to let it be as it will need a lot of re-arranging and additional functions to remove opencv dependency from contributed header files.

Current Work: (Est. completion by June 13)

I apologize for the delay in replying back. I was out of city for the long weekend and it took me a while to make the necessary changes. Thank you for correcting me in the early stage.

Best

Chandra

jlblancoc commented 8 years ago

Hi Chandra,

I took a fast look at your code and I still see many cv::Mat definitions, right?

What I meant is that it will be better to use:

There are methods to convert mrpt::utils::CImage back and forth cv::Mat, I think you could locate them by searching for cv::Mat inside mrpt::hwdrivers.

About the gcc warning: took it seriously, don't do this!! :-) Search online about methods to convert between the two datatypes, a brute force casting will 100% lead to a runtime crash!

Best, JL

ghost commented 8 years ago

Hi Jose

I took a fast look at your code and I still see many cv::Mat definitions, right?' The branceh pnp_source_mod should not have any opencv/cv::Mat definitions in pnp_algos.h. All of them have been moved to pnp_algos.cpp As mentioned, there are opencv declarations in the contributed opencv methods say epnp.h, dls.h

What I meant is that it will be better to use:

  • mrpt::utils::CImage whenever you need to get/return an image.
  • Eigen::XXX for vectors, matrices, etc. containing "numbers", not images.

There are methods to convert mrpt::utils::CImage back and forth cv::Mat, I think you could locate them by searching for cv::Mat inside mrpt::hwdrivers. I will remove opencv definitions from the contributed lib header files say epnp.h, dls.h etc. as well and add the CImage declarations wherever possible.

About the gcc warning: took it seriously, don't do this!! :-) Search online about methods to convert between the two datatypes, a brute force casting will 100% lead to a runtime crash! It should be done.

Best, Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Mon, Jun 6, 2016 at 5:44 PM, Jose Luis Blanco-Claraco < notifications@github.com> wrote:

Hi Chandra,

I took a fast look at your code and I still see many cv::Mat definitions, right?

What I meant is that it will be better to use:

  • mrpt::utils::CImage whenever you need to get/return an image.
  • Eigen::XXX for vectors, matrices, etc. containing "numbers", not images.

There are methods to convert mrpt::utils::CImage back and forth cv::Mat, I think you could locate them by searching for cv::Mat inside mrpt::hwdrivers.

About the gcc warning: took it seriously, don't do this!! :-) Search online about methods to convert between the two datatypes, a brute force casting will 100% lead to a runtime crash!

Best, JL

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-224111635, or mute the thread https://github.com/notifications/unsubscribe/ARyEsd0YA2ig7asKLZQ2Q4PV8vNJ3KLhks5qJKLPgaJpZM4IOJqq .

jlblancoc commented 8 years ago

The branceh pnp_source_mod should not have any opencv/cv::Mat definitions

Ah, ok I see! Then, the only public header is supposed to be pnp_algos.h, right? If not, the rest of methods (php.h,...) will have to be adapted as well to only expose Eigen / MRPT classes in their API.

Some further comments on how to improve the code in pnp_algos.h:

#include <mrpt/vision/link_pragmas.h>

and add VISION_IMPEXP to every public class/structure, e.g.:

class VISION_IMPEXP CPnP

All good!

ghost commented 8 years ago

Ah, ok I see! Then, the only public header is supposed to be pnp_algos.h, right? If not, the rest of methods (php.h,...) will have to be adapted as well to only expose Eigen / MRPT classes in their API.

Yes, the idea is to expose only the pnp_algos.h

Some further comments on how to improve the code in pnp_algos.h:

  • Don't add a using namespace to a header! ;-)
  • Your class definition will work OK in GCC + Linux, but will not in Windows, because any class going into a DLL (.so in Linux) needs some special attributes or it will not be visible to user code for linking. You will have to add:
    1. Completed

<

include <mrpt/vision/link_pragmas.h>

and add VISION_IMPEXP to every public class/structure, e.g.:

class VISION_IMPEXP CPnP

  1. Will add it ASAP
    • The class itself and the method will need some Doxygen documentation, but it has not to be done now.
  2. Working on doxygen docs.

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Tue, Jun 7, 2016 at 2:23 AM, Jose Luis Blanco-Claraco < notifications@github.com> wrote:

The branceh pnp_source_mod should not have any opencv/cv::Mat definitions

Ah, ok I see! Then, the only public header is supposed to be pnp_algos.h, right? If not, the rest of methods (php.h,...) will have to be adapted as well to only expose Eigen / MRPT classes in their API.

Some further comments on how to improve the code in pnp_algos.h:

  • Don't add a using namespace to a header! ;-)
  • Your class definition will work OK in GCC + Linux, but will not in Windows, because any class going into a DLL (.so in Linux) needs some special attributes or it will not be visible to user code for linking. You will have to add:

include <mrpt/vision/link_pragmas.h>

and add VISION_IMPEXP to every public class/structure, e.g.:

class VISION_IMPEXP CPnP

  • The class itself and the method will need some Doxygen documentation, but it has not to be done now.

All good!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-224200851, or mute the thread https://github.com/notifications/unsubscribe/ARyEsduuTKUYn08dSXkcZEDRGo1KL8evks5qJRx5gaJpZM4IOJqq .

jbriales commented 8 years ago

Hi Chandra, only for your knowledge let me remind you that the midterm evaluation is approaching. I think you have are doing well so far. Maybe it would be interesting to summarize in which state are we right now:

Just tell us a little to keep us updated :)

Best

Jesus

ghost commented 8 years ago

Hi @jesusbriales Apologies for the slight delay in replying back. Hope this helps.

Progress so far :

Current Work::

Tasks to be completed (Estimated Timeline) :

Task Complications:

Best

Chandra

jbriales commented 8 years ago

Great update @mangi020, as I thought you're in the good way. Did you get to fix the issues with the U-PnP algorithm?

Best

Jesus

jbriales commented 8 years ago

Hi @mangi020, I have already submitted your midterm evaluation. First of all, I'm glad to tell you that we all agree that you pass this midterm evaluation without any problem, that's good! :+1: Nevertheless, I think it's a good point to collect some constructive points:

  1. Remember the indications provided by @jlblancoc wrt the namespaces. If you take e.g. the class CFeature in vision and look at its headers and source you will notice how:
    • The headers are all nested in a mrpt::vision namespace. In your case you can put all your classes into the mrpt::vision::pnp namespace.
    • There is no using namespace in any header, not even using namespace std! I think you still have one in pnp_algos.h.
    • In the source file you can have as many using namespace as you want (e.g. using namespace mrpt::vision::pnp) to avoid cluttering. This is part of the coding style in MRPT (but also for many other libraries) and it's a good lesson to take away.
  2. I would suggest moving a little forward on the benchmark framework. This framework can be useful not only for the final benchmarking but also as a debug tool for discovering possible faults, so pervasive when coding this kind of functionalities for the first time. Thus, moving ahead right now can benefit the remaining tasks further. Some simple features that might enrich the benchmark readily could be showing e.g. some usual statistics of the errors (mean, median, etc.).

Best,

Jesus

ghost commented 8 years ago

Hi @jesusbriales

further.

I have already submitted your midterm evaluation. First of all, I'm glad to tell you that we all agree that you pass this midterm evaluation without any problem, that's good! :+1: @jlblancoc , @jesusbriales , @famoreno Cheers and Thanks for being an awesome mentor. Looking forward to the rest of the project. I have also completed the mid-term evaluation.

Remember the indications provided by @jlblancoc wrt the namespaces. If you take e.g. the class CFeature in vision and look at its headers and source you will notice how:

The headers are all nested in a mrpt::vision namespace. In your case you can put all your classes into the mrpt::vision::pnp namespace.

The idea would be to have the structure as follows:

namespace pnp { class epnp(){}; class dlspnp (){}; ... }

There is no using namespace in any header, not even using namespace std! I think you still have one in pnp_algos.h. Left over by mistake. Will be corrected.

In the source file you can have as many using namespace as you want (e.g. using namespace mrpt::vision::pnp) to avoid cluttering. This is part of the coding style in MRPT (but also for many other libraries) and it's a good lesson to take away.

I would suggest moving a little forward on the benchmark framework. This framework can be useful not only for the final benchmarking but also as a debug tool for discovering possible faults, so pervasive when coding this kind of functionalities for the first time. Thus, moving ahead right now can benefit the remaining tasks further. Some simple features that might enrich the benchmark readily could be showing e.g. some usual statistics of the errors (mean, median, etc.).

Started working on it. It is true that a number of algos have been added and the #benchmarks are very few.

Also, the bugs with UPnP is not yet fixed. That is the only algorithm which is not consistent with the observed results. Need to check it step by step to make appropriate corrections. This is supposed to be the best algorithm.

I do apologize for any delays in replying back. I am out of city and will not be back till the weekend. Will try to complete the header file fix for merge request https://github.com/mangi020/mrpt/pull/20 in a day or ASAP.

Hope it helps.

Best

Chandra

jbriales commented 8 years ago

Great Chandra,

I'm outside until Monday too, so no need to hurry.

Best

Jesus

jbriales commented 8 years ago

Ok, first a few "late" comments:

Now, I had a thorough look to the UPnP paper and code (both yours and the original one by Kneip) and I see the algorithm is not trivial. I understand you may have a few issues getting it to work. So I wondered why don't you use, at least for now, the original implementation in the OpenGV library as a dependency, at least for now? That way you could avoid spending too much time getting it to work and focus in the rest of milestones instead.

Best,

Jesus

PD: Try to keep empty lines between text blocks when replying and using blockquotes (use Preview!). Otherwise the quotes get mixed with your answer and it's quite difficult to identify each part! :)

ghost commented 8 years ago

Hi @ jesusbriales https://github.com/jesusbriales I do apologize for not being very active in the last couple of weeks. I have a very important deadline to meet on July1. I will make it up during the upcoming long weekend of July 4th.

Current Tasks (To be completed by July 4 weekend):

  1. Put pnp into mrpt::vision namespace; Move private headers from mrpt/include directory to src/ directory; Improvise basic code structure
  2. Fix UPnP (Use OpenGV code)
  3. Test and fix LHM algo (new to be added)
  4. Complete documentation (doxygen)

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sat, Jun 25, 2016 at 4:51 AM, Jesús Briales notifications@github.com wrote:

Ok, first a few "late" comments:

  • The structure you propose namespace pnp { class epnp(){}; class dlspnp (){}; ... } seems good, but is there any good reason to not to put that pnp namespace inside the more global mrpt::vision namespace?
  • Wrt the private headers, if you are exposing only _pnpalgos.h the rest of them should be placed in the same folder as the source counterparts (in mrpt/libs/vision/src/pnp, rather than mrpt/libs/vision/include/mrpt/vision/pnp). This is just to be consistent with the privacy of these headers.

Now, I had a thorough look to the UPnP paper and code (both yours and the original one by Kneip) and I see the algorithm is not trivial. I understand you may have a few issues getting it to work. So I wondered why don't you use, at least for now, the original implementation in the OpenGV library http://laurentkneip.github.io/opengv/ as a dependency, at least for now? That way you could avoid spending too much time getting it to work and focus in the rest of milestones instead.

Best,

Jesus

PD: Try to keep empty lines between text blocks when replying and using blockquotes (use Preview!). Otherwise the quotes get mixed with your answer and it's quite difficult to identify each part! :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-228527500, or mute the thread https://github.com/notifications/unsubscribe/ARyEsdI9bg8G11fHl3j7rEgiWRiqR7CJks5qPPoggaJpZM4IOJqq .

ghost commented 8 years ago

Hi @jesusbriales I do apologize for the short break from MRPT. I had to complete the revisions for my Masters thesis by July 1 and hence the rush. PFA the latest changes.

This local commit headers_namespace into branch ## pnp-dev) modifies the code structure and layout of the pnp algorithm toolkit.

# Tasks Completed:

# Fixes of re-structuring:

  1. Moved Private header files (eg. dls.h, epnp.h) from public include directory (include/mrpt/vision/*.h) to the src directory (mrpt/src/pnp/*cpp)
  2. Moved _include/mprt/vision/pnp/pnpalgos.h -> _include/mrpt/vision/pnpalgos.h (only one header file)

# Guidance required in re-structuring:

#Current Tasks

@jlblancoc I tried to compile and build the MRPT/master on Raspberry-Pi 2. All the modules compiled successfully but I got errors in unit-tests (specifically in _mrpt/libs/base/src/math/matrix_ops1unittest.cpp). The plan is to test real-time working of our algorithm on the Pi-Camera (end of project).

jlblancoc commented 8 years ago

Hi,

I tried to compile and build the MRPT/master on Raspberry-Pi 2. All the modules compiled successfully but I got errors in unit-tests (specifically in mrpt/libs/base/src/math/matrix_ops1_unittest.cpp). The plan is to test real-time working of our algorithm on the Pi-Camera (end of project).

What OS are you using? If it's Ubuntu, you could directly install MRPT from our PPA, which supports the PI's armhf processor as well.

In Launchpad, all unit tests seem to pass OK for armhf (see buildlog in: https://launchpad.net/~joseluisblancoc/+archive/ubuntu/mrpt/+build/10042205 ).

What is the exact error you are finding?

Cheers,

ghost commented 8 years ago

Hi Jose I am using the Ubuntu Mate for Raspberry Pi. The error is not being displayed. PFA a snapshot of the build failrure from my Pi system.

Also if you are using NX no machine, you can log into the Pi (server), with the following ip and port, (67.165.173.163:20084), username - chandra, password- mrpt_chandra

Best

Chandra

Chandra Mangipudi Masters Student Mechatronics and Intelligent Machines Lab Dept. of Mechanical Engineering Univ. of Minnesota Ph - 612-226-8346 Web- http://cp.branded.me/

On Sun, Jul 3, 2016 at 1:14 AM, Jose Luis Blanco-Claraco < notifications@github.com> wrote:

Hi,

I tried to compile and build the MRPT/master on Raspberry-Pi 2. All the modules compiled successfully but I got errors in unit-tests (specifically in mrpt/libs/base/src/math/matrix_ops1_unittest.cpp). The plan is to test real-time working of our algorithm on the Pi-Camera (end of project).

What OS are you using? If it's Ubuntu, you could directly install MRPT from our PPA, which supports the PI's armhf processor as well.

In Launchpad, all unit tests seem to pass OK for armhf (see buildlog in: https://launchpad.net/~joseluisblancoc/+archive/ubuntu/mrpt/+build/10042205 ).

What is the exact error you are finding?

Cheers,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MRPT/GSoC2016-discussions/issues/1#issuecomment-230137735, or mute the thread https://github.com/notifications/unsubscribe/ARyEscNKGRC0f4GRnSGaaul4PmFkrriPks5qR1NXgaJpZM4IOJqq .

famoreno commented 8 years ago

Hi @mangi020!

Quick answer to some of your questions. I'll come back later with more.

Certain algorithms integrated from the OpenCV libaray require OpenCV to be installed before they can be called upon or built upon. Define CFLAGS and src/include code to only compile OpenCV dependent algorithms if OPENCV is installed.

Use this code snippet in *.cpp where the installation of OPENCV is a requisite and must be checked (also to check an specific OPENCV version)

#if MRPT_HAS_OPENCV
    #   if MRPT_OPENCV_VERSION_NUM < 0x240
            THROW_EXCEPTION("This function requires OpenCV > 2.4.0")
    #   else

    [OPENCV-dependant method code]

    #endif

Unit test location for pnp ???

Probably the best place for unit test location is mrpt/libs/vision/src/pnp/, follow the instructions in here and here to create the tests (see also existing tests for more examples).

ghost commented 8 years ago

Hi @jesusbriales I have written the basic approach to solving the system of equations. method.pdf

However, I do not have much experience in convex optimization and would like to know if you have some good references/introduction papers which I can follow specifically to understand the following topics - Macaulay Matrices, Grober bases, Stotter-Moller equations, Hankel matrices, SVD for polynomial equations. The objective is to solve constrained multi-variate quadratic polynomial equations.

I could find a paper which does the exact same optimization that I would like to perform using a single step SVD 15-25.pdf

but I could not understand most of it. Your guidance would be really helpful.

Also if there are better approaches/suggestions, it would be great to learn from.

@famoreno Thanks. The links to unit testing were really helpful. Started working on implementing the same.

jbriales commented 8 years ago

About the number of potential points to check I'm not sure if I understood well your claim. If you think about the possible stationary points in a quadratic polynomial problem with 9 variables, according to Bezout's theorem there will be up to 2^9=512 possible stationary points. This is quite higher than what you expect, 1 per variable. Since the complexity of the problem grows exponentially with the number of variables, I wouldn't discard using quaternions instead of the matrix elements (see e.g. UPnP's paper).

I'm afraid I don't know much about the topics you are asking for. I myself tried to understand some of them in the past (e.g. Macaulay matrices and Grobner bases) but didn't succeed, so the amount of related work that I could suggest is quite limited:

In general, I would be quite careful with using these approaches since solving all the stationary points and then checking all for finding the global minimum seems like overkilling. In a different approach, there is also much theory about global optimization of polynomial problems using convex optimization, where only one global minimum point is searched. See e.g. GloptiPoly: Global optimization over polynomials with Matlab and SeDuMi.

Finally, a quite different approach I (and @jlblancoc) are maybe more familiar with is to treat the rotation as a variable in a Lie group (manifold), which turns the problem into a non-constrained optimization. You can find a gentle and practical introduction to these topics in Sections 8, 9 and 10 of @jlblancoc's report. The main downside in this approach is that the arising problem is more non-linear and only local optimization methods can be applied. In principle, global optimization doesn't look possible under this approach, but I found recently a paper describing a method that could be promising used together with this method (I haven't had the time to test it yet): Yang, Y. (2015). A globally and quadratically convergent algorithm with efficient implementation for unconstrained optimization. Computational and Applied Mathematics, 34(3), 1219-1236.

I hope this helps,

Jesus

jlblancoc commented 8 years ago

Hi guys! Sorry, I'm really busy these days (and who is not?) and couldn't catch up with all the interesting discussion in time.

Just my two cents:

@mangi020

I have written the basic approach to solving the system of equations.

Your idea of changing the parameterization to convert it into quadratic seems plausible at first sight, but then I think that the constraints render it back into nonlinear with multiple possible minima... Imagine this problem in R^2:

J = 1/2 [x1 x2]^T W [x1 x2]

honoring a sphere constraint:

x1^2+x2^2=1

(Your case is a hypersphere (9-sphere) plus the orthogonality conditions.)

If you visualize the cost function as a paraboloid in 3D, the solution space lies in its intersection with a circumference. Depending on the weights in W, there would be at least 2 solutions, if I'm visualizing it right...

To be more confident on the solution, have you tried to formulate it with Lagrange multipliers?

@jesusbriales Thanks for the reference to the 2015 paper! From the abstract it looks pretty much like a variation of Levenberg-Marquardt, but will read it...

ghost commented 8 years ago

Hi @jesusbriales @jlblancoc
Thank you for the 2015 paper. I do apologize for my mistake regarding the number of solutions. There are going to be more solutions than 1. I am working on a solution. The idea is to use Lagrangian multipliers to solve for the gradient equations. I should be done shortly.

Just a brief note -

Let J =x^2 + y^2 + z^2 be the cost function to be optimized then dJ _ dx = 2x =0, dJ _ dy = 2y =0, dJ _ dz = 2z =0 will represent straight lines. Hence there won't be as many solutions as 2^3 or 2^n as we are looking at curve fitting on the gradient of the least squares cost function (which would be straight lines) subject to orthogonality constraints (which would be non-linear).

In the meanwhile, some of the tasks I have completed (branch -> pnp-dev) are as follows:

Completed Tasks:

Help Required:

Final module Prechecks: Also as we are approaching the final deadline, it would be great to get feedback on the final touches to be added to the PnP module. Some of the concerns addressed in the Midterm review are listed below:

Current Tasks:

PendingTasks:

jbriales commented 8 years ago

Hi, sorry for the delay. I'm quite busy these days preparing for leaving for a conference, and I won't be back until late July. Anyway, keep us aware of the progress in the theoretical issues (I didn't completely catch the curve fitting idea, but I would rather see it in context when you provide an updated document ;)) As for the little issues involving code, I'm sure @jlblancoc or @famoreno will cover my back and help you during these 2 weeks :)

Regards, Jesus

jlblancoc commented 8 years ago

Hi @mangi020 ,

Just a quick / formal reminder about the most important tasks regarding GSoC2016:

Happy coding!

ghost commented 8 years ago

Hi @jlblancoc Sure thing. I will try to wrap up everything by the weekend.

Best

Chandra