Web Frameworks Library for MRPT (Rachit Tibrewal)

[jolting]

Initial Description

Robots are slowly becoming a part of the internet of things, and easy control and universal access will be a step in future of robotics. MRPT’s Web Framework Library should address this problem.A lightweight publisher/subscriber mechanism needs to be created for MRPT. The C++ server library and the javascript library will then leverage this to create further applications. There will be two libraries, a C++ server library for RPC on the robot. Another will be a javascript library which provides modules for such procedure calls.This task involve serialization of existing mrpt objects for JSON transfer, RPC protocol, websocket setup, 3D rendering of objects in js, reusable js components for sending and receiving data from user created app where the user is largely benefitted from easy to use library methods and objects.

---

• Student: @rachit173
• Main Mentor: @jolting
• Backup Mentor: @jlblancoc

[rachit173]

Update:

1. Pushing code related to rawlog-web-ui backend (C++ server) in mrpt-web.
2. Pushing code related to frontend of rawlog-web-ui to rawlog-web-ui. The frontend node package depends on mrpt-web-js package and thus one needs to link those using npm link. Instructions can be found in the repo's README. I think this will be required until the mrpt-web-js is published to npm. @jolting is there a simpler method for this. Should I continue with creating separate repos for the apps? Also, I was not clear on what type of components should be created? Could you give some examples. Currently, I am creating components in the app for use there. I was thinking of creating this app before starting component development, so that I can get a better idea about what kind of components are useful in apps.

P.S. I have done some major restructuring to mrpt-web dev-rpc-pubsub is now master.

[jolting]

npm link the right thing to use for development. Eventually we'll publish something under @mrpt/<some lib name>. Feel free to come up with a better name.

[rachit173]

    if (classID == CLASS_ID(CActionRobotMovement2D))
    {
      // ----------------------------------------------------------------------
      //              CActionRobotMovement2D
      // ----------------------------------------------------------------------
      cs.changeSelection(1);

      CActionRobotMovement2D::Ptr act =
        std::dynamic_pointer_cast<CActionRobotMovement2D>(sel_obj);

      CPose2D Ap;
      CMatrixDouble33 mat;
      act->poseChange->getCovarianceAndMean(mat, Ap);

      ss << "Robot Movement (as a gaussian pose change):\n";
      ss << " Mean = " << Ap << endl;

      ss << format(" Covariance:     DET=%e\n", mat.det());

      ss << format(
        "      %e %e %e\n", mat(0, 0), mat(0, 1), mat(0, 2));
      ss << format(
        "      %e %e %e\n", mat(1, 0), mat(1, 1), mat(1, 2));
      ss << format(
        "      %e %e %e\n", mat(2, 0), mat(2, 1), mat(2, 2));

      ss << endl;

      ss << " Actual reading from the odometry increment = "
          << act->rawOdometryIncrementReading << endl;

      ss << format(
        "Actual PDF class is: '%s'\n",
        act->poseChange->GetRuntimeClass()->className);

      if (act->poseChange->GetRuntimeClass() ==
        CLASS_ID(CPosePDFParticles))
      {
        CPosePDFParticles::Ptr aux =
          std::dynamic_pointer_cast<CPosePDFParticles>(
            act->poseChange.get_ptr());
        ss << format(
          " (Particle count = %u)\n",
          (unsigned)aux->m_particles.size());
      }
      ss << endl;

      ss << "Estimation method: ";
      switch (act->estimationMethod)
      {
        case CActionRobotMovement2D::emOdometry:
          ss << "emOdometry\n";
          break;
        case CActionRobotMovement2D::emScan2DMatching:
          ss << "emScan2DMatching\n";
          break;
        default:
          ss << "(Unknown ID!)\n";
          break;
      };

      // Additional data:
      if (act->hasEncodersInfo)
      {
        ss << format(
          " Encoder info: deltaL=%i deltaR=%i\n",
          act->encoderLeftTicks, act->encoderRightTicks);
      }
      else
        ss << "Encoder info: Not available!\n";

      if (act->hasVelocities)
      {
        ss << format(
          " Velocity info: v=%s\n",
          act->velocityLocal.asString().c_str());
      }
      else
        ss << "Velocity info: Not available!\n";

      // Plot the 2D pose samples:
      unsigned int N = 1000;
      vector<CVectorDouble> samples;
      vector<float> xs(N), ys(N), ps(N), dumm(N, 0.1f);

      // // Draw a set of random (x,y,phi) samples:
      // act->poseChange->drawManySamples(N, samples);

      // // Pass to vectors and draw them:
      // for (unsigned int i = 0; i < N; i++)
      // {
      //   xs[i] = samples[i][0];
      //   ys[i] = samples[i][1];
      //   ps[i] = RAD2DEG(samples[i][2]);
      // }

      // lyAction2D_XY->SetData(xs, ys);
      // lyAction2D_PHI->SetData(ps, dumm);

      // plotAct2D_XY->Fit();
      // plotAct2D_PHI->Fit();
      /** Serialize xs, ys, ps, dumm data
       *
       * Code Here
       */
    }

this line of code is causing an error

     act->poseChange->drawManySamples(N, samples);

double free or corruption (out)

I checked the type of poseChange and it is mrpt::containers::deepcopy_poly_ptr

@jolting @jlblancoc Is this error due to the deepcopy_poly_ptr type? I had copied the code from RawlogViewer app, and made certain changes to it.

[rachit173]

I have added project boards, for more efficient completion of app and library, please tell in case any improvements to project. board1 board2

[jlblancoc]

Is this error due to the deepcopy_poly_ptr type?

I just tested the equivalent code inside RawLogViewer and it works without crashing. Please, could you put together a small .cpp file + its CMakeLists.txt that reproduces the crash? If yes, open an issue on the main MRPT/mrpt repo with that test code.

[jolting]

Minor critique of the code so far. I find mapMutation and mapAction cleans up the code quite a bit.

[rachit173]

I was not aware of the better style, I will make the change straight away.

[rachit173]

I have added the basic ideas about the library in the google slides presentation, it will be updated as the work progresses further. The slide aims to explain the role of each of the libraries. slides

[jlblancoc]

Hunter will probably take a deeper look to your recent changes, but here comes a quick overview to the slides. Minor typos (I think) and change requests:

• "It comprises of:" -> "It comprises:"
• "Mrpt..." -> "mrpt" or "MRPT" looks better, doesn't it?
• Class names (like "CWebSocket", etc.): it would be great to have those names linked (href) to some documentation for the classes, or at least, to the .h of the corresponding class. Ideally, the former... I don't know how easy is to setup a Doxygen job in a readthedocs site or alike? Also, one can always just use github pages. See this example, published here.
• Adding a schematic graph with boxes, arrows, etc. would be helpful. At the highest level (mrpt-web, mrpt-web-js) and perhaps also with the relationship between some of the C++ classes. Just thinking aloud... but I'm quite a visual guy: an image is normally worth 1000 words ;-)
• It would be "educative" to show an example of how some (small!) piece of data is represented as JSON, showing the native C++ class name, etc.

The RawLogViewer app looks impressive! Again... it would be great to provide detailed instructions on how to launch it (perhaps a link to a wiki page?).

[rachit173]

Thanks @jlblancoc The changes requested seem necessary, I will work on them.

@jolting how should the documentation of mrpt-web-js proceed, should I do in GitHub pages, wiki or what is the popular practice?

[jlblancoc]

The changes requested seem necessary, I will work on them.

Good, thanks!

For the C++ documentation, I would like to use doxygen but am unaware of it, can you provide some pointers about it( or some links).

You can start installing doxygen (apt-get), and using its -g parameter to create a brand new config file. Browse it and the Doxygen docs to see the meaning of the most important parameters. Mostly, the INPUT key which instruct doxygen where to search for source code.

The Doxyfile file could be created just on the root directory.

Then, just invoking doxygen will generate an HTML report of all C++ classes. You can disable all other non-HTML generators (LaTeX, RTF, etc.). As you browse the generated HTML you'll discover plenty of ways of improving it ;-)

Take a look at MRPT's Doxyfile template file (although it's overcomplicated for your project needs)

However, I prefer to push that a little towards the end.

OK, no rush for this second evaluation.

[jolting]

I forked your repo https://github.com/MRPT/mrpt-web-js I added a circleci config.

Looks like the unit tests hangs there after it finishes.

https://circleci.com/gh/MRPT/mrpt-web-js/2

I also changed the package name so that it begin with @mrpt. I previously registered that for the MRPT org.

[jolting]

@jolting how should the documentation of mrpt-web-js proceed, should I do in GitHub pages, wiki or what is the popular practice?

For documenting javascript libraries, the conventional way is a fancy webpage that shows off your web dev skills. A wiki is a good place to start though. I'd rather see finished well documented code rather than a fancy webpage. You can always add that later.

If you do plan on throwing up a webpage quickly for either documentation or demo give surge.sh a shot. https://surge.sh/

[rachit173]

Since you have created MRPT/mrpt-web-js,rawlog-web-ui, should I now fork them or can I continue using my existing repos and create pull requests. I had setup project boards there so, it would be useful, if I could continue to develop on those repos :-)

[jolting]

Keep working on your board. I was just trying to get a build pipeline for npmjs.com

[rachit173]

Thanks for the evaluation. I had added plotly.js, since two.js did not provide direct graphing and plotting. I was planning on completing the rawlog-web-ui and improving the mrpt-web-js library by adding modules, tests and documentation. This would be the plan for next week. After that what should be my focus, another app, or examples? I plan to keep the last week for documentation and unit tests, so, I will have 10 days for either an app or examples.

[jolting]

If you can produce a youtube video + documentation on how to set everything up from scratch it would be great. The target audience is C++ programmers who don't know how to setup node or javascript.

After that more apps or keep adding capabilities to the raw log viewer.

[jlblancoc]

The target audience is C++ programmers who don't know how to setup node or javascript.

I'll be an archetypal example of those guys, so will provide you valuable feedback on your tutorial ;-)

[jolting]

Here's an example of what I consider a good video tutorial format: https://www.vuemastery.com/courses/intro-to-vue-js

[jolting]

If you get done with the rawlog viewer, the documentation and video tutorials. Then you can work on the Reactive Navigation demo.

[rachit173]

I will start documentation, build configuration of everything.

[rachit173]

Started with something basic: ESDoc for automated build of source documentation and published it on http://mrpt-web.surge.sh/ thinking of adding tutorials and further examples here itself added CI for docs on branch dev-docs Improved docs, added examples (code snippets) to many of the classes.

[jlblancoc]

ESDoc for automated build of source documentation and published it on http://mrpt-web.surge.sh/

👍 looks great!

Shouldn't that also include:

• Short instructions/(link to)examples on how to use those classes in real apps?
• Instructions to launch the RawlogViewer demo app & a link to the most relevant parts of the code?

[rachit173]

Thanks, I am going to add tutorials to the manual section

[jolting]

Looks like the mrpt-web-js unit tests are still broken.

[jolting]

What do you think about specifying a directory on the server in which rawlogs will be served up from?

It is a bit awkward to have to know the name of the file on the file system.

[jolting]

I think there needs to be a method for connecting to a server other than localhost:5000.

[jolting]

For the 3d animation is there any way to move the pivot point?

[jolting]

One thing to note about the 3d scan visualization is that it is capable of rendering multiple sensors. Each sensor will have its own sensor label and have a different sensor height.

[jolting]

Let me know if the 2D plots work. I'm excited to try them out.

[rachit173]

I have added address bar for specifying the websocket address and port, 2d plots works in map and path generation component and will soon work for individual rawlog (debugging it).

For the rawlog address, how do you feel about sending file from client to server using websocket (we can then use file dialog box). If we wish to go with directory, then should I create a specific directory from which the server will give the list of files available for visualisation from the directory?

3d animation currently uses trackballcontrols, for changing the pivot the press the right click and move the mouse. There are other controls out there, https://threejs.org/examples/?q=controls#misc_controls_map, but have their own pros and cons.

For 3d scan visualisation, are you aware of some rawlog with multiple sensor data. Then I would check how it functions with the RawLogViewer app and modify the app for multiple sources.

[jolting]

For the rawlog address, how do you feel about sending file from client to server using websocket (we can then use file dialog box). If we wish to go with directory, then should I create a specific directory from which the server will give the list of files available for visualisation from the directory?

Just specify a directory with a list of rawlog files. That should be sufficient.

3d animation currently uses trackballcontrols, for changing the pivot the press the right click and move the mouse. There are other controls out there, https://threejs.org/examples/?q=controls#misc_controls_map, but have their own pros and cons.

Cool thanks for clarifying.

For 3d scan visualisation, are you aware of some rawlog with multiple sensor data. Then I would check how it functions with the RawLogViewer app and modify the app for multiple sources.

It's incredibly useful for a 2.5 D representation from depth cameras or the uncommon case of using multiple lidar scanners at different heights.

Sometimes the map looks different at various heights. https://www.youtube.com/watch?v=jCFvAOuV_H8

[rachit173]

For building mrpt-web, serialization file CSchemeArchiveBase is required from mrpt library. How should I prepare it for merging with MRPT ? This would be required for the video demonstration, or for now I could make the video using mrpt build from my own fork. These are the works I remember are pending :

• I am currently facing this issue for making the main 2d plots in rawlog viewer app.
• The video is still remaining, I have started work on it.
• Adding extended examples to js library.
• Rawlog loading from directory on server (only a small problem to be solved)

I wanted to finish off work by August 6. I have interviews on campus from 7 - 12 August, it would be very generous if you could help me in doing so. I would like to continue adding more features and working on the project in general after GSoC.

Also GSoC, requires a final submission, in the form of a link. Since this project spans across repositories, can you give an idea of how I should submit the link showing the details of work done. Should I setup a github page?

[jlblancoc]

Hi, On this:

Instructions to launch the RawlogViewer demo app & a link to the most relevant parts of the code?

Did you write some instructions on the ESdoc manual pages? Something really short would be enough... thanks!

On the RawLogViewer issue, I'll check my Eigen version just in case it's newer than yours (now I'm on Windows)...

[y73isc00l]

Yes will push it by today for launching rawalog app I am making a video.

On Thu, Aug 2, 2018, 13:10 Jose Luis Blanco-Claraco < notifications@github.com> wrote:

Hi, On this:

Instructions to launch the RawlogViewer demo app & a link to the most relevant parts of the code?

Did you write some instructions on the ESdoc manual pages? Something really short would be enough... thanks!

On the RawLogViewer issue, I'll check my Eigen version just in case it's newer than yours (now I'm on Windows)...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MRPT/GSoC2018-discussions/issues/6#issuecomment-409836220, or mute the thread https://github.com/notifications/unsubscribe-auth/AXfDdDTqJXEDMYpoTfiHm5FoB11eiVHBks5uMqz6gaJpZM4Tguu2 .

[rachit173]

Yes will push it by today (manual pages) , for launching rawalog app I am making a video.

[jolting]

I've recently opened a few of the sample RawLogs in 18.04. You do have to be careful to compile against GTK3 and there are a few tricks you need to get OpenGL to work with wayland(if you use that by default), but besides that no issues.

AFAIK, RawLogViewer doesn't have an issue with the version of Eigen that ships with 18.04.

[rachit173]

Update:

• mrpt-web-js docs : added examples and brief descriptions in manual
• Created a simple video explaining launch and functionality of rawlog-viewer.(Sorry for my poor video editing skills, i will try to make a more fun video later)
• Started work on my final submission link.

Any pointers regarding final submission link, like what more verticals should be added, best practices would be great :)

[jlblancoc]

Great video!! :+1:

So, apparently you finally solved the crash problem in your rawlog-viewer app, right? It doesn't seem to crash on our tests...

[rachit173]

I havent been able to, will try on ubuntu 16.04

[jolting]

Good Job! I'm wondering if there is someplace we can host this as a demo. The javascript part can go on surge.

The websocket server could be containerized.

[jlblancoc]

Good Job! I'm wondering if there is someplace we can host this as a demo. The javascript part can go on surge.

The websocket server could be containerized.

Sure, we could host it on mrpt.org in a new subdomain. Just let me know what do you need exactly, and as far as it's safe we'll do it.

Consider doing it with a front-page which may be the entrypoint to a number of web demos, even if for now we only have the rawlog-viewer app...

Cheers!

[jolting]

I've cloned all the wiki pages for the following repos. https://github.com/MRPT/mrpt-web https://github.com/MRPT/mrpt-web-js https://github.com/MRPT/rawlog-web-ui

Furthermore, I fixed the unit test for mrpt-web-js. This now uses jest. That was fairly easy to setup. Also CircleCI now auto deploys to @mrpt/mrpt-web package on npmjs.org. That should make using the package a little easier.

[rachit173]

Thanks @jolting, I will make the report today and then can continue to work on bugs and issues, after submitting report today.

[jlblancoc]

@rachit173 : I just gave you write permissions to all related @MRPT repositories, so you can directly contribute there more easily. You may (if you want) use those final URLs in your final report; your contributions are clearly identified there even if that's not your fork, as you know.

A proposal @jolting , @rachit173 : would it make sense to add TravisCI / CircleCI to https://github.com/MRPT/mrpt-web as well?

[jolting]

yes

[jlblancoc]

I just enabled travis on that repo.

[rachit173]

Hi, I am back, thanks for your patience and support. I have submitted report, please have a look at it. https://rachit173.github.io/gsoc2018/

[rachit173]

Thanks for the evaluation and opportunity to work with the organization. I wish to work on the existing project as well as MRPT in general.