FEATURE: Add 'GPS map' as plot window option

[MatinF]

Hi Daniel,

As we're soon launching our GPS-to-CAN module, we've had a large number of users ask for the possibility to plot Longitude/Latitude in the asammdf GUI to show routes - in combination with showing other signals.

I am unsure whether this would be a huge development effort, or if there is an opportunity to use an existing Python module for achieving this. Either way, I've tried providing some thoughts on the design of this below in case you think it could be a good fit.

Design outline

• The map would be selectable when a user is to plot a signal, e.g. so that the list of options becomes "Plot/Plot GPS Map/Numeric/Tabular/Bar"
• In order to create a valid map plot, the user would have to select two signals (Longitude and Latitude) at the same time when clicking the plot button, i.e. as in below screenshot

• To determine the type of signal (longitude vs. latitude), my suggestion would be to rely on a test of whether the signal names include long or lat (case insensitive) in the name, as I expect that should cover 90%+ of cases
• The map would plot the long/lat data as a GPS route, similar to what you see in our dashboard playground
• Similar to how your current plot windows "sync" the cursor, the GPS map would have a "dot" showing the timestamped position in sync with the cursor in other windows (e.g. speed, RPM, ...) - this way users can analyze the data for a specific position of a vehicle

Possible modules to use As inspiration, I found a number of Python modules that might be useful for this. I believe the simplest would be to rely on Open Street Maps (OSM) modules to avoid e.g. handling Google Maps API keys etc.

• Folium
• Open Street Maps Python modules overview

If you think this could be a good addition, I'm happy to help review further what could be useful modules for integrating this.

Martin

[MatinF]

Some additional articles for inspiration:

• https://thedatafrog.com/en/articles/show-data-google-map-python/
• https://ipython-books.github.io/147-creating-a-route-planner-for-a-road-network/

One alternative route might be to utilize functionality from Dash by plotly. It's designed for creating plots in the browser via Python, so it might not be possible to integrate into asammdf. Though if there's a way to e.g. create a 'browser' window inside the GUI then that might be possible.

[danielhrisca]

@MatinF I've added a GPS window in the development branch. It requires pyqtlet

[MatinF]

That sounds really cool! I tried updating, as well as installing PyQt5 and pyqtlet, but I get below error when trying to open the GUI:

Traceback (most recent call last):
  File "c:\users\marti\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\marti\appdata\local\programs\python\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\marti\AppData\Local\Programs\Python\Python37\Scripts\asammdf.exe\__main__.py", line 4, in <module>
  File "c:\users\marti\appdata\local\programs\python\python37\lib\site-packages\asammdf\gui\asammdfgui.py", line 5, in <module>
    from PyQt5 import QtWidgets
ImportError: cannot import name 'QtWidgets' from 'PyQt5' (unknown location)

Might be related to this: https://stackoverflow.com/questions/47164979/cannot-import-pyqt5-modules

[MatinF]

Turned out I needed this package as well:

pip install PyQtWebEngine

See also this thread: https://stackoverflow.com/questions/51154871/python-3-7-0-no-module-named-pyqt5-qtwebenginewidgets

[MatinF]

https://user-images.githubusercontent.com/26184427/118793638-15ca0400-b899-11eb-8ec6-682fb60cde5b.mp4

[MatinF]

This is really awesome Daniel (as always) - this will be a powerful tool for many of our users!

I've added some suggestions/inputs below:

1) When clicking the GPS trace view, there seems to be a visual issue with the drop down, see below image:

2) When clicking the dropdown, it looks like the list of signals is "duplicated" in the sense that there's a section first with indented signals named e.g. CAN2.XYZ.XYZ, which seemingly combine the channel and message name. I think this part should be removed so that only the actual signals are shown.

3) When I close the GPS plot and try to open it up again, it seems to be all white instead of showing the map. Restarting asammdf solves this. After closing the plot the first time, the problem then reoccurs and a restart is required. I think some sort of setting is not "reset" correctly when the plot is shut down.

4) On a more visual note, if it's possible to use Open Street Maps as an option rather than the current map, I think that would make for a cleaner map on which the data will be more readily visible.

5) When opening the plot initially, perhaps it's possible to "zoom in" more closely on the data so that the route being plotted takes up more space in the initial view.

6) When zooming in a lot the map disappears - I've not seen this happen with Open Street Maps, so maybe it's related to the specific map provider being used currently.

7) If possible, it would be cool to have Heading be an optional 3rd input for the map. Essentially, the default value for this would be "None" and the user could then just press OK. But if they select a heading from the list, the result would be that the heading would be plotted as the icon (so instead of a position icon, it would be an arrow icon or similar). However, I can't see if this is part of pyqtlet and if it is not, then I think this might be too much work for an initial feature version.

8) Moving the plot cursor nicely moves the cursor of the other plots as well - awesome job on this! :-) In my small log file (3 MB), the cursor seems to be a bit laggy on the map. I'm not sure if this is just the expected behavior or if it is e.g. related to there being many plot points on the map. If so, an advanced feature later on might be some light resampling or similar - but I think for now, I will try with larger data sets and see how it reacts.

Here is the sample data I used in the video above incl. DBC: sample-data-gps.zip

Again, I think this will be a really really cool feature for asammdf! Great work Daniel!

[danielhrisca]

this definitely need more polish

[MatinF]

I think it's already a really great start, I will be using this for e.g. troubleshooting and quick reviews of GPS data. I think the main points above would be 1, 2, 3 and maybe 4 - the rest is more in the 'nice to have / ideas' category :-)

[MatinF]

I did some more testing with this and I think it's a really great addition and would be awesome to include in the next release.

In regards to the initial dropdown, I think it could make sense to take the approach of including both the bus channel, message name and signal (as you do in the first part of the dropdown) - thus allowing users to e.g. plot data in cases where they have lat/lon signals on two separate CAN networks with similar names etc.

[danielhrisca]

I would like the latitude and longitude to be generic, so that it can used with any signals not just from bus logging

[MatinF]

Hi again Daniel,

I think it makes sense to be able to select the signals from all the available signals (as today). I'm not sure I understand how it would work without first decoding the log file to physical values (e.g. using a DBC)? But maybe I'm misunderstanding your thinking on this

[danielhrisca]

The signals could be recorded directly from the internal memory of an ECU using for example CCP or XCP

[danielhrisca]

3. When I close the GPS plot and try to open it up again, it seems to be all white instead of showing the map. Restarting asammdf solves this. After closing the plot the first time, the problem then reoccurs and a restart is required. I think some sort of setting is not "reset" correctly when the plot is shut down.

It looks like there is a bug in the pyqtlet library

[MatinF]

Ok - it looks like their project is not super well maintained. Perhaps a work-around would be if there's a way to force-reset the map in a similar way as to what is achieved when asammdf is restarted, i.e. something that has the same effect. If not, I still think the GPS map would be really useful even if this part would not be an easy thing to solve

[JulienGrv]

Hi together, I don't know about leaflet but did you look at alternatives like deck.gl? There is an official Python binding for it named pydeck. Maybe worth a look?

[MatinF]

Hi Daniel, just wanted to hear your thoughts on this - do you think the best next step would be to raise an issue on the pyqtlet github page, or do you think there would be a work-around on it?

Perhaps some of the other items (like the longitude/latitude dropdowns) could be adjusted and the GPS map could then be part of the next release in a 'beta' version. I think it would be a very useful tool even if there are a few kinks to it.

[danielhrisca]

This is not really working reliably at the moment and I also can't put enough time behind it. I also want to release 6.3.0 since it is long overdue.

I will disable this feature for now and hopefully it will be ready for a later release.

[MatinF]

Hi Daniel, just FYI wanted to let you know that we have an increasing number of asammdf GUI users that would love to have this feature :-) if you get the chance to give it a look again at some point that would be awesome - maybe including it as a "BETA" plot option in an upcoming release.

[MatinF]

This is now re-included in the latest development version and works nicely, thanks Daniel! Looking forward to the 7.0.0 release

[MatinF]

Thanks a ton for adding this Daniel!

One minor suggestion: If it is possible, it would be great if the plot would automatically be zoomed more in on the route displayed from the start. Currently, the map is zoomed out a lot - maybe there is a possibility to set the default view so that it contains the entire route +X% to each side of the map. Not sure if this is possible, but would make it quicker to get a view of the relevant data :-)

[skadd4life]

I would love to use this feature but I don't see it in the latest release or in the documentation. Did it make it?

[danielhrisca]

It only works with PySide6<=6.2.2

[skadd4life]

I see that I can make a GPS plot but none of my signals show up in the list. are there requirements for how latitude and longitude should be formatted? I have UTM x and y data that I could convert to lat/lon.

If I can't get the map to work, it would be cool to be able to plot two signals and specify the x axis as something other than timestamp. I remember in INCA MDA, I can plot two signals and specify an x and y axis - that would be sufficient for my use case, I could plot my UTM x and y and show a trail, not specifically over a map, and that would be good enough.