search

autowarefoundation / autoware_ai

Apache License 2.0
19 stars 7 forks source link

[Feature] Autoware Maps: Next Generation Autoware Map Format #408

Closed messi49 closed 5 years ago

messi49 commented 5 years ago

Design

We created the next generation map format for Autoware (called Autoware Maps Format).

Goal

Notification

Please see this, then let's start to discuss. https://docs.google.com/presentation/d/12vZzMfVLAjeuGSS7qykpzDjbOagJBBQJ7D33c-QtJPI/edit?usp=sharing

hakuturu583 commented 5 years ago

@messi49 I saw your documentation and found that there is a lane height field in the waypoint. Is it the altitude of the target lane??

hatem-darweesh commented 5 years ago

@messi49 I am interested in this new feature, but I have some questions first.

messi49 commented 5 years ago

@hakuturu583 A lane height field in the waypoint means height limit.

messi49 commented 5 years ago

@hatem-darweesh Thank you for comment.

Is this a personal effort , or request from Autoware team ?

Request from Autoware and AMS team is.

Should it replace the copy-righted vector map format that currently supported by Autoware ?

Autoware will support both vector map format of Aisan Technology and Autoware Maps Format for the time being. In the future, Autoware Maps Format will mainly be supported.

what is the architecture of this new design, specially interoperability with current Autoware framework ? related to previous question, what is the relation between this format and the map-server ?

Could you be more specific?

will data be local with autoware or shared by some kind of connection.

Map data is used at local. In additionaly, there is plan that Autoware maps service that will be released from Tier IV.

if the map will be saved locally , what is the data storage format (DB for example sqlite , or .csv files for example current moriyama map, or single file .kml like the one OpenPlanner uses) ?

CSV file format is.

For already implemented Data structure check RoadNetwork class in https://github.com/CPFL/Autoware/blob/master/ros/src/computing/planning/common/lib/openplanner/op_planner/include/op_planner/RoadNetwork.h

Let me confirm

inejose commented 5 years ago

@messi49 regarding to the goal:

Change PCD format plane rectangular coordinate to MGRS(UTM) in order to support global.

we are stuck in this problem since some months, we would like to use Autoware in Europe with the appropriate UTM conversion but the methods and libraries suggested in autowarefoundation/autoware_ai#79 autowarefoundation/autoware_ai#309 autowarefoundation/autoware_ai#880 and many others seem not to work.

Please, could you provide some indications or code extract about this point before the release? We really appreciate it.

JWhitleyWork commented 5 years ago

Is there a reason that a new format was chosen instead of an existing, open format such as OpenDRIVE (which is very popular in the autonomous vehicle space) or OSM XML or a CSV representation of an existing open format?

I would also argue that what you're representing is essentially a relational database and something like PostGIS + PostgreSQL (which are both open-source) would be more portable and easily accessed than a group of CSV files. SpatiaLite is another alternative.

messi49 commented 5 years ago

@en-sevilla Thank you for your comments. I understand this matter. Due to this map format change, we will prepare the map tools of coordinate transformation and so on. I will notify you as soon as is is ready.

messi49 commented 5 years ago

@JWhitleyAStuff Thank you for pointing it out.

Is there a reason that a new format was chosen instead of an existing, open format such as OpenDRIVE (which is very popular in the autonomous vehicle space) or OSM XML or a CSV representation of an existing open format?

Other existing open format are too rich for Autoware. New Autoware maps format is very simple and cost of creating map is low. In addition, we will prepare the map converter (OpenDrive → Autoware Maps Format) in the future.

I would also argue that what you're representing is essentially a relational database and something like PostGIS + PostgreSQL (which are both open-source) would be more portable and easily accessed than a group of CSV files. SpatiaLite is another alternative.

Yes, data architecture of Autoware maps format is essentially a relational database. You can easily insert database with importing map data of Autoware maps format.

JWhitleyWork commented 5 years ago

@messi49 - some questions I have about the format:

Even though you say that other open formats are too rich for Autoware, they already have these types of rules defined and can be as simple as an XML file. It isn't necessary to use all of the features of the existing formats - you can choose to only support certain object types in those formats. I understand the want to make the Autoware Maps format simple, but in making the format simple, you either leave a lot of ambiguity (which is not good in safety-critical applications like Autonomous Vehicles) or you have to have many rules about what each field means and how they can be used. In my opinion, the ability to include more detail in the format itself is better, even if you don't use all of the features in most use cases.

hadiTab commented 5 years ago

@messi49 I think the lane structure should include more than just the number of lanes and lane number. I would suggest also including the id of the "lane to the right" and "lane to the left" as well.

Otherwise, global planner would need to search for the adjacent lanes to find them, which increases the computational cost of planning a route and makes the global planner more complex. Similarly, the local planner would need to search for adjacent lanes, if it were to use them to avoid obstacles or overtake another vehicle instead of simply "knowing" it has another lane to its left or right.

shinpei0208 commented 5 years ago

Joshua,

The Autoware Maps format is used internally in Autoware so as to improve computation time and keep source code as simple as possible. This format is compatible with most of other existing formats, such as Open Drive. That is, you can use Open Drive maps with Autoware, though internally they are converted to the Autoware Maps format in order to maximize the real-time performance of road feature searches.

Shinpei

2018年11月13日(火) 7:25、Joshua Whitley さん（notifications@github.com）のメッセージ:

@messi49 https://github.com/messi49 - some questions I have about the format:

  • What is the difference between Waypoint::width and Lane::width_limit?
  • Why would the waypoint define a width - isn't that only associated with a lane?
  • What is the difference between Waypoint::height and Lane::height_limit?
  • What is the definition of Waypoint::yaw angle?
  • Is Waypoint::velocity the same as Speed Limit?
  • Is an Area always within a single lane? Can it cover multiple lanes? Can it exist outside of lanes?
  • Will a lane always have exactly one start and one end waypoint?
  • If a road is too narrow for two cars to pass by each other (very common), will two lanes be defined with one waypoint being the start of one and the end of another and vice versa?
  • Will Lanes always be defined by a set of Waypoints or can Waypoints be outside of lanes?
  • What is the definition of Wayarea and how is it different than Area?

Even though you say that other open formats are too rich for Autoware, they already have these types of rules defined and can be as simple as an XML file. It isn't necessary to use all of the features of the existing formats - you can choose to only support certain object types in those formats. I understand the want to make the Autoware Maps format simple, but in making the format simple, you either leave a lot of ambiguity (which is not good in safety-critical applications like Autonomous Vehicles) or you have to have many rules about what each field means and how they can be used. In my opinion, the ability to include more detail in the format itself is better, even if you don't use all of the features in most use cases.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/CPFL/Autoware/issues/1701#issuecomment-438043868, or mute the thread https://github.com/notifications/unsubscribe-auth/AAToCzSEpTjtffgENx-7SkiXbxw7AIWQks5uue_ugaJpZM4YSVLb .

dejanpan commented 5 years ago

@shinpei0208 @messi49 first off thanks for your proposal. I think that it is great that we are moving away from the Aisan's map format.

However I entirely agree with @JWhitleyAStuff that now creating yet another custom format is the worst possible solution to go for. Even Apollo at least took OpenDrive as a base and extended it: https://github.com/ApolloAuto/apollo/issues/603.

From the top of my head I see the following features missing in your proposal:

  1. Roundabouts
  2. 4 way stop intersections
  3. Definition of local, global and inertial coordinate system
  4. You are missing important lane types (e.g. construction site, bike lane, ..)
  5. ...

So I suggest the following: Please start over with this issue and first list what the shortcomings of OpenDrive format are for Autoware. I do not see any.

In addition many map makers (e.g. Atlatec, Carmera, ...) already support OpenDrive and they will seamlessly integrate with Autoware if we support OpenDrive.

Also, @messi49 - Josh had a bunch of really good technical questions above. Please address them.

mitsudome-r commented 5 years ago

@JWhitleyAStuff @dejanpan
I think what we are trying to do here is to create a map format that is consistent with what Autoware can do.

For example, current Autoware cannot handle 4 way stop intersections, bike lane, or roundabouts properly, and it could cause an unnecessary confusion or misunderstanding of Autoware's capability to users if there is a field in map format to describe such attributes. This can be avoided if we create a map format that is not insufficient nor over-informed for Autoware, and then add related features to the format as Autoware's capability is improved. This way we can keep the final input to Autoware simple and suited for the system as @shimpei0208 suggested.

Therefore, I think creating a new map format adapted to Autoware is not such a bad idea, though I doubt if we have to design it from scratch.

JWhitleyWork commented 5 years ago

@mitsudome-r I'm not completely opposed to a simpler format that is Autoware-specific. However, since this essentially defines the "safe areas" for an autonomous vehicle to travel, it needs to be pretty explicit in how its parts are defined - much clearer definitions than what are provided in the above document - hence my questions. One misunderstanding from an unclear definition could lead to a loss of life.

This is why I recommended using just a subset of the available features of an existing format. OpenDRIVE and OSM XML are already clearly defined. Using only the features that are currently supported by Autoware in an existing format makes it somewhat future-proof and easily expandable. Even extending an existing format like Apollo would give you these benefits. Using your own format and building it piece-by-piece seems like a lot of duplicated effort and will require significantly longer than the currently-provided schedule (completion by mid-December) to get it right.

If the intended feature set of Autoware will only include following a set path and not doing any routing of it's own, then the input format isn't really a "map" (in the road network sense), but more of a "path." If you're intending to define your own "path-following" format, that's a whole different scenario. In that case, I would say the format could be trimmed down even further. However, if you want Autoware to do it's own dynamic routing, you're going to need something pretty detailed.

messi49 commented 5 years ago

@JWhitleyAStuff I respond to your questions with the following.

What is the difference between Waypoint::width and Lane::width_limit?

The Waypoint::width mean lane width at the point of waypoint. The Lane::width_limit is the minimum value of waypoint of lane.

Why would the waypoint define a width - isn't that only associated with a lane?

Lane area is defined by using the waypoint width. Currently, Autoware use wayarea for detection.

What is the difference between Waypoint::height and Lane::height_limit?

This is same relation between Waypoint::width and Lane::width_limit. The Waypoint::height mean height limit at the point of waypoint. The Lane::height_limit is the minimum value of waypoint of lane. However, I think that the waypoint need not to have value of height limit with respect to height limit.

What is the definition of Waypoint::yaw angle?

The waypoint::yaw mean the direction of vehicle driving.

Is Waypoint::velocity the same as Speed Limit?

Yes. Although the waypoint::velocity at corner is not always same.

Is an Area always within a single lane? Can it cover multiple lanes? Can it exist outside of lanes?

An area can be referred to from multiple lanes. It can exist outside of lanes. A relation between lane and area is flexible.

Will a lane always have exactly one start and one end waypoint?

Yes. ※The waypoint of junction is referred to from multiple lanes.

If a road is too narrow for two cars to pass by each other (very common), will two lanes be defined with one waypoint being the start of one and the end of another and vice versa?

Yes. There are another waypoints in each of the lane orientation.

Will Lanes always be defined by a set of Waypoints or can Waypoints be outside of lanes?

The Lanes will always be defined by a set of Waypoints.

What is the definition of Wayarea and how is it different than Area?

The wayarea is detection area of Autoware. ※The currently version Autoware use wayarea. However, we want Autoware to use waypoint width and area.



@dejanpan Thank you for comment. The Lane attribute such as 4 way stop intersections can easily be added.

New Autoware maps format like intermediate representation of autoware used internally. So, Autoware can support multiple map format by preparing the converter of OpenDrive, TomTom, Aisan.

alexandrx commented 5 years ago

Sorry to join late. I've been pushing comments at the corresponding Slack channel (but without ears), and just got pointed to this issue.

There are several researchers/companies currently using Autoware or considering. ADAS maps dependency is the weak point as they are not available. There are several research work to automatically create HD maps for autonomous vehicles (for free) and other crowd-sourcing map solutions at low cost/high quality. Several companies I've met are willing to integrate with Autoware and the all-time question is "what is the format Autoware uses?"

We need to support existing and future formats, and the Autoware Maps Format is a good idea. But I think it should be a super-set of existing world renown map specifications such as OpenDrive (mentioned previously), otherwise maps won't be usable in other contexts.

Anyway, here is yet another approach, presented at ITSC2018 (last week): Lanelet2 Lanelet2 is released under BSD 3-clause and this is the github repository: https://github.com/fzi-forschungszentrum-informatik/lanelet2 Includes some examples (as lessons) how to use lanelet2. It seems maps can be edited with the openstreetmap editor (according to is predecessor project, https://github.com/phbender/liblanelet) Here is a video of somebody using it: https://www.youtube.com/watch?v=svnchV618FE

And last but not least, here is the conference paper: http://www.mrt.kit.edu/z/publ/download/2018/Poggenhans2018Lanelet2.pdf

alexandrx commented 5 years ago

OpenDrive and Lanelets have got a lot of attention in the research community. The project CommonRoad! (http://commonroad.in.tum.de/) uses OpenDrive and Lanelets. CommonRoad! allows to benchmark motion planners on diverse roads and vehicles conditions (I already suggested it to the OpenPlanner author @hatem-darweesh ) Here is the Gitlab repository: https://gitlab.com/commonroad/commonroad.gitlab.io/tree/master And here is the 2018 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI) conference paper: https://mediatum.ub.tum.de/doc/1449005/299979664679.pdf and the code to convert OpenDrive to Lanelet: https://gitlab.com/commonroad/commonroad.gitlab.io/tree/master/tools/opendrive2lanelet

shinpei0208 commented 5 years ago

Guys, thanks for all the discussions and useful feedback here. To make it clear, we will support those well-known map formats and Open Drive would be a default format. They are frontend formats. The Autoware Maps, on the other hand, is a backend map engine and thus its format is a backend format. It could be converted to as many frontend maps, i.e., Open Drive etc., so that as many map users as possible can use Autoware.

2018年11月15日(木) 13:19、Alexander Carballo さん（notifications@github.com）のメッセージ:

OpenDrive and Lanelets have got a lot of attention in the research community. The project CommonRoad! (http://commonroad.in.tum.de/) uses OpenDrive and Lanelets. CommonRoad! allows to benchmark motion planners on diverse roads and vehicles conditions (I already suggested it to the OpenPlanner author @hatem-darweesh https://github.com/hatem-darweesh ) Here is the Gitlab repository: https://gitlab.com/commonroad/commonroad.gitlab.io/tree/master And here is the 2018 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI) conference paper: https://mediatum.ub.tum.de/doc/1449005/299979664679.pdf

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CPFL/Autoware/issues/1701#issuecomment-438912049, or mute the thread https://github.com/notifications/unsubscribe-auth/AAToC8ZF5luwXi-f4tmjybjGsyk3EjU6ks5uvOtOgaJpZM4YSVLb .

dejanpan commented 5 years ago

@shinpei0208 this was definitely not clear from the issue title and description. But it brings up a good point in that we should design architecture of a mapping module for Autoware first. Since I believe you guys plan to do implement a mapping module on your own, right?

Once you do that it will become clearer what actually the scope of this issue is and also whether the internal representation of your mapping module should be closer to the data structures defined by OpenDrive or not.

dejanpan commented 5 years ago

https://commonroad.in.tum.de/

dejanpan commented 5 years ago

@alexandrx above 2 comments are super useful and informative. Thank you very much.

@messi49 and @shinpei0208 read this paper carefully: http://www.mrt.kit.edu/z/publ/download/2018/Poggenhans2018Lanelet2.pdf.

It tells you that:

  1. if you are building maps for AD, you need to understand what is your map used for, e.g.: image

  2. you need to define your basic data structures for backend, e.g. Points and line strings belonging to the physical layer and lanelets, areas and regulatory elements belonging to the relational layer

  3. you need a design document for your mapping framework with the modules, e.g.

    1. Core: The core module contains the basic primitives and layers described above. In addition, geometry functionalities are included to calculate center lines, distances and overlaps, for example
    2. Traffic Rules: This serves to interpret the rules contained in the map depending on the type of road user and country. It also determines whether lane changes are possible or whether a certain road user is permitted to enter a lanelet.
    3. Physical: This module allows direct, comfortable access to the elements of the physical layer.
    4. Routing: With the help of traffic rules, routing graphs can be set up to determine the exact route to be driven, including possible lane changes, or to predict possible routes and points of conflict for other road users. It is also possible to construct maneuverable zones by combining adjacent areas and lanelets.
    5. Matching: This module is used to assign lanelets to road users or to determine possible positions on the map based on specific observations of the sensors.
    6. Projection: Contains functionality to convert global latitude/longitude coordinates to local, metric coordinates.
    7. IO: The input/output module contains functions for reading and writing maps from various map formats, in particular the OSM format.
    8. Validity: Module that searches and reports typical mapping errors in a map.
    9. ROS: A connection to the Robot Operating System (ROS) which is often used in demonstrators.

Point 3-vii talks about the external map interface (e.g. OpenDrive, OSM).

@messi49 @shinpei0208 in the light of above explanation, which of the above points is this issue about?

Looking at this slide https://docs.google.com/presentation/d/12vZzMfVLAjeuGSS7qykpzDjbOagJBBQJ7D33c-QtJPI/edit#slide=id.g46f2754f85_25_0, I still think that it is about 3-vii. If this is the case, then I suggest that we do NOT use "Autoware Maps Format" and use OpenDrive instead. If another map provider then wants to integrate into Autoware they should write a converter from their format to OpenDrive rather than from their format to "Autoware Maps Format".

alexandrx commented 5 years ago

@dejanpan I agree. OpenDrive exists for longer time than any of the others and is used by so many companies.

alexandrx commented 5 years ago

We can consider the implementation by Institute of Transportation Systems of the German Aerospace Center (Deutsche Zentrum für Luft- und Raumfahrt - DLR, http://www.dlr.de/ts/en), available at github:

https://github.com/DLR-TS/xodr

It uses XSD for code generation based on OpenDrive schemas. Need to make sure XSD license works for us.

messi49 commented 5 years ago

We decided to use Open Drive Format. ※Autoware Maps Format is data structure of internal process of Autoware.

image

dejanpan commented 5 years ago

@messi49 @shinpei0208 @JWhitleyAStuff @alexandrx @hadiTab

I am a complete mapping noob but I took the time, did some reading and talked to some people that have done AD maps before. Here are my current findings:

In general an AD map consists of:

  1. Data in exchange format => NDS*, gps, PointCloud, ...
  2. Library to access above data + IO API
  3. Map model
  4. Data compiler => compiles the data from raw data in exchange format into a binary and faster loadable version.
  5. Tools (e.g. visualizer)
  6. Interface format The most important above is the map model, interface format is secondary.

Map model is often built around the limited set of use cases but then people quickly start to run into issues with it. This is what happened before in Autoware and will happen again with above proposal.

Examples of map models:

  1. Lanelet (or Lanelet2) is very good. No known drawbacks.

  2. OpenDrive is meant for simulation. What is missing are topology, areas, conditions, AGV lanes, missing modeling relationships (traffic sign - lanes), …

Commercial options:

  1. Here is offering a library and API, they have a model for AD driving. They also offer data in the compiled format. They have a very good documentation. However the they do not have a point cloud data.

  2. Deepmap, they offer a black box solution (mapping and localization) and also API for maps.

* NDS - data definition language, also provides reference implementation for library and api. It is more used for navigation systems. It basically defines the PSF (physical storage format) for the map data.

@messi49 @shinpei0208 @JWhitleyAStuff @alexandrx @hadiTab can you confirm or deny my comprehension above?

To me it now seems that we have 3 options:

  1. OpenDrive + https://github.com/DLR-TS/xodr. With this solution it seems that OpenDrive model lacks important features and I do not know how it includes data in exchange format and what tools are available for debugging/visualization.
  2. We go with Here. But them not having PointClouds in the map is probably a no-go.
  3. We go with Lanelet2. This to me is currently the best option.
sgvandijk commented 5 years ago

I am not a mapping expert either, but just wanted to add to @dejanpan's findings based on my own:

[1] https://vires.com/download/OpenDRIVE/OdrMgrUserManualRevL.pdf [2] http://www.opendrive.org/docs/OpenDRIVEFormatSpecRev1.4H.pdf

JWhitleyWork commented 5 years ago

@dejanpan @sgvandijk - I have just completed an element-by-element comparison of OpenDRIVE with another format so I'm somewhat familiar with its capabilities and shortcomings. There are specific objects which are not well-defined for AD scenarios in OpenDrive. Those include active traffic signals (stop lights or turn lights) and their sub-components and orientations, LED signs and their text and associated traffic rules, the actual location at which a road sign or signal takes effect on the road or lane, and many of the ground markings on lanes and their associated traffic rules.

I am not yet familiar enough with lanelets2 (CommonRoad) to be able to comment as to it's coverage of these and other general AD scenarios. However, the shortcomings of OpenDRIVE for this specific use do worry me as to using the OpenDRIVE standard in it's current form as the AD base map (@dejanpan 's item 3).

On the plus side, item 1 in his post is covered by the XML specification (XSD) provided by OpenDRIVE. Item 2 is covered by any application/API which can load an XSD and associated XML document (see PyXB as an example in Python and XODR as mentioned above for C++). Item 4 would be specific to the application - in this case AutoWare - and would have to be developed for any mapping format that Tier IV chooses. Item 5 is covered by tools provided by Vires and third parties, as mentioned by @sgvandijk.

As another aside, lanelets2 seems to provide all of these same items, but, again, I'm not familiar enough with the actual specification to comment on it's feasibility for use in this application.

JWhitleyWork commented 5 years ago

I've reviewed the lanelet2 specification to some extent and I believe it covers all the necessary primitives that we are looking for. However, it has a significant shortcoming in the storage format. I created the issue fzi-forschungszentrum-informatik/Lanelet2#3 to ask about this drawback. An additional, but, IMO, not as critical drawback is that it is fairly new and untested (development only began on GitHub a month ago).

One additional note: One of the reasons that the lanelet2 specification would work for this application is that it is very vague and implementation-defined.

JWhitleyWork commented 5 years ago

Worth noting: According to the OpenDRIVE.org website, the organization ASAM has taken over administrative management of the OpenDRIVE standard. I am attempting to contact them about adding additional XML elements to represent the missing information necessary for the driving task.

I have yet to find a format which contains descriptions of all data required for the automated driving task in general. I have so far reviewed OpenDRIVE, RoadXML, Apollo OpenDRIVE, OSM XML (which is the lanelet2 storage format), ArcGIS shape files, and a couple others. However, OpenDRIVE and RoadXML seem to contain the vast majority of the required elements.

dejanpan commented 5 years ago

@JWhitleyAStuff @sgvandijk @alexandrx @hadiTab @messi49 @shinpei0208 sorry for late reply.

I have pinged 3 mapping experts and here is what I have learnt in addition.

Lanelet2 (conversation with the creator Fabian Poggenhans)

  1. It is a 2nd shot (after original Lanelet) and put out there to wait for the feedback from the users
  2. Their IO interface is based on open street map (OSM)
  3. Geo-referenced large data (images, pointclouds) are not included in the map
  4. They have functions to project between the lat/lon geographic world and local metric coordinates (currently implemented WGS 84)
  5. you can use JOSM to visualize and modify maps
  6. They do not use tiling and do not use data compiler (point 4 above)
  7. Visualization in RVIZ => working on it
  8. Most supported use case: Navigation, planning the route on the lane level, prediction of other participants
  9. Currently not supported use case: no LiDAR-based localization
  10. Otherwise Lanelet2 checks all other boxes mentioned here: https://github.com/CPFL/Autoware/issues/1701#issuecomment-440518903
  11. Algorithm developers are supposed to use their data structures and IO modules in their algorithms.

I also asked Fabian about the map model proposed by @messi49 and this was his reply:

The approach of tagging all the information to waypoints reminds me a bit of the Opendrive approach. Our experience with this is, that such an approach generalizes very badly to real scenarios. Over the time more and more specific information is required and more and more tags must be added (for other participants, for different traffic signs, other countries, etc). In the end, the number of tags gets so complex that it is really hard to “debug” or understand the map data. This is why Lanelet2 uses a more “physically” motivated approach to tagging.

Also the representation of the lane width makes it very hard to get the original shape of the lane back. This is often important when you want to predict if other participants interfere with the current trajectory. It gets even harder when the road elevation is considered as well. Also it is harder to find changes in the map if the original lane boundaries are lost.

They would be willing to accommodate changes and possibly we could even contract them from the Autoware foundation.

OpenDrive

OpenDrive is in the process of becoming an industry standard and there are at least two companies out there which can produce OpenDrive maps. There is a bunch of software which can import (parts of) OpenDrive.

From a technical view, OpenDrive is strongly tied to the idea of reference lines. Each road has a reference line which is an axis of a curved coordinate system. Road borders are described in this coordinate system (as splines describing the lateral offset from reference line). This works nicely in many situations and provides smooth (mathematically differentiable) descriptions of laneborders. This is good, two, especially for simulations. However, it has drawbacks. For some real situations, it is difficult to describe lane borders in terms of that curved coordinate system.

Another weakness is that the standard is sometimes too generic. It says that there can be objects and how these can be described, but it does not define a standard on how to model certain objects like Guardrails, Crosswalks etc.

But maybe the largest drawback is in describing traffic rules. You can place objects like holding lines in the model, but you cannot clearly describe what the car has to wait for on that line (is it a traffic light, is it a yield situation, is it a crosswalk?)

This above is basically in line with what @JWhitleyAStuff has stated above too.

Conclusion

From everything above I would suggest that we pick Lanelet2 as a library for handling map data in Autoware (including its map model and APIs). I see following advantages compare to @messi49 proposal and OpenDrive:

  1. It is open-source and very well thought through
  2. It is quite proven (based on Lanelet)
  3. It is flexible and extensible
  4. It comes in as a full package
  5. It moves away from the idea of reference lines to a more physically feasible approach

I agree with @JWhitleyAStuff that the project is young but

  1. it is tested and under CI: https://github.com/fzi-forschungszentrum-informatik/Lanelet2/blob/master/.gitlab-ci.yml
  2. very nicely written and documented, e.g. https://github.com/fzi-forschungszentrum-informatik/Lanelet2/blob/master/lanelet2_core/include/lanelet2_core/geometry/Area.h

It would be great if we could agree on this by the end of this week.

dejanpan commented 5 years ago

@JWhitleyAStuff I've only seen this comment https://github.com/CPFL/Autoware/issues/1701#issuecomment-442615933 after I posted my comment above.

In addition to missing XML elements in OpenDrive, there seems to also be a problem that the approach is fundamentally flawed in that it is tied to the reference lanes.

What does Apollo OpenDRIVE do wrt this? Do we know anybody that is successfully using OpenDrive on either mapping or algorithms side?

JWhitleyWork commented 5 years ago

@dejanpan wrt Apollo's use of OpenDRIVE, I can't provide too much detail (they are being somewhat closed about their format) but I can say that it is fundamentally a different base protocol. They describe their implementation as "based on OpenDRIVE" but, after studying it, I would say it is more of a "rewrite of OpenDRIVE." It gets around the fundamental problem with using OpenDRIVE as a basis for real-world driving by decoupling the road boundary definitions from the center spline definition.

I wouldn't say this problem for our use of OpenDRIVE makes it a fundamentally flawed standard, just that some companies are repurposing the standard for a use case for which it was not initially designed (real-world autonomous driving versus simulation).

I've been thinking a lot about this lately and I feel like the evolution of OpenDRIVE has been a path of patching in "real-world" features to a standard intended only to describe an ideal, virtual world. This is part of where I think RoadXML gets it right. They define individual layers for the "real-world," physical representation of roads, signs, and objects, then they have a seperate, "semantic" layer to describe regulatory concepts which overlap that physical space like speed limits, construction zones, and stop points.

When smashing together the conceptual, regulatory space with the physical representation of that space into a single, flat standard, I feel like some information loss will necessarily be suffered to gain the benefit of computational simplicity. However, I feel like it is the job of the "data compiler" to reduce the complexity of the underlying map data to a level that is usable in real-time.

If Lanelet2 supports this type of seperation of concerns or has found an elegant way to handle this problem that I can't immediately conceive of in the mapping format as it is contained in memory, I'm all for using it. We can always figure out a good transmission format in the long run. If they don't have it figured out, I recommend taking a quick look at RoadXML.

alexandrx commented 5 years ago

@dejanpan As to your question Do we know anybody that is successfully using OpenDrive on either mapping or algorithms side? Although it is part of the discussion of OpenDrive for virtual-world rather than real-world maps, for the sake of completeness it's worth mentioning that CARLA also uses OpenDrive for their maps. https://github.com/carla-simulator/carla/tree/master/LibCarla/source/carla/opendrive

dejanpan commented 5 years ago

@JWhitleyAStuff if you see their paper, p6, Lanelet2 has these separation of concerns. For how much of the architecture is actually implemented in the code and how fast and memory hungry it is, I have asked the Lanelet2 author to comment.

Regarding https://www.road-xml.org/index.php. It seems a little bit dead to me. Have you been in touch with someone from the project lately? Also it seems to me that they only figured out the map model. Do they any other items from my list above?

poggenhans commented 5 years ago

Separating the physical information, its semantic meaning and the interpretation of the semantic is the whole point of the lanelet2 structure.

It uses what you could consider as a "bottom-up" approach, while OpenDrive is basically a top-down approach: Lane boundaries (Linestrings) are physical things with a real-world meaning. Based on their type, lanelet2 reasons for example where lane changes are possible. Lanelets connect these boundaries to form lane sections, Regulatory elements connect lanelets with driving restrictions, which are itself based on physical things e.g. like a traffic light. This explains that in more detail.

From this representation, the actual routing information for each type of traffic participant is generated dynamically from the map data and "compiled" to a routing graph that can be then used for routing queries.

Concerning the memory/computation impact: In the implementation, every primitive does not actually hold the data that it uses, in only holds pointers to it. A lanelet holds pointers to the boundary instances, the boundaries hold pointers to the individual point instances. Since adjacent lanelets can share boundaries and boundaries can share points, this actually reduces the memory usage compared to a naive implementation where everything has its own copy. On the other hand you have the downside of a sub-optimal cache locality because the data is more scattered in memory. But it is also the only way to be able to consistently modify map data. If one single point is updated, all boundaries that use it can observe the change.

dejanpan commented 5 years ago

@JWhitleyAStuff what do you think? @poggenhans thx.

JWhitleyWork commented 5 years ago

@dejanpan I still think there is a more efficient storage mechanism than OSM but that is certainly something I can live with. I also like that they seperate the in-memory representation of the map from the storage format. Lanelet2 gets my stamp of approval.

dejanpan commented 5 years ago

@JWhitleyAStuff @sgvandijk @alexandrx @hadiTab @messi49 @shinpei0208 should we organize a short online meeting to conclude on this and create the tickets going forward?

hadiTab commented 5 years ago

@dejanpan I have limited practical experience with maps and most of it has been with Apollo's HD map and Autoware's Vector Maps. I read the paper for Lanelet2 and I do like their approach to taking real-world physical elements and building on top of that. I also like the way the layered approach handles overlaps where points or lines would belong to multiple elements. I am not a huge fan of the reference-line approach often taken in other formats as well. Based on these factors and the discussion here I also see Lanelet2 as the best option presented here.

However, the map format is somewhat central to Autoware and affects many aspects of routing, motion planning, decision making, and so on. Any significant change to the map format would require heavy modifications or even rewriting of these modules. My main concern is whether or not the Autoware team intends or is prepared to undertake such an effort. I do think that it can significantly improve Autoware; however, based on the map format originally proposed in this issue I got the impression that the goal may have been to minimize the impact on other modules. Can someone from the Autoware team clarify this? @messi49 @shinpei0208

dejanpan commented 5 years ago

@hadiTab Autoware will in a matter of weeks become part of the Autoware foundation which will afford to hire software developers for such porting tasks.

based on the map format originally proposed in this issue I got the impression that the goal may have been to minimize the impact on other modules

@shinpei0208 @messi49 @alexandrx please comment on the above but I did not understand it this way. The plan was in my POV to move away from Aisan's format to here proposed that is again made from scratch: https://docs.google.com/presentation/d/12vZzMfVLAjeuGSS7qykpzDjbOagJBBQJ7D33c-QtJPI/edit#slide=id.g481d3617ed_1_19. It is maybe waypoint centric and as such similar to Aisan's one but it is not the same which will not minimize the impact.

hatem-darweesh commented 5 years ago

Dear All, sorry for my late reply.

I have worked with three different HD map approaches including AISAN technology one (used by Autoware). I used these maps for Planning.

  1. I tried my own approach which was using Open Street maps (OSM) Map Model , then converted to KML base file format for the ease of IO operations using XML. and finally link it to customized data structure, data structure was driven from the Planning Task itself.

osm approach

  1. Using DB based maps.

db approach

  1. AISAN Maps (current OpenPlanner implementation)

aisan-tech - op approach

Common Considerations:

Finally I modified the proposal document, concentrated on the Map Model needed for planning, maybe further modifications are needed for object filtering, ground removal and localization. https://docs.google.com/presentation/d/1PTO2gIAkyVlv3y2jt5VbJOd6DgAidqeRViuyO7GDAmg/edit?usp=sharing

dejanpan commented 5 years ago

@hatem-darweesh thanks for your feedback. I still did not understand why could we not use lanelet2 directly in Autoware for e.g. your use case of planning? Why do we need to create something from scratch and why can't we propose modifications to lanelet2?

Could you maybe come up with a table like this:

Features needed for planning Features covered by here proposed format Features covered by lanelet2
hatem-darweesh commented 5 years ago

@dejanpan , My main suggestion is to always have a middle layer so changes to map storage system and any node uses the map are minimal , and most of the changes will happen at this middle layer.

I like the idea of listing the features needed from maps, I will work on it , also we need to hear from other modules point of view. (following your proposed table)

@amc-nu @kitsukawa @yk-fujii @kfunaoka @s-azumi, Can you please list the Features that you need from the road network map (vector map) , I will try to compare them to lanelet2 and proposed format. for example:

msmcconnell commented 5 years ago

In terms of road models there is also the Navigation Data Standard (NDS) OpenLaneModel. http://www.openlanemodel.org/index.html#download It seems many auto manufacturers use NDS. However, the free OpenLaneModel is only for non-commercial use. It may not be enough of an exposed license for Autoware, but I am not a lawyer so I wanted to point it out just in case.

JWhitleyWork commented 5 years ago

Is there any update on the final decision of format or when it might be available in Autoware?

kfunaoka commented 5 years ago

This issue focuses on which format is the best. I've created a map architecture https://github.com/CPFL/Autoware/issues/1937. Now just a figure, need to dig down more to discuss in detail.

DmitryRedin commented 5 years ago

@hatem-darweesh, Hello, how to load KML file to OpenPlanner? Could you post example of kml file?

DmitryRedin commented 5 years ago

@hatem-darweesh, does OpenPlanner use wayarea.csv?

hatem-darweesh commented 5 years ago

Hi @DmitryRedin , currently OpenPlanner doesn't use wayarea, because most of our test maps doesn't have wayarea information. attached is sample kml for moriyama map.

moriyama_complete_2018.zip

icolwell commented 5 years ago

I'm not sure whether a decision was made, but I strongly recommend using lanelet2 as a core map representation. In a previous project we used the original lanelet model (with some small custom additions) with great success. It's easy to create maps with JOSM and they contained all the information we needed to do mission/motion planning, and object tracking/prediction.