Supporting localization for new land areas via Mapillary

[brigr]

Dear all,

I would like to extend the vanilla OrienterNet system to support localization in a new area of land which, as always, is bounded by a large rectangular patch. The patch is defined by two points A and B with latitude and longitude coordinates. I want to use the Mapillary service to provide such a functionality in the OrienterNet.

To do this, if I am correct (please correct me if I am wrong), I first need to update the file at ./maploc/data/mapillary/splits_MGL_13loc.json which is internally copied into datasets/MGL once you call "python3 -m maploc.data.mapillary.prepare --token MYTOKEN". There, I have to create two new key-value pairs for the region I need to support.

Here are some basic questions:

1) How can we obtain the "image IDs" used in the definitions of areas such as in sanfrancisco_soma? What script is needed to obtain the image IDs corresponding to the rectangular patch of a new geographic area? Can someone point to a Mapillary snippet that does so (especially the one that was actually used by the developers of the OrienterNet to create entries for, e.g., sanfrancisco_soma)?

2) In the file ./maploc/data/mapillary/splits_MGL_13loc.json, what is the difference between two key-value pairs that are put under "train" and under "val"? In particular, how should the image IDs be obtained (collected/selected) for either case?

Many thanks in advance.

[sarlinpe]

Hi Sotiris, Are you trying to A) localize your own images in a new area or to B) augment the training/evaluation dataset with examples of (images,pose,map) in a new area?

A) If you have lots of images of the same area, it is better to create a single large map tile once rather than querying OSM for each image. To do so, download the relevant OSM data from Geofabrik and create the tiles as: https://github.com/facebookresearch/OrienterNet/blob/8a30494c128bf8450d1520057c19d808dde9edc5/maploc/data/mapillary/prepare.py#L354-L360

B) You would indeed need to augment splits_MGL_13loc.json with Mapillary image IDs for the training and validation splits. I will add code to generate such IDs from a bounding box and optionally filter by sequence ID, username, or camera model - expect this next week. Training and validation areas are spatially disjoint - the code will also include such splitting.

I will update this issue after the code update.

[brigr]

Dear Paul,

Hi Sotiris, Are you trying to A) localize your own images in a new area or to B) augment the training/evaluation dataset with examples of (images,pose,map) in a new area?

I am trying to find all images that belong to a rectangular area by "image ID" (as per the Mapillary API). The reason I am publicly asking the simple question of how to obtain those, is that there are different Mapillary API versions which at some point cause confusion as to how to generate/obtain them. For example, I downloaded all "image IDs" that fall within a circle centered around a point, but once I update the JSON file mentioned in this conversation, prepare.py is unable to get the metadata for the image IDs (which leads me to ask what I am doing wrong and crack the complexity of the API). On the other hand, I am still studying the implementation of the OrienterNet.

I suppose that for a particular geographic area, I do have to obtain examples to augment the training / evaluation dataset in order to have a working OrienterNet that can answer queries for a particular land area.

A) If you have lots of images of the same area, it is better to create a single large map tile once rather than querying OSM for each image. To do so, download the relevant OSM data from Geofabrik and create the tiles as:

https://github.com/facebookresearch/OrienterNet/blob/8a30494c128bf8450d1520057c19d808dde9edc5/maploc/data/mapillary/prepare.py#L354-L360

Thank you for the pointer. I am going to look into this.

B) You would indeed need to augment splits_MGL_13loc.json with Mapillary image IDs for the training and validation splits. I will add code to generate such IDs from a bounding box and optionally filter by sequence ID, username, or camera model - expect this next week. Training and validation areas are spatially disjoint - the code will also include such splitting.

I will update this issue after the code update.

That would be very helpful. It seems to be a requirement in order to avoid diving into the OrienterNet implementation from scratch. I believe most people would need to go the Mapillary way in applying OrienterNet on a particular land area.

[rometaidla]

Was the code to generate training and validation split added?

Thanks

B) You would indeed need to augment splits_MGL_13loc.json with Mapillary image IDs for the training and validation splits. I will add code to generate such IDs from a bounding box and optionally filter by sequence ID, username, or camera model - expect this next week. Training and validation areas are spatially disjoint - the code will also include such splitting.

I will update this issue after the code update.