Simplest way to add custom datasource

[gsavvid]

We've built some nice and interactive maps with harp.gl using the datasource you provide (https://vector.hereapi.com/v2/vectortiles/base/mc) and by extending some of the open source styles.

Now we are looking for a way to add a few extra properties to the datasource. For example, let's assume we want to add a flag isRenovated to certain buildings and have some additional data-driven styling. The question is how can we achieve this in the most seamless way? Or to be more precise, how can we achieve this with the least changes possible to our existing styles? In theory, I see two options.

Option 1: Not sure if this datasource from Here is open but if it is, we could copy it, update it based on our needs, and upload it to a new XYZ Space. This would be the easiest because we could use the existing styles without any changes.

Option 2: We have retrieved the buildings from OSM and uploaded them to a new XYZ space. I couldn't find any official documentation on how to render data from an XYZ Space so I followed the example here: https://github.com/heremaps/harp.gl/pull/471 and it seems to be working. My question here is (assuming that Option 1 is not possible), is there a way to see the properties of the buildings in the datasource from Here that we are currently using (https://vector.hereapi.com/v2/vectortiles/base/mc) so that we make sure that our datasource has the same properties? If could do that, we wouldn't have to create a custom style from scratch.

Thank you in advance!

[musculman]

Hi @gsavvid, thank you for using harp.gl!

Regarding your questions, the HERE Vector Data is not open in the sense that it is not use OSM, but the specification is open. It is based on Tilezen. The HERE map data (vector.hereapi.com) is documented here. The layer section should contain the details that you are after, as it describes all the possible properties of each of the layers of the HERE Vector Data.

That being said, you cannot download the data and upload it again to XYZ but adding an additional property isRenovated or other. So, I understand that option 1 would not be possible. But the option 2 as I understand it should work provided the documentation I mentioned is enough for you.

Please, let us know if that helped or not. Thanks!

[nzjony]

@gsavvid , have you see the https://www.harp.gl/docs/master/examples/#object-picking.html example? You can click on a building and see the properties. Is that what you mean by:

is there a way to see the properties of the buildings in the datasource from Here that we are currently using (https://vector.hereapi.com/v2/vectortiles/base/mc)

[gsavvid]

Thank you both very much. This answers my questions.

[gsavvid]

Unfortunately, I haven't had any luck with this. I did compile a GeoJSON file that follows the Tilezen scheme and uploaded it to an XYZ Space. I see in the console that the tile requests are successful but no buildings are rendered when using berlin_tilezen_base.json (or in fact any of the other open themes). I can also see the building footprints in geojson.tools so there's nothing wrong with the coordinates.

This is how I create my datasource:

return new OmvDataSource({
      name: "buildings",
      baseUrl: "https://xyz.api.here.com/hub/spaces/LYaKOWIj/tile/web",
      apiFormat: APIFormat.XYZSpace,
      styleSetName: "tilezen",
      authenticationCode: "my-auth-code-is-here"
});

I made the LYaKOWIj space public but here's a little part of it for convenience:

{
  "type": "FeatureCollection",
  "etag": "942934f6c925e35119428ddc01071703",
  "features": [
    {
      "type": "Feature",
      "id": "d1ee6d89a16ad618336d2adf9eb0f63a",
      "geometry": {
        "type": "Polygon",
        "coordinates": [ ... ]
      },
      "properties": {
        "@ns:com:here:xyz": {
          "space": "LYaKOWIj",
          "createdAt": 1593697864213,
          "updatedAt": 1593697864213,
          "uuid": "2cdf4e35-a8d1-421a-8e20-72de064ef86f",
          "tags": [
            "nyc-manhattan-central-park"
          ]
        },
        "sort_rank": 475,
        "kind": "building",
        "min_zoom": 15,
        "height": 7.3
      }
    },
   ...

I'm wondering if there are some fields missing. For example, I see in berlin_tilezen_base.json that there's the following condition ["==", ["get", "$layer"], "buildings"] which makes me think that perhaps the problem is that thebuildings layer is missing from my GeoJSON. However, if I wrap the entire GeoJSON inside "{ buildings": { ... } }, then the upload to my XYZ space fails.

Once again, I'd be very grateful if you could shed some light on this.

[gsavvid]

I managed to get the buildings to render but the extruded polygons don't have any fill colour. Basically, I use the berlin_tilezen_base.json but just changed the where condition and in particular replaced ["==", ["get", "$layer"], "buildings"] with ["==", ["get", "kind"], "building"]. That's the closest I could get.

[nzjony]

@gsavvid , good job, looks like you are making good progress.

Off the top of my head, the issue you are facing is probably the winding order, I guess the order of the coordinates is however not in your control and controlled by the xyz space. Can you try changing the winding order of the extruded-polygon material? See: https://threejs.org/docs/#api/en/constants/Renderer

It is pretty impressive you have made it this far, I don't think any of us have tried doing this yet.

[gsavvid]

@nzjony thanks for the hint. Could you elaborate a little bit more perhaps? How can I access/update the extruded polygon material?

[nzjony]

@gsavvid , sorry the hint I gave you was wrong, I searched in three.js and couldn't see how this is actually used.

Try this:

Go to DecodedTileHelpers.ts and search for:

if (isExtrudedPolygonTechnique(technique)) {
        material.flatShading = true;

and as a test, add material.side = THREE.BackSide;

directly beneath

Does that fix the issue?

If so, we would need to fix xyz to output the coordinates in the correct order... or perhaps internally we can iterate them in reverse order if there is some particular reason they are like this.

[gsavvid]

@nzjony thank you for the quick response. Looks like we're getting somewhere now. material.side = THREE.BackSide; had the opposite effect:

But material.side = THREE.DoubleSide; seems to solve the issue although the texture of the material looks a little bit strange now. If you zoom in you'll see there are some sort of strips on the buildings:

[nzjony]

The stripes are from the shadow map z fighting with the actual geometry, see: https://threejs.org/docs/index.html#api/en/materials/Material.shadowSide. It seems that the shadowSide can be different from the side.

But anyway, it makes more sense to fix the underlying root cause for why the coordinates are in the opposite winding order to other data sources, I will ask internally if anyone knows.

[atomicsulfate]

Hi @gsavvid, The winding order problem you're experiencing only affects data sources using "APIFormat.XYZSpace" API format. Try switching to "APIFormat.XYZJson" in the meantime while we work on a fix.

[nzjony]

@gsavvid , see: https://github.com/heremaps/harp.gl/commit/910282cb6baecd498c5d3bbcac37157762e7a84b

It should have fixed this issue. Can you take a look and please verify (and close if fixed).

[gsavvid]

Hi @atomicsulfate and @nzjony thank you both for the progress on this. Switching to APIFormat.XYZJson fixes indeed the issue.

About https://github.com/heremaps/harp.gl/commit/910282cb6baecd498c5d3bbcac37157762e7a84b: I'm not sure what's the best way to test it since there's no (beta) release with this fix. I modified the OmvData.js file by copy-pasting checkWinding() (and of course a call to this method accordingly) but I see no results. Am I missing something? FYI our project is built with ReactJS.

[nzjony]

hi @gsavvid , sorry for the delay, the change you mentioned is now in the 0.19 release, see: https://github.com/heremaps/harp.gl/releases/tag/v0.19.0

Can you please upgrade and give it a try.