GeoJSONVectorTileDataSource vs Vector Data Source

[lasyakoechlin]

Are there any performance benefits of using:

• GeoJSONVectorTileDataSource
• CartoVectorTileLayer

As opposed to:

• LocalVectorDataSource
• VectorLayer

---

Problem:

I have an indoor data set which is very, very heavy. I have used Carto in the past for indoor polygons without issues. I have setup spatial indexes and geometry simplifiers. This one data set if very large and occasionally can lead to map lag, especially on older phones or if there's a lot of other user activity going on and phone memory is low. On newer phones that doesn't tend to be as much of an issue, but if memory

---

Question:

Given that I have tried everything I can think of or found online in terms of performance optimization, I was wondering if using the new GeoJSONVectorTileDataSource might be able to provide performance improvements?

Does it change the rendering mechanism in some way that could handle much larger datasets?

Thanks.

[lasyakoechlin]

Also, if you have any sample code on how to convert VectorElements into an appropriate Geojson layer that would be great. Thanks!

[mtehver]

The short answer: very likely yes, there would be performance benefits. VectorTileLayers are highly optimized for rendering performance and low memory consumption compared to VectorLayers. The supported feature set is also larger for VectorTileLayers, though not strictly superset of VectorLayers/elements.

The biggest difference between VectorLayers and VectorTileLayers is styling: CartoCSS is used for VectorTileLayers and styling is decoupled from actual features (elements). CartoCSS is a rule based system that uses properties of the features (elements). SDK uses heavily extended dialect of the original CartoCSS defined by MapBox, extensions are documented here: https://github.com/CartoDB/mobile-sdk/wiki/CartoCSS-notes

For a complex example of CartoCSS, you can take a look at CARTO basemaps: https://github.com/CartoDB/mobile-sdk/blob/develop/assets/cartostyles-v2.zip

Unfortunately our public projects do not have any samples demonstrating this. A will put this into our TODO list, but I can not give any timeframe for this.

[lasyakoechlin]

Thanks for the helpful response! So, I could use some guidance about how to go about setting this up.

It's clear to me how to create the GeoJSONVectorTileDataSource.

Because I want to test the migration from VectorLayers to using the GeoJSONVectorTiles, I want to just write an intermediary script to convert my current VectorElements to use the new data source, to facilitate quick prototyping.

Here is what I am currently doing...

mGeoJsonDataSource = new GeoJSONVectorTileDataSource(0, 24);

mTileVectorLayer = new CartoVectorTileLayer(mGeoJsonDataSource, mCartoCSS);

FeatureVector featureVector = new FeatureVector(); for(int i = 0 ; i < v.size(); i++) { VectorElement element = v.get(i); Feature feature = new Feature(element.getGeometry(), new Variant(element.getMetaData())); featureVector.add(feature); }

FeatureCollection featureCollection = new FeatureCollection(featureVector);

try { mGeoJsonDataSource.setLayerFeatureCollection(0, new EPSG3857(), featureCollection); } catch (Exception e) { LogHelper.e(getClass(), ErrorHelper.getError(e)); }

What do I need to do to associate specific polygons (or vector elements) with specific CartoCSS features? For example, let's say I want to define some custom features for the CartoCSS... Say, "Desk", "Stairs", and "Room". And I want the polygons to plot in different colours accordingly.

I assume that I must use the Variants somehow to establish this connection because the Geometry doesn't contain that information; however, it's unclear to me how to use the Variants for that purpose.

As well, I was wondering if it would be possible to on the fly change the "state" of a specific polygon. For example, if I had two CartoCSS features "Room" and "Room-Selected" (or highlighted) with different colors, could I programatically change the behavior?

Thanks!

[mtehver]

The snippet looks ok, I would make the following change by creating a vector tile layer within the datasource:

int layerIndex = mGeoJsonDataSource.createLayer("baselayer");

mGeoJsonDataSource.setLayerFeatureCollection(layerIndex, new EPSG3857(), featureCollection);

Assuming the the metadata of a polygon has the following structure (as JSON):

{ "class": "floor" }

Then this CartoCSS should render polygons with blue color:

#baselayer {
  [class='floor'] {
    polygon-fill: #0000ff;
  }
}

For interactivity or highlighting there are a few options: if possible, I would suggest creating an overlay layer that contains only 'highlighted' elements. If this is not possible due to styling decisions, etc, then it possible to dynamically update CartoCSS. For example, assuming each element has unique id, the following CartoCSS can be used:

#baselayer {
  [class='floor'] {
    polygon-fill: ([id] = 156 ? #ffffff : #0000ff);
  }
}

It is possible to use 'style parameters' instead of hard-coded id 156 in the example above, I suggest looking the basemaps style for an example.

[lasyakoechlin]

Hi thanks for your reply! I'm struggling with a number of things, and I have a few follow-up questions.

Questions: 1. In my previous implementation, I have a set of VectorLayers representing each floor. Each VectorLayer has a data source, and when I change floors, I toggle the visibility of all layers.

With your recommendation to create the vector layer within the data source, I would lose some functionality (e.g., changing opacity and visibility of the vector layer itself).

So, how would you recommend implementing that functionality (specifically the toggling of floors)?

2. I haven't been able to figured out how to set a custom asset package. I was able to do so in a really old version (back in the nutiteq days), but the APIs have changed and it's a bit unclear to me how to go about doing so.

I also want to avoid having to re-do too much work... But, I will need to potentially dynamically adjust the styling based on different buildings, so I want to be able to programatically adjust different parameters. I understand how to do that with CartoCSS, conceptually.

If I create the CartoVectorTileLayer manually, I need to provide an AssetPackage, but I can't figure out how to create that.

Do I need to create a CartoCSSStyleSet? MBVectorTileDecoder?

I saw that using the AssetUtils.loadAsset(...) I could get a BinaryData object, but this whole process is unclear to me.

Also, this would only work with a predefined .zip file and would not necessarily allow for dynamically adjusting the CartoCSS file. Any guidance on this would be greatly appreciated.

3. In your example above using the structure: #baselayer { [class='floor'] { polygon-fill: #0000ff; } }

You mentioned to add the meta data as '{ "class": "floor" }'

I assume that this means a Variant with key "class" and value "floor"?

4. In your highlighting example, you mentioned the use of: #baselayer { [class='floor'] { polygon-fill: ([id] = 156 ? #ffffff : #0000ff); } }

How would I be able to set this Id dynamically? Would I have to maintain a reference to specific polygons and change their "id" to either 156 or something else to trigger the plotting behaviour?

I think I'm not understanding the specifics of how to actually go about doing this programatically (i.e., how to use the above CartoCSS style).

Thanks so much for your time!

[mtehver]

Answers to the questions:

1. If you need to set visibility of individual layers, then I suggest creating multiple VectorTileLayer instances.

2. Look at the code sample below.

3. Correct. I suggest looking at the code sample below.

4. Look at the code sample below.

I prepared a small demo using asset packages, vector tile feature listeners and parametrised CartoCSS styling here: https://gist.github.com/mtehver/991e4e4e7de8a524ded62a4edc88ed62

[lasyakoechlin]

Thank you so much for that very helpful code! I was able to piece it together with your previous comments and get the "class" behavior working, as well. So, that was super helpful!

I have a few follow-up questions.

1. Does this GeoJsonDataSource and CartoCSS support 3D polygons?

The only thing I could see that had a height was the "Building" structure / geometry... But it was unclear to me how to use this or if it was possible to do so. (https://carto.com/developers/styling/cartocss)

How would you suggest going about 3D polygons? Do I need to add a VectorElement layer of 3D polygons on top?

2. Right now, I have it working by loading a CartoCSS .zip file directly from the assets folder. If I want to instead download a CartoCSS .zip file from my server (for example, that way I could have custom CartoCSS styling for different buildings), and then create my data source from the downloaded .zip (where the path would not be in the asset folder necessarily), how can I go about doing so?

I know one option might be to try writing to the asset folder with the new .zip (not sure that's feasible).

[lasyakoechlin]

Also, I'm experimenting with text options and I'm having an issue with

text-placement: interior;

Basically, when I use it, the text is being plotted along the lines (boundary) of the polygons rather than inside of them.

I assume this is a known issue? I found a few similarly asked questions online but wasn't able to find any solution.

The text portion of my CartoCss is as follows.

text-name: [name];
text-placement: interior;
text-placement-type: simple;
text-align: centre;
text-fill: @text_fill;
text-face-name: "Noto Sans Medium";
text-size: linear([view::zoom], (0, 5), (20, 20));
    ....

I'm not sure if maybe I'm providing conflicting information like text-size and text-placement? Any assistance is appreciated.

[mtehver]

Regarding the questions:

1. 'Building symbolizers' are the way to go. A snippet from our basemap style:

    building-height: [render_height] ? [render_height] : 10;
    building-min-height: [render_min_height];
    building-fill: lighten(@buildings,10%);
    building-fill-opacity: linear([view::zoom], (15, 0.0), (18, 0.25));

Regarding the geometry, simply use normal polygons as geometry. Height is taken from element properties, in this example from 'render_height'. The height should be given in meters.

2. You can construct BinaryData from byte array in Java

3. Regarding 'interior' text-placement. This is not supported and not on our roadmap for the near future. But you can probably get a similar effect by using text-dy (text-dy: 5, for example).

[mtehver]

Hi,

without seeing all the code I suspect you are missing either:

1. Reference to a font containing Arabic glyphs. For example, in our basemap style @mont_it_md is defined as:

@mont_it_md: "Montserrat Italic", "Noto Sans Medium", "Noto Sans Regular", "NanumBarunGothic Regular", "Koruri Regular", "Noto Sans Tibetan Regular", "Noto Sans Khmer Regular";

Only 'Noto Sans' contains Arabic glyphs, 'Montserrat' does not.

2. You are missing actual font in your .zip asset. In our basemaps the font that provides Arabic glyphs is 'NotoSans-Medium.ttf'. Note that the font file names and font names are different, in CartoCSS you should use actual font names, not font file names.

Regards, Mark

On Fri, Apr 17, 2020 at 8:01 AM lasyakoechlin notifications@github.com wrote:

Awesome! The Building symbolizers work great. Thanks for the BinaryData tip.

I was also wondering about multiple language support. I saw in the latest Carto styles the follow:

In labels.mss:

@name: [nuti::lang] ? ([name:[nuti::lang]] ? [name:[nuti::lang]] : ([name:[nuti::fallback_lang]] ? [name:[nuti::fallback_lang]] : [name])) : [name];

With the simple usage of:

mylayer {

text-name: @name;

text-face-name: @mont_it_md;

....

}

When I was using VectorElements, and I created a Text element and set the text to a non-English string, everything automatically rendered no problem.

I'm having an issue that I'm only seeing English text. I have copy/pasted from this zip: https://github.com/CartoDB/mobile-sdk/blob/master/assets/cartostyles-v2.zip http://url

I copied over the fonts and used the above @name https://github.com/name and the above font, etc.

I also copied over the following nultiparameters:

"nutiparameters": {

"selected_id": {

  "default": -1

},

"lang": {

  "default": "en"

},

"fallback_lang": {

  "default": "en"

}

}

With the following CartoCSS text labels code:

text_labels {

text-name: @name;

text-orientation: [rotation];

text-fill: @text_fill;

text-face-name: @mont_md;

text-size: linear([view::zoom], (0, 5), (17, 7), (20, 14));

}

In code, I have the following logic when populating the text:

   VariantObjectBuilder builder = new VariantObjectBuilder();

    builder.setString("name", "EnglishName");

    builder.setString("rotation", String.valueOf(0.0));

    builder.setString("name:ar", "الاسم الانجليزي" ); // example in arabic

    PointGeometry pointGeometry = new PointGeometry(mapPos); // some location

    mMBVectorTileDecoder.set("lang", "ar"); // try to set nulti parameter to arabic

    // continue with adding to FeatureCollection, etc.

If I remove the "name:ar" variant, everything plots in English properly. If I make the "name:ar" be an English string, it also works. And, if I experiment with different variants, for example, "name_ar", then even when I pass in arabic, it produces English, which makes sense, because based on the above @name https://github.com/name logic chain, it should fall back on English. But, what that tells me is that my format of "name:ar" seems to be correct.

My suspicion was that I was just missing the Arabic font or something, but I found various other font files that were in other repos (for example: https://github.com/hotosm/HDM-CartoCSS/tree/master/fonts http://url where here they have explicit fonts for Arabic among others.

When I imported all those fonts and tried using them in the same way that is done in their corresponding .css, I obtained the same behavior as above.

I'm really not sure what I'm missing, but I'm guessing it's something in the java code itself, rather than in the CartoCSS? Maybe I'm setting the Variant incorrectly or not telling Carto that the language has changed properly. Maybe there's dedicated language codes?

Thanks for your help!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/CartoDB/mobile-sdk/issues/344#issuecomment-615043010, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVBWDPSDNYEQ662ARG2MWLRM7PCXANCNFSM4LRUGUVA .

[lasyakoechlin]

• I was able to resolve it by manually including the Arabic font. Thanks, though!
• I was able to manually load the BinaryData

1. Next issue... I'm noticing a bit of a delay when I update style parameters. For example:

mDecoder.setStyleParameter("selected_id", Integer.toString(id));

The colour does change, but it's not exactly smooth. Please see the attached video file (found here: ٢٠٢٠٠٤١٧_١٨٣٨٠٩.zip).

Sometimes its a very smooth change but other times it's kind of a big laggy almost in the way that the colour appears to fill one tile before the other.

I would almost rather if it changed colours with a delay and updated both tiles simultaneously (don't know if that's possible).

EDIT: I was able to resolve the second issue on my own, the first issue is still outstanding

Thanks for all your help!

[mtehver]

@lasyakoechlin This is expected, as the map is divided into tiles and tiles are updated independently and asynchronously. There are 2 things to try:

1.Use tileLayer.setSynchronizedRefresh(true). I am not 100% sure this works, if it does not, please give us feedback about this.

2. Another option is to enable multiple 'tiling' threads by calling mapView.getOptions().setTileThreadPoolSize(4). The default is 1 thread, so using 4 threads should make the issue much less noticable.

The first one is the suggested fix for the issue and the second one is somewhat a workaround.

[lasyakoechlin]

So, the tileLayer.setSynchronizedRefresh(true) does work, but it also incurred a very significant delay in both general map rendering and in updating of the colours for the selected polygons.

I believe that is caused by the fact that the layer I am using has many polygons on it.

The mapView.getOptions().setTileThreadPoolSize(4) works brilliantly. There is occasionally a very small delay in the tile rendering, but it almost gives the illusion of a "swipe animation", so it looks smooth. It may have also sped up other aspects of the map rendering. Thanks!

---

1. Very dark 3D polygons using Building Symbolizer: Specifically, I want to show several comparisons so that I can better explain.

VectorTiles with 2D Polygons only:

VectorElements with 2D + 3D Polygons:

You will notice that the color scheme is nearly identical (I think the color codes should be exactly the same, but I'm not 100% sure because the VectorElements screenshot is from a previous build).

But now, when I enable the Building Symbolizer, I get very different effects.

For each of the below images, I experimented with different values of the building-fill-opacity and the lighten(@color, @percent), lightening percentage.

VectorTiles with 2D + 3D Polygons:

As you can see, It feels like the color that I'm seeing is quite different. As in, the exact same color code results in a substantially different color. I'm guessing this is a known effect since your above sample code included the lighten(@building,10%).

Are there any tips/recommendations on how to improve this behavior? Or any recommendations to better approximate the coloring that I was getting using VectorElements?

When I was using VectorElements, I was doing the following:

• 2D polygon with outline (because I needed the line divisions between stores)
• 3D polygon on top of it with same color and opacity = 0.7

Any advice or recommendations would be great. For context, here are some exerts of how I've setup the CartoCSS file.

#indoor {
    building-height: [nuti::use_3d] ? ([render_height] ? [render_height] : 0.0) : 0.0;
    building-fill-opacity: [nuti::use_3d] ? ([render_height] ? @height_opacity : 0.0) : 0.0;
    polygon-opacity: 1.0;

/* Note that I've simplified the below to take out all the selection-related code */
    [class='room'] {        
        polygon-fill: @room_fill;                       
        line-color: ([name] = @room_line;
        line-width: @polygon_line_width;

                /* adjusted for different %'s, 10%, 25%, 50%, ... */
        building-fill: lighten(@room_fill, @lighten_percent_3d); 
    }

}

2. Negative Heights

When I was using VectorElements (Polygon3D), I was able to assign a negative height to an element to give the illusion of it protruding downwards. Specifically, I was using this for floor openings (e.g., That does not seem to be working for the Building Symbolizers.

Thanks!

[lasyakoechlin]

3. Is there any support on the mobile-sdk for polygon pattern fills? I tried experimenting with:

polygon-pattern-file polygon-pattern-opacity

without any success. If not, is this in the pipeline?

[mtehver]

@lasyakoechlin Regarding the questions:

1. I have now changed the behavior of buildingsymbolizer in vector tile renderer to match 3D polygons of VectorLayer. The old behavior was to apply lighting/shading to all parts of the building. The new behavior is to apply shading only to sides of the building. The new version behaves like Mapnik, which is a 'reference implementation'.

2. I have now enabled this.

3. No, this is not supported, nor planned.

The changes will be available in 4.3.0RC3, which I plan to release this weekend.

[mtehver]

Closing this, as the questions seem to be answered.

[lasyakoechlin]

Thanks for the help!