Additional Classification Methods and Issues with Current Ones

[makella]

@rochoa @javisantana

Are we planning on introducing additional classification methods to Turbo Carto and fixing the ones that currently aren't behaving as expected?

Below I outline the biggest issues I see with our current classification methods and provide some possible fixes.

I'm not sure if you guys are already thinking about these things but I think they are important and would really love to hear what you guys think about my suggestions.

Mostly, I feel like if we are going to be smart about assigning colors to data distributions, we should be equally smart about suggesting the right classification method. I think the classification method is more important than the color ramp.

The biggest issues I see:

• Our version of Jenks gives different results than other implementations I've seen on the same data
• Head/Tails only works if the distribution is skewed in one direction. If it is skewed the opposite direction, the map is really wrong and I can't even explain what is happening.
• Standard Deviation - we don't have it.
• A lot of times, people know their data best and want to assign manual classification methods.
• Sometimes what I've seen happening is that important values are colored with the color for the lowest value in the ramp even though they fall somewhere else in the data and are assigned default symbology.

My recommendations:

Jenks

• As much as we can, fix our Jenks since it is one of the most common classification methods used.

  Skewed Distributions

• Heads/Tails
  • Make sure there is an inverse method

and / or

• Arithmetic Progression
  • In this method, the widths of the category intervals are increased in size at an arithmetic (additive) rate. If your first category is one unit wide and you choose to increment the width one unit at a time, the second category would be two units wide, the third three units wide, and so forth to the end of the distribution. This method can be applied effectively to data that is J-shaped with a peak at the low end of the distribution.
• Geometric Progression
  • In this method, the widths of the category intervals are increased in size at a geometric (multiplicative) rate. If your first category is 2 units wide, the second would be 2x2 or 4 units wide, the third 2x2x2 or 8 units wide, and so forth to the end of the distribution. This method can be applied effectively to data that is J-shaped with a peak at the low end of the distribution but with a long "stretch" between low and high values.
• Inverse Methods
  • Basically, the opposite of the two above when your data is J-shaped with a peak at the high end of the distribution, the inverses of the arithmetic and geometric progressions can be used. By inverting the class breaks, the smallest intervals between class breaks will be closest together at the high end of the distribution.

source: http://www.colorado.edu/geography/gcraft/notes/cartocom/section6.html

Near Normal Distribution

• Standard deviation NEW
  • In this method, the standard deviation of the distribution is used to set the class breaks above and below the average. This method can be applied to distributions that approximate a normal curve.

    Manual Classification

• Right now, this can be done through the CartoCSS. I'm wondering if there is a way to make this easier/more obvious for people.

  Global Variables

• I think we should set the global polygon-fill, marker-fill, line-color variable to a NO DATA color as opposed to the lowest value's color

cc: @saleiva @andrewxhill

[AbelVM]

Let's revisit this, please. Jenks and Head&Tails method are not so reliable. And we lack log methods used as default in scientific studies.

There's already a PR with Log and StdDev methods already there https://github.com/CartoDB/cartodb-postgresql/pull/236

[maxhartshorn]

+1 @AbelVM

It would be really great to be able to add manual classification breaks in Turbo Carto. It's common to have a data set that you want to group by percent change (e.g. 10%, 20%, 30%, 40%, and >50% change). People know their data the best and should have the ability to customize the breakpoints at which their data is categorized.

This can be done with CartoCSS but it would be good to see it in TurboCarto, especially since the CartoCSS markup is not readily available in Carto Builder.

[AbelVM]

@maxhartshorn

You have access to CartoCSS from Builder using the style console indeed

Once there, according to TurboCarto docs, to apply arbitrary breaks, you can use code like:

marker-fill: ramp([my_field], ("#009392","#91b8aa","#f1eac8","#dfa0a0","#d0587e"), (-534, -121, 0, 0.5, 292),"<=");

[maxhartshorn]

Thanks @AbelVM I missed that in the docs!

[coregis]

@AbelVM I have used CartoCSS in Builder to create manual breaks, which is useful when I need to use the same break points to compare datasets with different min/max values. However, the legend does not reflect the manual break points. Do I need to build a custom legend with HTML/CSS to show the custom values? Or is there some trick to getting the legend to work?

[AbelVM]

As long as manual breaks are static and not managed by TurboCARTO, you need to edit the legend HTML to reflect your breaks