microsoft / datamations

https://microsoft.github.io/datamations/
Other
66 stars 14 forks source link

Refactoring #102

Closed giorgi-ghviniashvili closed 2 years ago

giorgi-ghviniashvili commented 2 years ago

@sharlagelfand could you please test this? will this work?

image

image

sharlagelfand commented 2 years ago

Thanks @giorgi-ghviniashvili, unfortunately not - giving the same error:

TypeError: encoding[d] is null

giorgi-ghviniashvili commented 2 years ago

@sharlagelfand can you share full error message? Also share the specs array you are passing? it seems like that encoding is not valid or missing. Better to add couple of console.log statements to find out why it fails? Does it call App with correct specs?

sharlagelfand commented 2 years ago

thanks for your help debugging @giorgi-ghviniashvili! I pushed some changes to this branch that solve the problem on my end

It looks like there is a slight issue with the grid generating now where the IDs assigned to the points are a bit off, causing the animation to sort of criss-cross between frames - I don't quite know how to explain it, so here's a video!

In the main branch, in the first frame the points on the left side remain on the left side in the second frame, and same with the points on the right side remaining on the right side, so the animation just splits them:

https://user-images.githubusercontent.com/15895337/138319564-d5956bd3-ba46-4a84-86d1-3e644fe11199.mov

But in this branch, the points on the left in the first frame go to the right in the second frame, and vice versa:

https://user-images.githubusercontent.com/15895337/138319608-9cf38176-0752-46e7-aa08-3303a44d4797.mov

Here are the specs for these animations (the same in both):

[
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Initial data"
    },
    "data": {
      "values": [
        {
          "n": 100
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      }
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Group by Degree",
      "splitField": "Degree",
      "axes": false
    },
    "data": {
      "values": [
        {
          "Degree": "Masters",
          "n": 72
        },
        {
          "Degree": "PhD",
          "n": 28
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      },
      "color": {
        "field": null,
        "type": "nominal"
      },
      "tooltip": [
        {
          "field": "Degree",
          "type": "nominal"
        }
      ]
    }
  }
] 

If you can help figure out why this is happening - thank you!

giorgi-ghviniashvili commented 2 years ago

@sharlagelfand that's beautiful actually! If I wanted to do it intentionally, it would be difficult :D Alright, now seriously: there is gemini_id generation issue, because I switched to dynamic rows and fixed columns.. I will fix that.

giorgi-ghviniashvili commented 2 years ago

@sharlagelfand @jhofman I fixed that in this way:

https://user-images.githubusercontent.com/6615532/138684685-ffcd0d07-352f-4f66-8060-26f59a3a16d4.mov

sharlagelfand commented 2 years ago

thanks @giorgi-ghviniashvili, I do find the animation style a bit distracting and might take away from the point, would definitely be curious to hear @jhofman's thoughts!

I noticed that it looks like points are flying on and off the grid, you can see a bit better if you just use e.g. 5 points

https://user-images.githubusercontent.com/15895337/138731048-c6ecdfc3-4818-43cd-831b-7c168aeb685f.mov

Here are the specs:

[
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Initial data"
    },
    "data": {
      "values": [
        {
          "n": 5
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      }
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Group by Degree",
      "splitField": "Degree",
      "axes": false
    },
    "data": {
      "values": [
        {
          "Degree": "Masters",
          "n": 2
        },
        {
          "Degree": "PhD",
          "n": 3
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      },
      "color": {
        "field": null,
        "type": "nominal"
      },
      "tooltip": [
        {
          "field": "Degree",
          "type": "nominal"
        }
      ]
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "jitter",
      "axes": false,
      "description": "Plot Salary within each group",
      "splitField": "Degree",
      "xAxisLabels": ["Masters", "PhD"]
    },
    "data": {
      "values": [
        {
          "gemini_id": 1,
          "Degree": "Masters",
          "datamations_x": 1,
          "datamations_y": 81.9445013836958,
          "datamations_y_tooltip": 81.9445013836958
        },
        {
          "gemini_id": 2,
          "Degree": "Masters",
          "datamations_x": 1,
          "datamations_y": 82.8953005648218,
          "datamations_y_tooltip": 82.8953005648218
        },
        {
          "gemini_id": 3,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 84.4868333523627,
          "datamations_y_tooltip": 84.4868333523627
        },
        {
          "gemini_id": 4,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.8469139884692,
          "datamations_y_tooltip": 83.8469139884692
        },
        {
          "gemini_id": 5,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.7531382157467,
          "datamations_y_tooltip": 83.7531382157467
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": {
          "values": [1, 2],
          "labelExpr": "round(datum.label) == 1 ? 'Masters' : 'PhD'",
          "labelAngle": -90
        },
        "title": "Degree",
        "scale": {
          "domain": [0.5, 2.5]
        }
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "title": "Salary",
        "scale": {
          "domain": [81.9445013836958, 84.4868333523627]
        }
      },
      "tooltip": [
        {
          "field": "datamations_y_tooltip",
          "type": "quantitative",
          "title": "Salary"
        },
        {
          "field": "Degree",
          "type": "nominal"
        }
      ]
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "axes": false,
      "description": "Plot median Salary of each group"
    },
    "data": {
      "values": [
        {
          "gemini_id": 1,
          "Degree": "Masters",
          "datamations_x": 1,
          "datamations_y": 82.4199009742588,
          "datamations_y_tooltip": 82.4199009742588
        },
        {
          "gemini_id": 2,
          "Degree": "Masters",
          "datamations_x": 1,
          "datamations_y": 82.4199009742588,
          "datamations_y_tooltip": 82.4199009742588
        },
        {
          "gemini_id": 3,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.8469139884692,
          "datamations_y_tooltip": 83.8469139884692
        },
        {
          "gemini_id": 4,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.8469139884692,
          "datamations_y_tooltip": 83.8469139884692
        },
        {
          "gemini_id": 5,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.8469139884692,
          "datamations_y_tooltip": 83.8469139884692
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": {
          "values": [1, 2],
          "labelExpr": "round(datum.label) == 1 ? 'Masters' : 'PhD'",
          "labelAngle": -90
        },
        "title": "Degree",
        "scale": {
          "domain": [0.5, 2.5]
        }
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "title": "median(Salary)",
        "scale": {
          "domain": [81.9445013836958, 84.4868333523627]
        }
      },
      "tooltip": [
        {
          "field": "datamations_y_tooltip",
          "type": "quantitative",
          "title": "median(Salary)"
        },
        {
          "field": "Degree",
          "type": "nominal"
        }
      ]
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "axes": false,
      "description": "Plot median Salary of each group, zoomed in"
    },
    "data": {
      "values": [
        {
          "gemini_id": 1,
          "Degree": "Masters",
          "datamations_x": 1,
          "datamations_y": 82.4199009742588,
          "datamations_y_tooltip": 82.4199009742588
        },
        {
          "gemini_id": 2,
          "Degree": "Masters",
          "datamations_x": 1,
          "datamations_y": 82.4199009742588,
          "datamations_y_tooltip": 82.4199009742588
        },
        {
          "gemini_id": 3,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.8469139884692,
          "datamations_y_tooltip": 83.8469139884692
        },
        {
          "gemini_id": 4,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.8469139884692,
          "datamations_y_tooltip": 83.8469139884692
        },
        {
          "gemini_id": 5,
          "Degree": "PhD",
          "datamations_x": 2,
          "datamations_y": 83.8469139884692,
          "datamations_y_tooltip": 83.8469139884692
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": {
          "values": [1, 2],
          "labelExpr": "round(datum.label) == 1 ? 'Masters' : 'PhD'",
          "labelAngle": -90
        },
        "title": "Degree",
        "scale": {
          "domain": [0.5, 2.5]
        }
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "title": "median(Salary)",
        "scale": {
          "domain": [82.4199009742588, 83.8469139884692]
        }
      },
      "tooltip": [
        {
          "field": "datamations_y_tooltip",
          "type": "quantitative",
          "title": "median(Salary)"
        },
        {
          "field": "Degree",
          "type": "nominal"
        }
      ]
    }
  }
] 
giorgi-ghviniashvili commented 2 years ago

@sharlagelfand I fixed that. here is the video:

https://user-images.githubusercontent.com/6615532/138831359-303907d3-2f60-47dc-9ab4-45ac1d6bd48d.mov

BTW, please start gemini_id from 0. I updated the spec:

image

jhofman commented 2 years ago

good catch on the points flying in from off the screen, glad that's fixed.

the alternate criss/cross style animation is kinda fun looking, but i agree that in the end it's too distracting. let's keep with the original of points staying on the side they start on.

jhofman commented 2 years ago

It looks like the criss-crossing is happening because of the dynamic number of rows now that we're trying to make things square-ish. specifically, we see weirdness when the size of the squares changes between the ungrouped and grouped key frames.

the current thought is that we use the first frame to decide the dimensions of the square, and then keep that for the remaining frame, but we're not certain that will always look reasonable.

let's make some test cases to visually check that things look sane:

@sharlagelfand, can you generate the specs for these?

sharlagelfand commented 2 years ago

Here are some test specs and their videos:

100 points to 50/50

https://user-images.githubusercontent.com/15895337/138937144-ba792f6b-9353-4405-b449-7317afbb4864.mov

[
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Initial data"
    },
    "data": {
      "values": [
        {
          "n": 100
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      }
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Group by group",
      "splitField": "group",
      "axes": false
    },
    "data": {
      "values": [
        {
          "group": "group1",
          "n": 50
        },
        {
          "group": "group2",
          "n": 50
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      },
      "color": {
        "field": null,
        "type": "nominal"
      },
      "tooltip": [
        {
          "field": "group",
          "type": "nominal"
        }
      ]
    }
  }
] 

100 points to 48/52 (to see the crossover when the number isn't round to 10 / sqrt(1000)

https://user-images.githubusercontent.com/15895337/138937175-30a9f210-c5d1-4fe0-8921-52dd390ef08d.mov

[
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Initial data"
    },
    "data": {
      "values": [
        {
          "n": 100
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      }
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Group by group",
      "splitField": "group",
      "axes": false
    },
    "data": {
      "values": [
        {
          "group": "group1",
          "n": 48
        },
        {
          "group": "group2",
          "n": 52
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      },
      "color": {
        "field": null,
        "type": "nominal"
      },
      "tooltip": [
        {
          "field": "group",
          "type": "nominal"
        }
      ]
    }
  }
] 

100 points to 87/13 (again to see the crossover)

https://user-images.githubusercontent.com/15895337/138937434-a4bbc02b-04b3-49b3-94f7-f9bc059224fd.mov

[
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Initial data"
    },
    "data": {
      "values": [
        {
          "n": 100
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      }
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Group by group",
      "splitField": "group",
      "axes": false
    },
    "data": {
      "values": [
        {
          "group": "group1",
          "n": 87
        },
        {
          "group": "group2",
          "n": 13
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      },
      "color": {
        "field": null,
        "type": "nominal"
      },
      "tooltip": [
        {
          "field": "group",
          "type": "nominal"
        }
      ]
    }
  }
] 

100 points to 20 groups of 5

https://user-images.githubusercontent.com/15895337/138937478-7de5d5d0-f40c-4f75-a4d8-92a8aef5bb90.mov

[
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Initial data"
    },
    "data": {
      "values": [
        {
          "n": 100
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      }
    }
  },
  {
    "height": 300,
    "width": 300,
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "meta": {
      "parse": "grid",
      "description": "Group by group",
      "splitField": "group",
      "axes": false
    },
    "data": {
      "values": [
        {
          "group": "group1",
          "n": 5
        },
        {
          "group": "group10",
          "n": 5
        },
        {
          "group": "group11",
          "n": 5
        },
        {
          "group": "group12",
          "n": 5
        },
        {
          "group": "group13",
          "n": 5
        },
        {
          "group": "group14",
          "n": 5
        },
        {
          "group": "group15",
          "n": 5
        },
        {
          "group": "group16",
          "n": 5
        },
        {
          "group": "group17",
          "n": 5
        },
        {
          "group": "group18",
          "n": 5
        },
        {
          "group": "group19",
          "n": 5
        },
        {
          "group": "group2",
          "n": 5
        },
        {
          "group": "group20",
          "n": 5
        },
        {
          "group": "group3",
          "n": 5
        },
        {
          "group": "group4",
          "n": 5
        },
        {
          "group": "group5",
          "n": 5
        },
        {
          "group": "group6",
          "n": 5
        },
        {
          "group": "group7",
          "n": 5
        },
        {
          "group": "group8",
          "n": 5
        },
        {
          "group": "group9",
          "n": 5
        }
      ]
    },
    "mark": {
      "type": "point",
      "filled": true
    },
    "encoding": {
      "x": {
        "field": "datamations_x",
        "type": "quantitative",
        "axis": null
      },
      "y": {
        "field": "datamations_y",
        "type": "quantitative",
        "axis": null
      },
      "color": {
        "field": null,
        "type": "nominal"
      },
      "tooltip": [
        {
          "field": "group",
          "type": "nominal"
        }
      ]
    }
  }
] 
giorgi-ghviniashvili commented 2 years ago

@sharlagelfand @jhofman 48/52 and 87/13 is expected behavior. The last 2 and last 3 circles move to the top first column of the next grid, which causes other columns to slide bottom - top, like a zuma game.

About 20 groups of 5, that's also expected, because columns divided by half - 10 rows then 5 rows.

Do you have any ideas how to improve these case visually?