[taboule] ^v2 improvements

vecna commented 2 years ago

[ ] In every trex website you can find static/js/global.js and a function that help in testing different server endpoint. Taboule doesn't use this utility and this prevent certain condition to work (I'm talking about this ttps://github.com/tracking-exposed/tiktok.tracking.exposed/blob/main/static/js/global.js )
[x] The csv button in the Taboule is a .txt while it should be a .csv
[ ] The filtering button in the Taboule is not filtering results. It would be nice if the .csv produced inside Taboule could contain just the filtered results and not all of them
[ ] /personal/ API might become just the equivalent of what summary now (.count of all the collections with a publicKey, and a check in supporters collection to return lastActivity and pseudonym)
[ ] Taboule should only call the 'metadata' API which is quite flexible already (as it take as parameters a wider list of possibilities such as the publicKey (usable in the current "personal page"), but as well also experimentId and researchTag which make it usable in other pages.
[ ] Taboule in regarding of tiktok should support foryou and following (feed), search, and profile.

updated after a chat with @ascariandrea

ascariandrea commented 2 years ago

[x] include Taboule in storybook
[x] include platform in queries names
[x] use react-json-view for json fields

ascariandrea commented 2 years ago

YT queries to fix

I'm fixing the GET /api/v2/metadata to return all the metadata now returned by the GET /api/v1/personal/:publicKey in the right format. So you can then use this API to display data properly on Taboule.

[x] youtube personal homes

This is the home object return from the GET /api/v2/metadata?publicKey=$publicKey&nature=home:

[
  {
    "_id": "633d574b1e7464a96e9fe88e",
    "id": "dfc1599f241d4928d9e908bd4ae4ab0687910f57",
    "blang": "it",
    "clientTime": "2022-10-05T10:07:07.000Z",
    "href": "https://www.youtube.com/",
    "login": true,
    "publicKey": "H7AsuUszehN4qKTj2GYYwNNzkJVqUQBRo2wgKevzeUwx",
    "savingTime": "2022-10-05T10:07:08.603Z",
    "sections": [{ "i": 0, "offset": 392613 }],
    "selected": [
      {
        "index": 3,
        "verified": true,
        "videoId": "xQCLf9T_M7Q",
        "sectionName": "Commenti",
        "recommendedSource": "Cercle",
        "recommendedHref": "/c/Cercle",
        "recommendedTitle": "Bedouin live at Petra, Jordan for Cercle",
        "recommendedLength": 6826,
        "recommendedDisplayL": "1:53:46",
        "recommendedLengthText": "1 ora, 53 minuti e 46 secondi",
        "recommendedRelativeSeconds": 13132800,
        "recommendedViews": 1440235,
        "isLive": false,
        "label": "Bedouin live at Petra, Jordan for Cercle di Cercle 5 mesi fa 1 ora e 53 minuti 1.440.235 visualizzazioni",
        "elems": 9512,
        "thumbnailHref": "https://i.ytimg.com/vi/xQCLf9T_M7Q/hq720.jpg",
        "publicationTime": "2022-05-05T10:07:07.000Z",
        "timePrecision": "estimated"
      }
    ],
    "type": "home"
  }
]

and Taboule should:

[x] display the selected values in a proper column
[x] add the yt video link to selected item
[x] display the selected count in a proper column
[x] display login information in a proper column
[x] display savingTime in a proper column
[x] youtube personal videos

This is the video object returned from the GET /api/v2/metadata?publicKey=$publicKey&nature=video

{
  "_id": "633d57071e7464a96e9fe73d",
  "id": "4573c628aad6f6ee09c078800c6859ed39fb0ab8",
  "authorName": "Luis Fast Beats",
  "authorSource": "/channel/UCIvDPo_MHKRm2Zs-uCJ-DBg",
  "blang": "en-US",
  "clientTime": "2022-10-05T10:05:54.000Z",
  "forKids": false,
  "href": "https://www.youtube.com/watch?v=9N5mXcdGzYc",
  "likeInfo": {
    "likes": null,
    "dislikes": null,
    "watchedLikes": null,
    "watchedDislikes": null
  },
  "login": true,
  "nature": { "type": "video" },
  "params": { "v": "9N5mXcdGzYc" },
  "publicKey": "H7AsuUszehN4qKTj2GYYwNNzkJVqUQBRo2wgKevzeUwx",
  "publicationString": "28 set 2022",
  "publicationTime": "2022-09-28T00:00:00.000Z",
  "related": [
    {
      "index": 1,
      "verified": true,
      "foryou": null,
      "videoId": "ferZnZ0_rSM",
      "params": { "v": "ferZnZ0_rSM" },
      "recommendedSource": "NPR Music",
      "recommendedTitle": "Anderson .Paak & The Free Nationals: NPR Music Tiny Desk Concert",
      "recommendedLength": 938,
      "recommendedDisplayL": "15:38",
      "recommendedLengthText": "15 minuti e 38 secondi",
      "recommendedRelativeSeconds": 189302400,
      "recommendedViews": 92302532,
      "recommendedThumbnail": "https://i.ytimg.com/vi/ferZnZ0_rSM/hqdefault.jpg",
      "isLive": false,
      "label": "Anderson .Paak & The Free Nationals: NPR Music Tiny Desk Concert di NPR Music 6 anni fa 15 minuti 92.302.532 visualizzazioni",
      "publicationTime": "2016-10-05T10:05:54.000Z",
      "timePrecision": "estimated"
    }
  ],
  "savingTime": "2022-10-05T10:05:58.490Z",
  "title": "\"STREET KEYS\" - Lil Baby Dark Piano Type Beat | Gloomy Type Beat | prod. by Luis Fast",
  "type": "video",
  "videoId": "9N5mXcdGzYc",
  "viewInfo": { "viewStr": "5.701 visualizzazioni", "viewNumber": 5701 }
}

[x] youtube personal searches

This is the search metadata returned from the GET /api/v2/metadata?publicKey=$publicKey&nature=search

{
  "_id": "633d576c1e7464a96e9fe949",
  "id": "2467fcaa2222832d9a77e5d1bd9f0e74d55ee73e",
  "blang": "en-US",
  "clientTime": "2022-10-05T10:07:29.000Z",
  "correction": [],
  "href": "https://www.youtube.com/results?search_query=dsa",
  "publicKey": "H7AsuUszehN4qKTj2GYYwNNzkJVqUQBRo2wgKevzeUwx",
  "query": "dsa",
  "results": [
    {
      "position": 2,
      "title": "Cercle",
      "authorName": "Cercle",
      "authorSource": "/c/Cercle",
      "sectionName": "Search results",
      "href": "/watch?v=xQCLf9T_M7Q",
      "videoId": "xQCLf9T_M7Q",
      "views": 1440235,
      "arialabel": "Bedouin live at Petra, Jordan for Cercle di Cercle 5 mesi fa 1 ora e 53 minuti 1.440.235 visualizzazioni",
      "isLive": false,
      "order": 1,
      "published": "5 months",
      "secondsAgo": 13132800
    },
    {
      "position": 3,
      "title": "The Action Lab",
      "authorName": "The Action Lab",
      "authorSource": "/c/TheActionLab",
      "sectionName": "Search results",
      "href": "/watch?v=pwx12kwgOAM",
      "videoId": "pwx12kwgOAM",
      "views": 483239,
      "arialabel": "How Does The Anti-Gravity Wheel Work? di The Action Lab 2 mesi fa 10 minuti e 19 secondi 483.239 visualizzazioni",
      "isLive": false,
      "order": 2,
      "published": "2 months",
      "secondsAgo": 5270400
    }
  ],
  "savingTime": "2022-10-05T10:07:36.137Z",
  "type": "search"
}

and Taboule should:

[x] display query in a proper column
[x] display results in a proper column and add the link to yt video's page
[x] display savingTime in a proper column

rekoke commented 2 years ago

Added button to re-run taboule from the current vie:

ascariandrea commented 1 year ago

For closing this one I'm waiting a dump of the prod DB from @Pandry to test the new UI with some real data

tracking-exposed / trex

[taboule] ^v2 improvements #441

YT queries to fix