search

ipfs / ipfs-gui

Creating standards and patterns for IPFS that are simple, accessible, reusable, and beautiful
105 stars 17 forks source link

Provide example snippet of data sent to our `countly.ipfs.io` endpoint for necessary features #130

Closed SgtPooki closed 1 year ago

SgtPooki commented 1 year ago

I will provide examples in this issue of the type of metrics that countly sends for our necessary metrics:

From #125:

  1. Collect necessary metrics by default. These will be enabled when using our projects moving forward. i.e. you cannot opt-out of these metrics. However, we are doing our best to ensure that these necessary metrics are the minimum required for us to prioritize work on our projects.
    • necessary metrics will be 'sessions' and 'views' as categorized by countly (i.e. const necessaryFeatures = ['sessions', 'views']). See This Issue for more information
      • sessions - tracks when, how often, and how long users use your website
      • views - allows for the views/pages accessed by a user to be tracked
        • Note that these view metrics are "App views" and not page URL (i.e. public/private gateway access views that include CID)
SgtPooki commented 1 year ago

I'm obtaining the data from the URLs below via:

const url = new URL(requestURL) 
console.log(JSON.stringify(decodeURIComponent(url.search.slice(1)).split('&').reduce((acc, qParam) => {
    const [key, value] = qParam.split('=')
    console.log(`${key}=${value}`)
    try {
        acc[key] = JSON.parse(value)
    } catch (err) {
        acc[key] = value
    }
    return acc
}, {}), null, 2));

On public gateway checker

metrics implementation was already merged (https://github.com/ipfs/public-gateway-checker/pull/309), but it's not opt-out yet. I.e. until you accept/decline, no metrics are sent. see https://github.com/ipfs-shipyard/ignite-metrics/issues/2

Prior to accept/decline

Zero requests sent (right now) image

After Accept

Two requests to countly.ipfs.io are sent:

Screenshot

image

Request 1

URL:

https://countly.ipfs.io/i?events=[{"key":"[CLY]_action","count":1,"segmentation":{"type":"click","x":995,"y":24,"width":1190,"height":3002,"view":"/","domain":"ipfs-public-gateway-checker.on.fleek.co"},"timestamp":1671052220562,"hour":13,"dow":3},{"key":"[CLY]_orientation","count":1,"segmentation":{"mode":"portrait"},"timestamp":1671052220565,"hour":13,"dow":3},{"key":"[CLY]_view","count":1,"segmentation":{"name":"/","visit":1,"view":"/","domain":"ipfs-public-gateway-checker.on.fleek.co"},"timestamp":1671052220566,"hour":13,"dow":3}]&app_key=3c2c0819434074fc4d339ddd8e112a1e741ecb72&device_id=155f5945-6eb8-4446-902d-35cff3d5d540&sdk_name=javascript_native_web&sdk_version=22.06.4&t=1&timestamp=1671052220989&hour=13&dow=3

JSON
{
  "events": [
    {
      "key": "[CLY]_view",
      "count": 1,
      "dur": 422,
      "segmentation": {
        "name": "/settings*"
      },
      "timestamp": 1667480662969,
      "hour": 13,
      "dow": 4
    }
  ],
  "app_key": "8fa213e6049bff23b08e5f5fbac89e7c27397612",
  "device_id": "632c2abd-c939-41cd-8080-4b33586da440",
  "sdk_name": "javascript_native_web",
  "sdk_version": 19.08,
  "timestamp": 1667480663018,
  "hour": 13,
  "dow": 4,
  "consent": {
    "sessions": true,
    "events": true,
    "views": true,
    "location": true
  }
}

Request 2

URL:

https://countly.ipfs.io/i?consent=%7B%22sessions%22%3Atrue%2C%22events%22%3Atrue%2C%22views%22%3Atrue%2C%22scrolls%22%3Atrue%2C%22clicks%22%3Atrue%2C%22forms%22%3Atrue%2C%22crashes%22%3Atrue%2C%22attribution%22%3Atrue%2C%22users%22%3Atrue%2C%22star-rating%22%3Atrue%2C%22location%22%3Atrue%2C%22apm%22%3Atrue%2C%22feedback%22%3Atrue%2C%22remote-config%22%3Afalse%7D&app_key=3c2c0819434074fc4d339ddd8e112a1e741ecb72&device_id=155f5945-6eb8-4446-902d-35cff3d5d540&sdk_name=javascript_native_web&sdk_version=22.06.4&t=1&timestamp=1671052221571&hour=13&dow=3

JSON
{
  "consent": {
    "sessions": true,
    "events": true,
    "views": true,
    "scrolls": true,
    "clicks": true,
    "forms": true,
    "crashes": true,
    "attribution": true,
    "users": true,
    "star-rating": true,
    "location": true,
    "apm": true,
    "feedback": true,
    "remote-config": false
  },
  "app_key": "3c2c0819434074fc4d339ddd8e112a1e741ecb72",
  "device_id": "155f5945-6eb8-4446-902d-35cff3d5d540",
  "sdk_name": "javascript_native_web",
  "sdk_version": "22.06.4",
  "t": 1,
  "timestamp": 1671052221571,
  "hour": 13,
  "dow": 3
}

After Decline

Two requests to countly.ipfs.io are sent: This will be the default going forward. i.e. necessary metrics will be toggled on, and not opt-out. See #125

Screenshot

image

Request 1

URL:

https://countly.ipfs.io/i?events=%5B%7B%22key%22%3A%22%5BCLY%5D_view%22%2C%22count%22%3A1%2C%22segmentation%22%3A%7B%22name%22%3A%22%2F%22%2C%22visit%22%3A1%2C%22view%22%3A%22%2F%22%2C%22domain%22%3A%22ipfs-public-gateway-checker.on.fleek.co%22%7D%2C%22timestamp%22%3A1671053598097%2C%22hour%22%3A13%2C%22dow%22%3A3%7D%5D&app_key=3c2c0819434074fc4d339ddd8e112a1e741ecb72&device_id=155f5945-6eb8-4446-902d-35cff3d5d540&sdk_name=javascript_native_web&sdk_version=22.06.4&t=1&location=&timestamp=1671053598126&hour=13&dow=3

JSON
{
  "events": [
    {
      "key": "[CLY]_view",
      "count": 1,
      "segmentation": {
        "name": "/",
        "visit": 1,
        "view": "/",
        "domain": "ipfs-public-gateway-checker.on.fleek.co"
      },
      "timestamp": 1671053598097,
      "hour": 13,
      "dow": 3
    }
  ],
  "app_key": "3c2c0819434074fc4d339ddd8e112a1e741ecb72",
  "device_id": "155f5945-6eb8-4446-902d-35cff3d5d540",
  "sdk_name": "javascript_native_web",
  "sdk_version": "22.06.4",
  "t": 1,
  "location": "",
  "timestamp": 1671053598126,
  "hour": 13,
  "dow": 3
}

Request 2

URL:

https://countly.ipfs.io/i?consent=%7B%22sessions%22%3Atrue%2C%22events%22%3Afalse%2C%22views%22%3Atrue%2C%22scrolls%22%3Afalse%2C%22clicks%22%3Afalse%2C%22forms%22%3Afalse%2C%22crashes%22%3Afalse%2C%22attribution%22%3Afalse%2C%22users%22%3Afalse%2C%22star-rating%22%3Afalse%2C%22location%22%3Afalse%2C%22apm%22%3Afalse%2C%22feedback%22%3Afalse%2C%22remote-config%22%3Afalse%7D&app_key=3c2c0819434074fc4d339ddd8e112a1e741ecb72&device_id=155f5945-6eb8-4446-902d-35cff3d5d540&sdk_name=javascript_native_web&sdk_version=22.06.4&t=1&location=&timestamp=1671053599096&hour=13&dow=3

JSON
{
  "consent": {
    "sessions": true,
    "events": false,
    "views": true,
    "scrolls": false,
    "clicks": false,
    "forms": false,
    "crashes": false,
    "attribution": false,
    "users": false,
    "star-rating": false,
    "location": false,
    "apm": false,
    "feedback": false,
    "remote-config": false
  },
  "app_key": "3c2c0819434074fc4d339ddd8e112a1e741ecb72",
  "device_id": "155f5945-6eb8-4446-902d-35cff3d5d540",
  "sdk_name": "javascript_native_web",
  "sdk_version": "22.06.4",
  "t": 1,
  "location": "",
  "timestamp": 1671053599096,
  "hour": 13,
  "dow": 3
}
SgtPooki commented 1 year ago

Note that more than two requests are sent after initial acceptance/decline based on interactions, but the data objects sent in subsequent requests should mostly conform to the examples given above. One example with necessary metrics, the following action (clicking on a link in the public gateway checker) is not sent:

{
  "events": [
    {
      "key": "[CLY]_action",
      "count": 1,
      "segmentation": {
        "type": "click",
        "x": 290,
        "y": 428,
        "width": 1363,
        "height": 3028,
        "view": "/",
        "domain": "ipfs-public-gateway-checker.on.fleek.co"
      },
      "timestamp": 1671055107000,
      "hour": 13,
      "dow": 3
    },
    {
      "key": "linkClick",
      "count": 1,
      "segmentation": {
        "href": "https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#The_HTTP_response_headers",
        "text": "CORS",
        "id": "",
        "view": "/"
      },
      "timestamp": 1671055107001,
      "hour": 13,
      "dow": 3
    }
  ],
  "app_key": "3c2c0819434074fc4d339ddd8e112a1e741ecb72",
  "device_id": "155f5945-6eb8-4446-902d-35cff3d5d540",
  "sdk_name": "javascript_native_web",
  "sdk_version": "22.06.4",
  "t": 1,
  "timestamp": 1671055108131,
  "hour": 13,
  "dow": 3
}
SgtPooki commented 1 year ago

Also see basic request data sent: https://support.count.ly/hc/en-us/articles/360037753291-SDK-development-guide#making-requests

There are some parameters that should be added to all requests even though they are not mandatory. Together with the required parameters they form the base request. Every request sent to the server should be formed from this base request. The parameters in this base request are:

"app_key" - the application key for this countly app (retrievable on the dashboard) "device_id" - the current users device ID "timestamp" - the timestamp in ms of when this request is created "hour" - the hour of the timestamp "dow" - the day of the week for this timestamp. 0 - sunday, ... , 6 - saturday. "tz" - this devices timezone offset "sdk_version" - the SDK's version "sdk_name" - the SDK's name

whizzzkid commented 1 year ago

@SgtPooki is there also sample of request (if-any) sent for the metrics collected when the client was off-line (if-any)?

SgtPooki commented 1 year ago

@whizzzkid apparently, countly does not handle collecting and sending offline metrics by default. I tested this via network throttling set to "offline" in brave.

If we want to accomplish that, we should open an issue in ipfs-shipyard/ipfs-ignite that handles it via methods discussed at https://support.count.ly/hc/en-us/articles/360037753291-SDK-development-guide#request-queue but for now we will leave it as is