HTTPArchive / tech-report-apis

APIs for the HTTP Archive Technology Report
Apache License 2.0
2 stars 0 forks source link

Adoption Report #5

Open maceto opened 11 months ago

maceto commented 11 months ago

Could you describe the origin/source of this data?

[
    {
        date: '2020-01-01',
        technology: 'Wordpress',
        adoption: {
            mobile: 10,
            desktop: 10,
            across_dataset: 20,
        },
    }
]

The goal is to create a script to query this data from BQ transform and save it in Firestore.

rviscomi commented 10 months ago

Query

CREATE TEMPORARY FUNCTION GET_ADOPTION(
  records ARRAY<STRUCT<
      client STRING,
      origins INT64
  >>
) RETURNS STRUCT<
  desktop INT64,
  mobile INT64
> LANGUAGE js AS '''
return Object.fromEntries(records.map(({client, origins}) => {
  return [client, origins];
}));
''';

SELECT
  date,
  app AS technology,
  rank,
  geo,
  GET_ADOPTION(ARRAY_AGG(STRUCT(
    client,
    origins
  ))) AS adoption
FROM
  `httparchive.core_web_vitals.technologies`
WHERE
  date = '2023-07-01'
GROUP BY
  date,
  app,
  rank,
  geo

Example record

{
  "date": "2023-07-01",
  "technology": "WordPress",
  "rank": "ALL",
  "geo": "ALL",
  "adoption": {
    "desktop": "2040628",
    "mobile": "3422903"
  }
}
maceto commented 10 months ago

@rviscomi, should we have any mandatory param for this endpoint?

rviscomi commented 10 months ago

Maybe just:

@sarahfossheim WDYT?

maceto commented 10 months ago

Example of how to consume this endpoint

curl --request GET \
  --url 'https://dev-gw-2vzgiib6.ue.gateway.dev/v1/adoption?geo=Mexico&technology=["Blueshift","GoCache"]&rank=ALL'
rviscomi commented 10 months ago

Per our chat, try reformatting in the Python to this structure:

{
  "Blueshift": [{"date": ...}],
  "GoCache": []
}
maceto commented 10 months ago

@rviscomi @sarahfossheim, all the changes discussed are already deployed.

New URL https://dev-gw-2vzgiib6.uk.gateway.dev/v1/adoption

Documentation: https://github.com/HTTPArchive/tech-report-apis#get-adoption