jhohlfeld / waterguardian

A website and app for the citizen science & open data project surrounding water quality in Brunswick, Germany.
https://waterguardian.vercel.app
3 stars 0 forks source link

Add import for manual measurements #10

Closed jhohlfeld closed 1 hour ago

jhohlfeld commented 3 days ago

We can access Excel Worksheets via the Microsoft Graph API:

  1. fetch a token using a registered azure application
  2. use the link of a shared excel document
  3. read the values from the first worksheet of the shared document and returns it

To test, call:

curl http://localhost:3000/graph/samples | jq "."

Make sure to configure this in your .env:

GRAPH_TENANT_ID=<tenant-id>
GRAPH_CLIENT_ID=<client-id>
GRAPH_CLIENT_SECRET=<client-secret>
GRAPH_ITEM_SHARE_URL=<document-share-link>

The result it a json like this:

[
  [
    "Datum",
    "Uhrzeit",
    "Lat",
    "Lng",
    "Alt",
    "Probennummer",
    "Stoffname",
    "Messwert"
  ],
  [
    45624,
    0.5625,
    8.2135422,
    53.1446486,
    7,
    1,
    "Stickstoff",
    0.3
  ],
  [
    45624,
    0.5625,
    8.2135422,
    53.1446486,
    7,
    2,
    "Phosphor",
    0.07
  ]
]

Performance improvements

This process involves up to 4 http calls, which may entail wait times in the order of 6 seconds. That's why it may be beneficial to fetch the data and store it in memory/on disk.

⚠️ Updates might be possible to achieve using a webhooks API: https://learn.microsoft.com/en-us/graph/change-notifications-delivery-webhooks

-> will do in #22