buerokratt / Analytics-Module

0 stars 14 forks source link

Cronjob to publish Analytics Module's data to the Open Data Portal #131

Open turnerrainer opened 1 year ago

turnerrainer commented 1 year ago

AS A Product Owner I WANT Analytics Module's data to be published to the Open Data Portal periodically in an automated manner SO THAT it wouldn't require any manual interaction

Acceptance Criteria

1AhmedYasser commented 10 months ago

New Requested Changes

  1. Possibility to create a cron task inside an api
  2. Possibility to pass body and params to http request, you can also add headers (Optional)
1AhmedYasser commented 9 months ago

@RayDNoper

We request 2 changes

  1. Adding CORs to cron manager to allow ruuter to call its api's
  2. Modify ruuter form data upload to allow dynamic file names
upload:
 call: http.post
 args:
   contentType: formdata
   url: "[#ANALYTICS_OPEN_DATA_SERVICE]/api/organizations/my-organizations/[#OPEN_DATA_PORTAL_ORGANIZATION_ID]/datasets/${incoming.body.datasetId}/upload"
   body:
     delimiter: ","
     headerIndex: "1"
     file:files:metrics${incoming.body.date}.csv: ${metricDataCSV}
   headers:
     Authorization: ${'Bearer ' + apiToken.response.body.data.accessToken}
 result: uploadResult
 next: check_upload_result
1AhmedYasser commented 9 months ago

Blocked waiting for requested changes

turnerrainer commented 9 months ago

@1AhmedYasser please take a look at and @RayDNoper provide your input regarding if https://github.com/buerokratt/Buerokratt-Chatbot/issues/30 can be used for this purpose.

Ruuter should have the necessary functionality already.

ValterAro commented 9 months ago

Short summary: I set up a daily data upload to avandmeddev, but no files are being sent. Expected result: the selected file should be sent to avaandmeddev.eeti.ee daily Real result: the metadata is there, but the status is "pooleli/unfinished" and there is no files attached How to reproduce: login and go to https://admin.dev.buerokratt.ee/analytics/reports and then create a daily uploaded dataset. Pic:

Image

ValterAro commented 9 months ago

Seems to work as intended