buerokratt / Analytics-Module

0 stars 12 forks source link

Cronjob to publish Analytics Module's data to the Open Data Portal #131

Open turnerrainer opened 11 months ago

turnerrainer commented 11 months ago

AS A Product Owner I WANT Analytics Module's data to be published to the Open Data Portal periodically in an automated manner SO THAT it wouldn't require any manual interaction

Acceptance Criteria

1AhmedYasser commented 8 months ago

New Requested Changes

  1. Possibility to create a cron task inside an api
  2. Possibility to pass body and params to http request, you can also add headers (Optional)
1AhmedYasser commented 8 months ago

@RayDNoper

We request 2 changes

  1. Adding CORs to cron manager to allow ruuter to call its api's
  2. Modify ruuter form data upload to allow dynamic file names
upload:
 call: http.post
 args:
   contentType: formdata
   url: "[#ANALYTICS_OPEN_DATA_SERVICE]/api/organizations/my-organizations/[#OPEN_DATA_PORTAL_ORGANIZATION_ID]/datasets/${incoming.body.datasetId}/upload"
   body:
     delimiter: ","
     headerIndex: "1"
     file:files:metrics${incoming.body.date}.csv: ${metricDataCSV}
   headers:
     Authorization: ${'Bearer ' + apiToken.response.body.data.accessToken}
 result: uploadResult
 next: check_upload_result
1AhmedYasser commented 8 months ago

Blocked waiting for requested changes

turnerrainer commented 8 months ago

@1AhmedYasser please take a look at and @RayDNoper provide your input regarding if https://github.com/buerokratt/Buerokratt-Chatbot/issues/30 can be used for this purpose.

Ruuter should have the necessary functionality already.

ValterAro commented 7 months ago

Short summary: I set up a daily data upload to avandmeddev, but no files are being sent. Expected result: the selected file should be sent to avaandmeddev.eeti.ee daily Real result: the metadata is there, but the status is "pooleli/unfinished" and there is no files attached How to reproduce: login and go to https://admin.dev.buerokratt.ee/analytics/reports and then create a daily uploaded dataset. Pic:

Image

ValterAro commented 7 months ago

Seems to work as intended