Publish data in Open Data Portal

[rasmusei]

AS AN Analytic module user I WANT TO be able to create single or templates of datasets to (recurringly) send selected anonymized data to the Open Data Portal SO THAT I can follow Estonia's Open Data publishing customs

Acceptance Criteria

• [x] User must be able to create new dataset according to https://avaandmed.eesti.ee/api/dataset-docs/, including Dataset metadata creation, Dataset file uploading, Data quality improvements, Dataset publishing or discarding (changed to dev environment https://avaandmeddev.eesti.ee/api/dataset-docs/.
• [x] explanation has to be available as tooltip that user needs to first create personal account in Open Data Portal, create API key and obtain access token and establish connection with the institution.
• [x] connectivity to Open Data Portal for user available under menu selection Raportid (reports)
• [x] list of metrics with tickboxes as defined in #17 available for selection
• [x] #21
• [x] #24
• [x] Data must be anonymized
• [ ] #121

  Possibility to set interval for automatic data uploads to Open Data Portal (daily, weekly, monthly)

• [x] Possibility to send data to Open Data Portal manually
• [x] Button to download dataset for preview
• [x] Dataset metadata manipulation

DSL

• [x] The service is only available via Private Ruuter
• [x] User making the requests must be authenticated

GUI

Tooltip

• [x] Based on Figma design - hover to "Dependencies"
• [x] Content as a configurable i18n value, no database used

List of metrics

• [x] GUI needed
• [x] List of metrics based on configurable i18n values, no database used

Recurring services

• [x] Time interval dropdown with time preference to define the recurrence of services
• [x] REST endpoint to get current settings
• [x] REST endpoint to apply changes
• [x] Settings saved in a Postgres database

Misc

• [x] Button to download dataset for preview
• [x] Option to send data to Open Data Portal manually
• [x] Text fields to add / edit metadata (see ODP documentation) of every dataset

Other

• [x] Service to set up recurring requests

BUG REPORT

All ACs that have not been ticked, are considered as bugs

• [x] #59

[turnerrainer]

Regarding to required developments, DSLs are needed to produce in order to get the correct datasets to publish. Ruuter, DMapper, Resql definitely are involved here.

@PaulaMerle does it make sense business-wise to allow everyone to publish any kind of data as public data?

[turnerrainer]

Publishing datasets to Open Data Portal would be done by using our central REST service for that (does not yet exist). Creating such base service would not be part of this issue, only using it / calling it out would be.

[PaulaMerle]

@turnerrainer I have made some changes to the ACs, pls check if everything now makes sense considering your latest comment on separate REST service.

[1AhmedYasser]

@turnerrainer pending ruuter ability to support PUT & DELETE requests and the ability to pass variables in the url without them being in the constants.ini

[ValterAro]

Ahmed will create a PR and this goes into acceptance testing.

[ValterAro]

As @1AhmedYasser wrote under the issue #89: " @turnerrainer Now This still needs ruuter changes to add the cronjob ability to run scheduled uploads for datasets as mentioned here Possibility to set interval for automatic data uploads to Open Data Portal (daily, weekly, monthly) in https://github.com/buerokratt/Analytics-Module/issues/3 and it will be addressed in development here https://github.com/buerokratt/Analytics-Module/issues/121

PR is up https://github.com/buerokratt/Analytics-Module/pull/118 Thank you, "

[turnerrainer]

As @1AhmedYasser wrote under the issue #89: " @turnerrainer Now This still needs ruuter changes to add the cronjob ability to run scheduled uploads for datasets as mentioned here Possibility to set interval for automatic data uploads to Open Data Portal (daily, weekly, monthly) in #3 and it will be addressed in development here #121

PR is up #118 Thank you, "

No Ruuter developments, just adding a cronjob to call out appropriate Ruuter endpoints making requests to ODP.

[ValterAro]

[BUG] short description: The list of metrics("Lisavalikud") on the ODP page are not unselectable. Expected result: When clicking on the check mark in front of the list item it deselects it. Real result: Nothing happens, no error. How to reproduce: Log in and go to https://admin.dev.buerokratt.ee/analytics/reports . Then try do unselect any item from the "Lisavalikud" list.

[ValterAro]

Waiting for cronjob update to be completed by Alar until this can be considered completed.

[rasmusei]

Blocked by https://github.com/buerokratt/Analytics-Module/issues/131

[1AhmedYasser]

Blocked by https://github.com/buerokratt/Analytics-Module/issues/131#issuecomment-1903860466

[ValterAro]

Short summary: when on https://admin.dev.buerokratt.ee/analytics/reports and trying to download the csv then the request goes through and you get a proper response, but it does not download a file. Expected result: When you press download csv, then you get the file Real result: It does the correct request and gets the response, but no file is saved to the computer. How to reproduce: go to https://admin.dev.buerokratt.ee/analytics/reports and try to download a csv Pic:

[ValterAro]

Seems to work as intended