Content management interface

[intarga]

Follow on from my conversation with Vegar:

Vegar

Hei Ingrid. Håper ferien har vært bra, og at du har fått ladet batteriene tilstrekkelig til å ta fatt på en herlig høst. Jeg fikk et spørsmål fra Elinah i går, som lurte på hva slags kompetanse ho trenger som innholdsfaglig ansvarlig for PODA. Denne rollen er en geofaglig rolle, men det vil være ønskelig at de kan ha direkte editeringstilgang til databasen. Det gjelder Elinah, Ingri, Hildegunn og Ketil og jeg ønsker at de skal bruke samme verktøy og rutiner. Fint om du kan være med på å beskrive hva som må være på plass forå få til dette. Det vil avlaste deg som teknisk systemforvalter av PODA.

Ingrid

Hello. That's a good question.

If, as you say, they will have "direct editing access" then the answer is easy, they need to learn SQL, specifically PostgreSQL. However I don't think anyone should have direct access. For data integrity and provenance reasons we should keep a record of manual changes, and direct access would enable people to bypass or corrupt this record. For an example of this you can look at stinfosys, where people frequently forget (or "forget" if they're trying to cover up their own past mistake) to update the updatedat and updatedby columns. Additionally, direct access would mean they could accidentally mess with partitions, indexes, and constraints, which could seriously compromise the production database in a way that would be difficult to recover from. I don't think we should leave that possibility open.

Given that, we will have to design a safe interface for them, and we will do our best to make it fit them, so ideally they shouldn't have to learn some new technology. For this though, we will need some more information. Specifically:

What past experience triggered us to create the role of content manager? In what ways does the content manager role overlap with what HQC does? In what ways is it different? Can we have a list of examples of changes content managers expect to make, ideally as exhaustive as possible?

[Lun4m]

For reference, the ODA content management wiki page

[ketilt]

What past experience triggered us to create the role of content manager? I cannot answer this, it predates my experience. I would ask Vegar, Per-Ove or Pål Sannes

In what ways does the content manager role overlap with what HQC does? In what ways is it different? The tasks described in the wiki are on a higher system level than HQC tasks and do not overlap with these. HQC edits specific data values, while the tasks described are editing timeseries or sets of timeseries.

Some of the tasks in the wiki might be outsourced from LARD, but then we need to establish where these needs should be met. (For example products, and granting access to restricted data.) Remigrating data will hopefully be unnecessary with LARD?

Can we have a list of examples of changes content managers expect to make, ideally as exhaustive as possible? We have done some work to document these needs through the Wiki Manuel linked above. I'm up for meetings discussing the need to expand this documentation and make users stories etc. if needed. The ideal, as Elinah has described to me, is to concretely describe relevant maintenance tasks for easier onboarding of new personnel. I think this harmonizes well with predictability in development as well.

I very much agree with the sentiment that we should have an interface and not allow direct SQL access. For ODA, I had a mind to extend the ODA viewer to support such tasks, but at the same time being aware that I was not the ideal person for making a very stable operational service. The Deleting data task has been a recurring headache for me, as this is the task that requires the most flexibility, and I frequently receive emails with instructions for data deletions/changes that are being done in kvalobs and KDVH and need to be replicated in ODA... Because we have no better way of handling such issues currently. Through this experience, I've had some ideas for how such a task can be generalized in an interface, and I can provide more details when needed.

[lcoram]

There exists stuff like: https://github.com/rust-dd/rust-sql But perhaps we need something more tailored if we get a list of the types of tasks we need to support...

[ketilt]

pgAdmin could be used for browsing the data, via read-only user credentials. No point trying to recreate the kind of functionality that it provides.

What remains is:

• Deleting data: marking timeseries for deletion/uploading replacements, and archiving the data that is deleted
• ~Product definition management~: done through github?
• Data access: depends how restrictions are handled. ODA needed this because stinfosys didn't handle all of it
• Filter management: filters must be edited in stinfosys, but we need to be able to preview the result of a change. I think the ideal scenario is that this preview is handled in the stinfosys group, but that LARD is queried for a full list of timeseries. (obspgm_h2 is incomplete)