search

climsoft / Climsoft

Climsoft Desktop for Windows - http://www.climsoft.org
GNU General Public License v3.0
14 stars 28 forks source link

Keeping intermediary data entry tables #695

Open isedwards opened 2 years ago

isedwards commented 2 years ago

I'd like to propose that all software that interacts with Climsoft 4 databases continue to use the intermediary data entry tables like form_daily2.

If we update the data entry form code to interact directly with observationinitial then we may be in a situation where some countries are still using intermediary tables and other are not... Climsoft desktop and OpenCDMS would have to support both scenarios correctly.

I think the intermediary tables are part of the Climsoft 4 design/schema and the recommended approach to move away from this would be to migrate users to a new data model. Originally this was intended to be Climsoft 5, but now it is likely that we may adopt WMO's Climate Data Model Standard - the first version of which will be available by October. At this point we would work directly with the observations table.

Patowhiz commented 2 years ago

@isedwards thanks for your suggestion.

The concept of interaction of data entry form with observationinitial directly is still under discussions. It's still not clear to the team what challenges may arise. Being the one that suggested this, I'm still exploring how this can be done correctly.

Ideally, this approach means slightly restructuring the observationinitial table to accommodate such functionality.

From the user perspective nothing is changed, they will still feel the same data flow (just a slight change of the meaning).

My suggestion is mainly influenced from our users request of improving the performance of uploading from the data entry forms to observationinitial. Such an operation to me is like changing a record's flag. Also note, any data imported just goes to observationinitial.

There is also several other advantages of using observationinitial directly but those can be worked on once this feature is implemented and tested.

Also note, that we generally agreed that, for this concept to be accepted, it must demonstrate that it causes no regression, in particular;

  1. Indeed improves the performance and does not affect the performance of other operations.
  2. Doesn't lead to unintended negative consequences like potential data loss.