Update Instructions for uploading user reference data in RDM UI

[santoshkaranam]

Will be done after milestone 1 meeting

[kvantricht]

To be reviewed by @boog-wur and @juanlaso-iiasa .

[boog-wur]

To do a review I need credentials. Who can provide me with the credentials?

[santoshkaranam]

Please register here

[boog-wur]

@santoshkaranam : I am trying to upload a geoparquet file (QGIS saves it as parquet). But the system does not allow this type

[boog-wur]

@Santosh: and how can I remove such entry? I am now forced to go back and enter the metadata again

[santoshkaranam]

@boog-wur as per RDM UI currently the geoparqet file should end with .geoparquet not jus .parquet , hence the error. Could you rename the file and try. Geoparquet file is bit different than a parquet file as it has additional geometry column metadata. This data is mandatory to be present in the parquet file.

Also we currently only support EPSG4326 WGS84

[jdegerickx]

@santoshkaranam, would be nice to also allow .gpkg files for user dataset upload

[boog-wur]

I did a first review of the upload of data by users. See https://vitoresearch.sharepoint.com/:w:/r/sites/21717-ccn-world-cereal/Shared%20Documents/System/RDM/[Evaluate_upload_private_user_data_set.docx](https://vitoresearch.sharepoint.com/:w:/r/sites/21717-ccn-world-cereal/Shared%20Documents/System/RDM/Evaluate_upload_private_user_data_set.docx?d=w5eb4ab24c2e64ecd8561f4d73478e624&csf=1&web=1&e=DyYIzw)?d=w5eb4ab24c2e64ecd8561f4d73478e624&csf=1&web=1&e=DyYIzw

@jdegerickx : could you check my proposed changes before your holidays?

@santoshkaranam : after Jeroen D. additions/comments we could go through the doc and discuss the improvements e.g. during the maps4gpp scheduled for 6 August

[santoshkaranam]

@jdries @boog-wur i have added support to .gpkg files and .parquet files.

[jdegerickx]

@boog-wur, thanks for the review. I have added my comments to the file directly.

[boog-wur]

@jdegerickx / @santoshkaranam : I see that you want to avoid the term "column". I wanted to do the same but then I saw that the whole UI uses the word column. So should we change column to attribute everywhere in the upolad UI (text, headers)?

[jdegerickx]

I personally prefer "attribute", but that is maybe less intuitive for a user, I don't know. We can also keep using column. We can always adjust later based on user feedback...

[boog-wur]

Yes attribute is more general and less technical. But Santosh has to change it in many places. Let's hear from Santosh. If too much work we might keep column for now.

[santoshkaranam]

@boog-wur @jdries i changed the name column --> to--> Attribute in website, have a look

[boog-wur]

Looks good, let's discuss other proposed changes when I am back from holidays thus Tuesday 6 August

[boog-wur]

Today Santosh and I discussed the review done by Jeroen Degerickx and myself. See Word document (link is in one of previous comments). Santosh will process the changes. Some improvements will be done later (next version). These are:

• support more co-ordinate systems
• automated generation of data set name / collection id by only asking identifier/key word
• ai-assisted mapping
• add sampling

[santoshkaranam]

@boog-wur I have updated the website with version1 changes as discussed.

[boog-wur]

@santoshkaranam: thanks, I still have a few issues. Could we meet tomorrow morning?

[santoshkaranam]

@boog-wur sure

[boog-wur]

Today we discussed what issues still can be included in the first version of the upload. Before October 2024 we aim to include:

• Automated generation of data set name / collection id by only asking identifier/key word
• Assisted mapping within the UI
• Add sampling If ready before October we can include these improvements in the MOOC and other dissemination events

On assisted mapping within the UI the following. There are two options: 1) AI assisted mapping which works good but depends on paid API service. 2) Solution based on google translate and egular expression and is free.

We might consider to first use option 2 as it is free and has no dependency on a third party. And we prefer to do the mapping within the GUI thus without the csv down and upload.

The issue to support more co-ordinate systems will be done later.

[boog-wur]

Concerning "Automated generation of data set name / collection id by only asking identifier/key word": according Santosh it is not easy to generate the datasetid from different elements. At this moment of the upload we do not know the different elements of the desired dataset id name. So we have to ask the user to come with a proposal, preferably based on our convention (available in separate document). In the editor we can show, in grey font, an example to support the user.

Note that the UI assisted mapping is ready and can be tested

[santoshkaranam]

added help text for ttitle, datasetId inputs.