search

HughCraig / GHAP

1 stars 0 forks source link

Download and upload of CSV spreadsheet doesn't work. #360

Open BillPascoe opened 2 months ago

BillPascoe commented 2 months ago

If I create a layer and add a place to it using the form, then download the CSV and then upload it again without changing anything, it causes an error, like: SQLSTATE[22P02]: Invalid text representation: 7 ERROR: invalid input syntax for type bigint: "" CONTEXT: unnamed portal parameter $3 = '' (SQL: update "tlcmap"."dataitem" set "extended_data" = <![CDATA[0]]><![CDATA[0101000020E6100000BE38EFFFD3DD614010C6D200E66941C0]]><![CDATA[0101000020E6100000BE38EFFFD3DD614010C6D200E66941C0]]>, "dataset_order" = 1, "route_id" = , "updated_at" = 2024-05-02 15:18:10 where "id" = 807757)

My intention is to get a template of the CSV by doing this, then do some testing by uploading CSV with various modifications.

gdzqzzx commented 2 months ago

Thank you for letting me know this. May I have more details on your testing?

  1. When you add a place, did you try to add all essential attributes only or did you also try to add either route_id or quantity?
  2. With the dataset (mobility_test_1_1.csv) I provided, will you still have issue on upload, modification and download?
gdzqzzx commented 1 month ago

In brief, this is caused by conflicts between the downloaded csv structure and uploaded csv structure.

  1. Unaccepted columns like layer_order (this also happens in current production on dataset_order);
  2. Empty values of route related columns from downloaded file. By default, route information would be generated for csv when quantity value is detected, and not all route related columns, for example route_original_id and stop_idx, are accepted in uploading.

It's unlikely to have a ultimate solution for this bug yet. Highlighting the requirement of accepted mobility dataset and giving clear example for user to replicate would be the best way to avoid the bug in this stage.