gem / oq-platform-ipt

OpenQuake Platform - Input Preparation Toolkit
2 stars 1 forks source link

Error when downloading an exposure model with a CSV file #132

Open CatalinaYepes opened 3 years ago

CatalinaYepes commented 3 years ago

The following error appears in the online version of the IPT, even after checking the headers

Error some error occured during csv headers check

An example is provided here: Exposure_Model_Antioquia_FIXED.csv.zip The tags are municipality, region, height, economic_income.

In addition, the tool should be flexible and simple check if the indicated tags are in the column headers. Now OQ accepts csv files with additional columns that can be ignored.

nastasi-oq commented 3 years ago

The encoding of "ñ" isn't correct UTF8 (below the right one compared with what you can find in the csv file passed as example). Behavior of different python versions on encoding errors is difficult to predict and to fix. Please fix the CSV file to avoid the error.

# Right encoding of "n + ~" (ñ) character
103 141 303 261 141 163 147 157 162 144 141 163 012
  C   a 303 261   a   s   g   o   r   d   a   s  \n

# Wrong encoding of "n + ~" (ñ) character
103 141 303 203 302 261 141 163 147 157 162 144 141 163 012
  C   a 303 203 302 261   a   s   g   o   r   d   a   s   \n
nastasi-oq commented 3 years ago

Looking for a tool that fix these kind of problems I found a python tool that works almost well: ftfy After installation you can use it as normal shell command There is also an unoffical web version: https://ftfy.now.sh/

CatalinaYepes commented 3 years ago

The IPT should be displaying a clear error message. So instead of saying some error occurred during csv headers check, it should say wrong encoding in csv file. Make sure to use utf-8.

nastasi-oq commented 3 years ago

The IPT should be displaying a clear error message. So instead of saying some error occurred during csv headers check, it should say wrong encoding in csv file. Make sure to use utf-8.

What I can get from the exception is what the error message say, we could add your suggestion as fix hint.