entrepreneur-interet-general / OpenScraper

An open source webapp for scraping: towards a public service for webscraping
http://www.cis-openscraper.com/
MIT License
93 stars 22 forks source link

Fix "TSV" generation #67

Open CBalsier opened 5 years ago

CBalsier commented 5 years ago

When exporting data by clicking on "download the set as a .tsv file", the resulting file presents shifted columns when data is not present. See for example in the following screenshots : export from the dataset Apriles-ODAS and "preview" from OpenScraper:

Screenshot from 2019-05-03 17-27-56 Screenshot from 2019-05-03 17-28-24

Suggested fix : a little bit like in https://github.com/entrepreneur-interet-general/OpenScraper/blob/c96c6d85a3e54b90d4f81e06541ab619d8f149f2/openscraper/controller.py#L1362 replace line 1367-1368

if id_field in item.keys() :
    item_list.append( " ".join(item[ id_field ]) )

by

if id_field in item.keys() :
    item_list.append( " ".join(item[ id_field ]) )
else :
    item_list.append("")

to ensure empty values are correctly written at line 1369 in the resulting file.