GetDKAN / dkan

DKAN Open Data Portal
https://dkan.readthedocs.io/en/latest/index.html
GNU General Public License v2.0
373 stars 171 forks source link

Add option for Mysql Import to support removing empty rows #4291

Closed swirtSJW closed 2 weeks ago

swirtSJW commented 1 month ago

Feature Description

As a data maintainer I would like the option to have the MySQL importer remove any empty rows immediately following the import. This should be an option because there may be some use cases where we want preservation of exactly what was provided in the CSV. The default on this option should default to off, so as not to introduce any changes.

Detailed Feature Description

When using the MySQL Import, provide an option to remove any empty rows.

Problem and Motivation

When harvesting CSVs, the chunk harvester would ignore any empty rows that it encountered. The MySQL Importer preserves any empty rows encountered. This can create a data change when switching from one method to the other.

Potential Benefits

Gives a bit more control over data managment.

Target Audience

Possible Implementation

Additional Context

No response

swirtSJW commented 1 month ago

image

swirtSJW commented 3 weeks ago

image