GetDKAN / dkan2

Deprecated: please use the 2.x branch of the dkan repo
https://github.com/GetDKAN/dkan
GNU General Public License v2.0
18 stars 16 forks source link

Some dummy content CSV files cause errors #272

Open thierrydallacroce opened 4 years ago

thierrydallacroce commented 4 years ago

Steps to reproduce:

  1. dktl install (from master)
  2. dktl drush dkan-dummy-content:create
  3. dktl drush cron
  4. dktl drush dkan-datastore:list
  5. dktl drush dkan-harvest:list

Will output errors such as:

[error]  Import for 144f86b3-9828-556a-be7c-9331b9843dc3 returned an error: Machine did not halt
[error]  Drupal\Core\Database\IntegrityConstraintViolationException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '53' for key 'PRIMARY': INSERT INTO {jobstore_dkan_datastore_importer} (ref_uuid, job_data) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1); Array
(
  [:db_insert_placeholder_0] => 53
  [:db_insert_placeholder_1] => 
)
in Drupal\Core\Database\Connection->handleQueryException() (line 689 of /Users/thierry.dallacroce/Code/dkan/docroot/core/lib/Drupal/Core/Database/Connection.php).
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '53' for key 'PRIMARY': INSERT INTO {jobstore_dkan_datastore_importer} (ref_uuid, job_data) VALUES (:db  
_insert_placeholder_0, :db_insert_placeholder_1); Array                                                                                                                        
(
  [:db_insert_placeholder_0] => 53                                                                                                                                           
  [:db_insert_placeholder_1] => {"parser":{"@type":"object","@class":"CsvParser\\Parser\\Csv","data":{"delimiter":",","quote":"\"","escape":"\\","recordEnd":["\n","\r"],"r  
  ecords":[],"fields":[],"field":"","machine":{"currentStates":["s_new_field"],"halted":false,"machines":[]}}},"identifier":"53","result":null,"timeLimit":9223372036854775807}  
)

I diagnosed the issues to the following dummy content CSV files:

  1. docroot/profiles/contrib/dkan2/modules/custom/dkan_dummy_content/files/district_centerpoints_0.csv has right-to-left characters
  2. docroot/profiles/contrib/dkan2/modules/custom/dkan_dummy_content/files/data_0.csv has Windows line endings
thierrydallacroce commented 4 years ago

This findings might reveal bugs or limitations within GetDKAN/csv-parser