dkan_harvest fails to handle long keyword value.

[dafeder]

CIVIC-5070

Trying to migrate a source that has mistakenly created kewords that are too long causes the a migration to fail with the following error:

QLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'name' at row 1: INSERT INTO {taxonomy_term_data} (vid, name, uuid) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2); Array ( [:db_insert_placeholder_0] => 2 [:db_insert_placeholder_1] => alcohol abuse, alcohol consumption, amphetamines, barbiturates, cocaine, demographic characteristics, drug abuse, drug use, drugs, hallucinogens, health care, health insurance, heroin, households, inhalants, marijuana, mental health, mental health services, methamphetamine, offenses, patients, prescriptions drugs, sedatives, smoking, stimulants, substance abuse, substance abuse treatment, tobacco use, tranquilizers, treatment, youths [:db_insert_placeholder_2] => a8860a1e-c2c2-4c7b-a169-7dc0e3dd1bdc ) (/var/www/dkan_demo/docroot/includes/common.inc:7376)

Steps to Reproduce

• Migrate the following source to see error: https://s3-us-west-1.amazonaws.com/nucivic-binaries/data.json

Acceptance Criteria

• If a source has a keyword larger than the max character limit, the dataset should be skipped and an error logged

Test Updates

• Add PHPUnit test for this scenario.

Initially reported by @dkinzer

[dafeder]

From @jacintocapote:

I will try to explain the error and because is a little complicate to fix. We can have from multiple fields this error (long value). I was investigating as capture the error to process and show a error but is a little difficult. For example for title, description, .... we need to do a call API drupal to get the schema of these fields. After get the schema we need to get the type (varchar, int, ...) and get the long of these fields and compare with the field mapped. In another case we can limit the to a incorrect size. We can limit the keyword and log a error but I think we need implement a general solution and is the reason to be more complicate and I spend the last days checking a solution.

[dafeder]

We're going to stick to the acceptance criteria for now. Specific to the keyword field, if the keyword is too long, flag a bug and skip it.

[acouch]

Closing this ticket as it has not been updated recently and is not on our near-term roadmap. Please re-open if you would like to discuss further or are actively working on it.