Closed angiehjort closed 5 years ago
confirm a fix
reopening now with a different error
clone dataset and branch from here https://github.com/geoinfobyran/ddf--sodertornsmodellen/tree/access-and-year-3-sv delete datapackage.json run validate-ddf -i
✅ getting an error message that makes sense now!
edit this file ddf--sodertornsmodellen/ddf--sodertornsmodellen--src/ddf--entities--tag.csv
tag,name,parent
geography,Geography,
education,Education,
education_by_levels,Education levels,education
employment,Employment,
population_by_4income_groups,Four income groups,income
income_by_gender,by gender,income
income,Income,
migration_by_gender,by gender,migration
migration,Migration,
migration_by_employment_by_gender,by gender,migration_by_employment
migration_by_employment,Migration — employed,
migration_by_education_by_gender,by gender,migration_by_education
migration_by_education,Migration — educated,
housing, Housing
cars, Cars
other, Other variables
put commas in the end of last 3 lines, thus making it valid but don't remove spaces before "Housing", "Cars" and "Other variables" run validate-ddf -i again
❌ getting empty error now! datapackage creation started... datapackage.json was NOT created: {}.
now remove spaces before "Housing", "Cars" and "Other variables" run validate-ddf -i again ❌ observe the process, i think root checking and resource processing happened twice. is that OK?
❌ see validation log, it looks weird again
loading generic content...generating ddfSchema...loading generic content...generating ddfSchema...
❌ run validate-ddf now, validator crashes, but manages to report that Validation was finished successfully! WTF?
Angies-MacBook-Pro-2:ddf--sodertornsmodellen angie$ validate-ddf
Error: Unexpected Error: column header mismatch expected: 2 columns got: 3 ]
at ParserStream.self.__transform (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/parser/parser_stream.js:138:48)
at ParserStream.__parseLineData [as __handleLine] (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/parser/parser_stream.js:65:18)
at /Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/parser/parser_stream.js:90:22
at asyncIterator (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/extended.js:13:17)
at Function.asyncEach (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/extended.js:31:10)
at ParserStream.__processRows (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/parser/parser_stream.js:88:18)
at ParserStream._parseLine [as _parse] (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/parser/parser_stream.js:190:26)
at ParserStream._flush (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/parser/parser_stream.js:255:18)
at ParserStream.prefinish (_stream_transform.js:137:10)
at emitNone (events.js:106:13)
Validation was finished successfully.
/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/async/dist/async.js:903
if (fn === null) throw new Error("Callback was already called.");
^
Error: Callback was already called.
at /Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/async/dist/async.js:903:32
at /Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/async/dist/async.js:3835:13
at db.fillCollection.err (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/lib/ddf-definitions/entity.js:32:17)
at ParserStream.csv.fromStream.on.on (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/lib/data/db.js:36:13)
at emitOne (events.js:121:20)
at ParserStream.emit (events.js:211:7)
at spreadArgs (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/extended.js:43:25)
at ParserStream.emit (/Users/angie/.nvm/versions/node/v8.15.0/lib/node_modules/ddf-validation/node_modules/fast-csv/lib/parser/parser_stream.js:314:17)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:139:11)
i retested everything ✅ ddf-validation@1.19.3
ddf-validation v1.19.0 node v8.15.0
clone dataset and branch from here https://github.com/geoinfobyran/ddf--sodertornsmodellen/tree/access-and-year-3-sv
delete datapackage.json run validate-ddf -i
observe that a new datapackage.json is created but there is no ddfSchema array inside it
also the content of validation log is strange
loading generic content...generating ddfSchema...loading generic content...generating ddfSchema...