Closed tucotuco closed 12 years ago
Yup, I'll bring the config.yaml
s into compliance once Rob and I have finalized the fields spec. For now, you can run loader.py -V
to turn off the validation, for instance if you just need to test the bulk upload. I'll add that note to the validation errors also.
Quick update: I've added the note to the validation option as promised. I've been working on bringing the config.yaml
s into compliance, but that requires getting halfway to the new config.yaml format we discussed a few weeks ago (since the current field spec specifically expects fields to work as per the new format, i.e. with a single set of fields which can be set at either the shapefile level or the layer level). I'm mostly done with the code in @305aedba2d; now I need to actually upgrade the config files. Should be finished soon, hopefully tomorrow!
Okay, I think this is fixed as of @1770a58318. I've glanced through the uploaded data and it looks okay to me, but I won't merge it into master until Monday, when I'll have had a chance to really make sure the DBF fields are being mapped correctly. But I'm pretty sure most things are working okay!
Because of the constraints of the field listings (which involves merging "Source" fields and "DBF" fields into fields divided only by their Required/Optional status), we're now halfway to @tucotuco's idea about have each field define default values, column names, and so on. I could work on that next week, if that makes sense.
+1 It would be nice to have that all wrapped up.
Okay, mapping and bulkloading are working for me as of @83c19169f71c. I'll close this issue and open another one for the new config.yaml
file format.
After a fresh install on Windows, running loader.py shows errors in the config.yaml file provided.
C:\Projects\MOL\workflow\mol-data>loader.py --url=http://localhost:8080/_ah/remote_api --config_file=bulkload.yaml INFO:root:Processing source directories: ['iucn', 'jetz']
ERROR:root: Unexpected fields found in section 'Collections:Required': collectionorigin, collectionpresence, collectionseasonality, collectionstartdate, contributor, coverage, creator, date, description, email, publisher, rights, scientificname, source, subject, title
ERROR:root: Unexpected fields found in section 'Collections:Optional': accessright, enddateaccuracy, format, identifier, language, relation, rights, startdateaccuracy, type
ERROR:root: Fields missing from section 'Collections:DBFMapping:Required': contributor, creator, format, pubdate, publisher, rights, title, type
ERROR:root: Unexpected fields found in section 'Collections:DBFMapping:Optional': contributor, dateend, datestart, infraspecificepithet, measurementcomment, polygonid
ERROR:root:'iucn\config.yaml', directory 'mammals' could not be validated. Please fix the errors reported above and retry.