search

VertNet / gulo

Shredding Darwin Core Archives with ferocity, strength, and Cascalog.
7 stars 5 forks source link

harvest failing with null pointer exceptions #63

Closed robinkraft closed 11 years ago

robinkraft commented 11 years ago

Examples

http://ipt.vertnet.org:8080/ipt/resource.do?r=uafmc_fish

"Done harvesting" "nysm_mammals"
NullPointerException   clojure.lang.Numbers.ops (Numbers.java:942)

This also failed at uam_bird (i.e. after uamobs_mamm).

Workaround

It's unclear yet whether this is happening because of something about the code or the resources themselves. The workaround is to modify the resources sql with WHERE cartodb_id > n, where n is the last resource to process successfully (e.g. nysm_mammals).

laurarussell commented 11 years ago

let me know if you need me to examine resources for issues. Just looked at uafmc_fish and it is missing an occurrenceID (id field in occurrence.txt) so that may be the issue on that one, but uam_bird does not have that issue, but perhaps has another. I can work on correctly uafmc_fish so that it has an occurrenceID.

robinkraft commented 11 years ago

Thanks @laurarussell. This seems to happen at the end of harvesting, even with only five resources. I'll do some major debugging on this once we've got the full workflow tested with five resources.