search

andrewxhill / MOL

The Map of Life
mol.colorado.edu/
19 stars 4 forks source link

loader.py won't bulkload on my computer #106

Closed gaurav closed 13 years ago

gaurav commented 13 years ago

As of @60bad99afb5, my loader.py has two problems I can't figure out:

  1. Running python loader.py (or python loader.py --url=http://localhost:8080/_ah/remote_api --config_file=bulkload.yaml) exits prematurely with the following error:

    INFO:root:All collection metadata saved to collection.csv.txt INFO:root:Collections in jetz: ['birds'] INFO:root:Processing 1 layers in the birds collection INFO:root:Extracting DBF fields from abeillia_abeillei.shp INFO:root:All collection metadata saved to collection.csv.txt Traceback (most recent call last): File "loader.py", line 265, in main() File "loader.py", line 242, in main filename = os.path.abspath('%s/%s/collection.csv.txt' % (sd, coll_dir)) NameError: global name 'coll_dir' is not defined

coll_dir in only defined inside the for collection in config.collections(): loop on line 101, so I think maybe the chunk of code referring to coll_dir might have fallen out of the loop at some point.

  1. If I run python loader.py -s iucn, the program exits without any error messages, but also without attempting bulkloading of the processed data (it appears to write the collection metadata to collection.csv.txt and exit). Running python loader.py --url=http://localhost:8080/_ah/remote_api --config_file=bulkload.yaml -s iucn produces exactly the same result. I'm not sure why it isn't attempting to bulkload the data.

Can anybody else reproduce this on their computers, or have I messed something up on my end?

eightysteele commented 13 years ago

Quick sanity check: What's your OS and Python version?

eightysteele commented 13 years ago

More sanity checking... I'm assuming you have the app running on http://localhost:8080 and that you're working off latest code in the master branch?

eightysteele commented 13 years ago

Just pulled latest from master and I can reproduce. @tucotuco: Bump..

gaurav commented 13 years ago

OS: MacOS X 10.6.8 Python: Python 2.6.1

Nope, I don't have the app running at all - I was hoping for some sort of "could not connect to the server" message. I just tried it again with the app on; I got exactly the same result I reported above.

Yep, I'm on the latest code: @60bad99afb5e.

eightysteele commented 13 years ago

With the latest commit @1bb1404 you can run it via ./loader.py -d which completes as a dry run without bulkloading. If you attempt to bulkload, it currently fails since polygonid isn't making it into collection.polygons.json.txt. Looking into that now.

eightysteele commented 13 years ago

Almost there. In @1bb1404 things are working except for that missing polygonid for iucn/mammals. Looking into that right now with @tucotuco.

eightysteele commented 13 years ago

Fixed and pushed. Can @tucotuco and @gaurav confirm?

tucotuco commented 13 years ago

Confirmed. jetz and iucn loading to local App Engine and maintaining polygonids.

On Thu, Aug 25, 2011 at 10:58 PM, eightysteele reply@reply.github.com wrote:

Fixed and pushed. Can @tucotuco and @gaurav confirm?

Reply to this email directly or view it on GitHub: https://github.com/andrewxhill/MOL/issues/106#issuecomment-1909824

eightysteele commented 13 years ago

Closing issue, pending feedback from @gaurav.

gaurav commented 13 years ago

Works for me! Thanks for fixing this!