Taskqueue new layer cleanup - Githubissues

andrewxhill / MOL

The Map of Life

mol.colorado.edu/

19 stars 4 forks source link

Taskqueue new layer cleanup #7

Closed andrewxhill closed 13 years ago

andrewxhill commented 13 years ago

SHP/SHX/Etc for an id should be cleaned up following a run

removed from the watched folder put in the permanent home of shp data, what can become our ftp directory down the road or, put in an error bin for later review

Error data should be stored someplace as well, open a new ticket if you think necessary.

eightysteele commented 13 years ago

I've started work on this. Here's the latest (in progress): https://github.com/andrewxhill/MOL/commit/bd3d98a038c712000a778976ef105e5625fd7b9a

eightysteele commented 13 years ago

Added some unit testing for Layer.cleanup(): https://github.com/andrewxhill/MOL/commit/13901f50d25eacb9bcebf2d8d09b227cfc5c6dcb

Also cleanup up the remote filesystem organization a bit: https://github.com/andrewxhill/MOL/commit/e64d10f283c5f8aeebb2c17db108c7040382de82

eightysteele commented 13 years ago

This stuff is ready for review. Changing issue status from 'Status-Started' to 'Status-InReview'.

andrewxhill commented 13 years ago

Again, if I'm using the keys literally to mean directories and then a file animalia/species/baetodes_noventus.shp does that appear to affect this pipeline portion? I'm fairly sure that the isdir and isfile checks are done after parsing out the components of the string, so i think no prob right?

eightysteele commented 13 years ago

Yeah, I'll review and see what's up here.

eightysteele commented 13 years ago

Layer.cleanup function is modified to work with the new key_name architecture. Changes pushed. Closing issue.