File Permission issue creating bulkload csv files

[tucotuco]

On Windows, running loader.py throws an error creating csv files:

C:\Projects\MOL\workflow\mol-data>loader.py --url=http://localhost:8080/_ah/remo te_api --config_file=bulkload.yaml -V INFO:root:Processing source directories: ['iucn', 'jetz'] INFO:root:Collections in iucn: ['mammals', 'amphibians'] INFO:root:Processing 1 layers in the iucn/mammals INFO:root:Extracting DBF fields from abditomys_latidens.shp Traceback (most recent call last): File "C:\Projects\MOL\workflow\mol-data\loader.py", line 427, in main() File "C:\Projects\MOL\workflow\mol-data\loader.py", line 422, in main source2csv(sd, options) File "C:\Projects\MOL\workflow\mol-data\loader.py", line 289, in source2csv dr = csv.DictReader(open(csvfile, 'r'), skipinitialspace=True) IOError: [Errno 13] Permission denied: 'abditomys_latidens.shp.csv'

[gaurav]

Hmm, I can't reproduce this on our Windows 7 computer here. John: Which version of Windows are you using? Did you use Git to clone the repository to your hard drive, or did you download the source as a tarball/zip from Github?

[tucotuco]

That was on a Windows XP machine (not currently in hand). I cloned from GitHub. Since you were unable to reproduce it, let me do a git pull and see what the current status is before you spend any further time on it.

[tucotuco]

OK, I pulled the latest revision and tried running

loader.py --url=http://localhost:8080/_ah/remote_api --config_file=bulkload.yaml -V

and there remains a problem. A new directory iucn/mammals/abditomys_latidens.shp.csv gets created with a file abditomys_latidens.csv inside. Very strange.

[gaurav]

Huh! That is odd. I'll try to reproduce that tomorrow.

[gaurav]

Hmm, I still can't reproduce it (at @75f02f2feb21). I'm on Windows 7 Home Premium with Python 2.7.2. I thought that perhaps it was something to do with spaces in the folder path, so I tried out the command which failed for you in both C:\Users\Gaurav\Desktop\Code\MOL and C:\Users\Gaurav\Desktop\Folder With Spaces In It\Map of Life, but both worked perfectly, and definitely without a new folder being created inside workflow/iucn.

One theory: we use shlex.split to split up command lines before passing them on to subprocess.call; I wonder if there's some weird symbols in your filenames (such as '<', '>', '"') which might be throwing shlex.split off. It's a bit of a long shot, but it's the only thing I can think of. If it's not that, I'll try hunting for a Windows XP/Vista computer.

[codyschank]

Tested on XP machine today and successfully uploaded shapefiles with loader.py.

[tucotuco]

OK, let's call it a corner case on Tuco's wonky XP machine and let it go? Think there are more valuable things to do than chase this one further.