search

kartaview / upload-scripts

Uploader tools for KartaView
MIT License
61 stars 30 forks source link

upload_photos_by_exif.py too many files open[github version] #2

Closed james2432 closed 8 years ago

james2432 commented 8 years ago

I get the following error when uploading big datasets on Windows using python 3.4.1

C:\pyscripts\openstreetview>python upload_photos_by_exif3.py -p C:\pyscripts\openstreetview\photos4\0\
Found 3548 pictures to upload
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2044.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2045.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2046.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2047.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2048.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2049.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2050.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2051.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2052.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2053.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2054.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2055.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2056.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2057.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2058.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2059.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2060.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2061.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2062.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2063.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2064.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2065.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2066.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2067.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2068.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2069.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2070.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2071.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2072.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2073.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2074.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2075.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2076.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2077.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2078.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2079.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2080.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2081.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2082.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2083.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2084.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2085.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2086.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2087.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2088.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2089.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2090.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2091.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2092.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2093.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2094.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2095.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2096.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2097.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2098.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2099.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2100.jpg'
[Errno 24] Too many open files: 'C:\\pyscripts\\openstreetview\\photos4\\0\\virb0131-2101.jpg'
Traceback (most recent call last):
  File "upload_photos_by_exif3.py", line 122, in thread
  File "C:\Python34\lib\concurrent\futures\_base.py", line 395, in result
  File "C:\Python34\lib\concurrent\futures\_base.py", line 354, in __get_result
  File "C:\Python34\lib\concurrent\futures\thread.py", line 54, in run
  File "upload_photos_by_exif3.py", line 112, in upload_photos
  File "C:\Python34\lib\site-packages\requests\api.py", line 107, in post
  File "C:\Python34\lib\site-packages\requests\api.py", line 53, in request
  File "C:\Python34\lib\site-packages\requests\sessions.py", line 454, in request
  File "C:\Python34\lib\site-packages\requests\sessions.py", line 375, in prepare_request
  File "C:\Python34\lib\site-packages\requests\utils.py", line 95, in get_netrc_auth
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1467, in exec_module
  File "<frozen importlib._bootstrap>", line 1571, in get_code
  File "<frozen importlib._bootstrap>", line 1623, in get_data
OSError: [Errno 24] Too many open files: 'C:\\Python34\\lib\\netrc.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "upload_photos_by_exif3.py", line 402, in <module>
  File "upload_photos_by_exif3.py", line 382, in main
  File "upload_photos_by_exif3.py", line 142, in thread
TypeError: a float is required
james2432 commented 8 years ago

is fixed with leaving less files open. closing