EscVM / OIDv4_ToolKit

Download and visualize single or multiple classes from the huge Open Images v4 dataset
GNU General Public License v3.0
809 stars 635 forks source link

Unable to download from OIDv5 #58

Open alessiomatt1 opened 5 years ago

alessiomatt1 commented 5 years ago

Hi! With the new version of OID it is impossible to download the images, thinking that this memory error is due to this incompatibility?

WIN10 PYTHON 3.7.4

[INFO] | Downloading Handgun. Traceback (most recent call last): File "main.py", line 37, in <module> bounding_boxes_images(args, DEFAULT_OID_DIR) File "C:\Users\aless\Desktop\YOLOv3_GUN\Tool\OIDv4_ToolKit-master\modules\bounding_boxes.py", line 60, in bounding_boxes_images df_val = TTV(csv_dir, name_file, args.yes) File "C:\Users\aless\Desktop\YOLOv3_GUN\Tool\OIDv4_ToolKit-master\modules\csv_downloader.py", line 21, in TTV df_val = pd.read_csv(CSV) File "C:\Users\aless\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 685, in parser_f return _read(filepath_or_buffer, kwds) File "C:\Users\aless\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 463, in _read data = parser.read(nrows) File "C:\Users\aless\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 1154, in read ret = self._engine.read(nrows) File "C:\Users\aless\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 2059, in read data = self._reader.read(nrows) File "pandas\_libs\parsers.pyx", line 881, in pandas._libs.parsers.TextReader.read File "pandas\_libs\parsers.pyx", line 924, in pandas._libs.parsers.TextReader._read_low_memory File "pandas\_libs\parsers.pyx", line 2165, in pandas._libs.parsers._concatenate_chunks File "<__array_function__ internals>", line 6, in concatenate MemoryError: Unable to allocate array with shape (14610229,) and data type object

owlisie commented 4 years ago

Have same error here. But I have this error with OID v4 not v5. Am I the only one who can't download any images from OID v4? I've followed the exact guideline.

python main.py downloader --classes Apple Orange --type_csv all Windows 10 Python 3.7.x

JeroenAppel commented 4 years ago

I ran in to the same issue some weeks ago. In the end, I was able to resolve this by using a Ubuntu docker container on my Windows machine. It seems like the memory allocation on a Windows machine is (in some cases) less flexible compared to Unix based systems.

owlisie commented 4 years ago

@alessiomatt1 Just solve the problem. In my case, I happened to run this code in x32 version of python not x64. After I removed the x32 exe and run the code in x64 python, problem solved! Though its super slow. x32 Python allows only 4GB of RAM memory. Check you python first and RAM memory second. Good luck!

github2016-yuan commented 4 years ago

Have same error here. But I have this error with OID v4 not v5. Am I the only one who can't download any images from OID v4? I've followed the exact guideline.

python main.py downloader --classes Apple Orange --type_csv all Windows 10 Python 3.7.x

Sadly, I ran into the problem too. I follow all the steps. In Pycharm, I use the command line with params. I got error that: columns, rows = os.get_terminal_size(1) OSError: [WinError 6] 句柄无效。

Win10 Pycharm Python3.8.1