search

VijayQin / DMHY-spider

This project aims at crawling on DMHY, and store the html and torrent of each animation in our local file systems and in database(SQLite3). The future work will be filtering animation we want by the given rules and alert us to those updated every day.
14 stars 2 forks source link

WindowsError [Error 3] is thrown because of folder path too long #3

Closed VijayQin closed 8 years ago

VijayQin commented 8 years ago

When downloading page(20:39 Aug 02 2016) https://share.dmhy.org/topics/view/438853_BDRIP_OVA_OVA_Sono_Hanabira_ni_Kuchizuke_wo_Anata_to_Koibito_Tsunagi_OVA_1920x1080_HEVC_10bit_FLAC_softSub_chi_jpn_eng_fre_ger_rus_ordered_chap.html a WindowsError [Error 3] is thrown

complete error message is following: Traceback (most recent call last): File "DMHY_DataBase.py", line 314, in

File "DMHY_DataBase.py", line 122, in start_requests self.parse_item(data[i], con) File "DMHY_DataBase.py", line 191, in parseitem os.makedirs(path) File "C:\Python27\lib\os.py", line 157, in makedirs mkdir(name, mode) WindowsError: [Error 3] : u'D:\Data\Desktop\Workspace\test\DMHY\Warehouse\2016\08\02\u5b63\u5ea6\u5168\u96c6\2039[BDRIP]\u82b1\u543b\u5728\u4e0a_\u4eb2\u543b\u90a3\u7247\u82b1\u74e3_\u604b\u4eba\u7684\u7f81\u7eca(OVA)\u305d\u306e\u82b1\u3073\u3089\u306b\u304f\u3061\u3065\u3051\u3092_\u3042\u306a\u305f\u3068\u604b\u4eba\u3064\u306a\u304e(OVA)_Sono_Hanabira_ni_Kuchizuke_wo__Anata_to_Koibito_Tsunagi(OVA)(1920x1080_HEVC_10bit_FLAC_softSub(chi+jpn+eng+fre+ger+rus)_ordered_chap)[\u7b80\u7e41\u4e2d\u65e5\u82f1\u6cd5\u5fb7\u4fc4\u5b57\u5e55]' image

fno2010 commented 8 years ago

This patch should fix the issue.

VijayQin commented 8 years ago

This error is caused by the limitation of path length in windows platform. The limitation seems to be 247. It has been fixed now.