Handle long path or filename problem

VijayQin / DMHY-spider

This project aims at crawling on DMHY, and store the html and torrent of each animation in our local file systems and in database(SQLite3). The future work will be filtering animation we want by the given rules and alert us to those updated every day.

14 stars 2 forks source link

Handle long path or filename problem #4

Open fno2010 opened 8 years ago

fno2010 commented 8 years ago

The older Windows version may have limited length of path and filename for NTFS. Windows 7 cannot support the length of path and filename exceeds 260 chars. Trying to proceed long path will lead to issue #3.

To handle this problem, there are several potential solutions:

Use UUIDs to store the path and link them to items in the DataBase. (Unreadable but maintainable)
Force to cut long filename. (Maybe readable)
Extract keywords and regenerate filename automatically. (Readable but hard to implement)

VijayQin commented 8 years ago

OK, I test the path limit practically, and I get the limit is 247. And I had fixed this problem by the solution 2. 3 is better. Maybe it will appear in the later version

fno2010 commented 8 years ago

Sure. We can talk about this problem later. Actually I don't think No.2 is a good solution. Some keywords in the title may be cut and make it unfriendly to read.

Personally I prefer No.1 and No.3:

No.1 is a really maintainable solution. Many package and repo managers are using this solution.
No.3 is related to Smart Filter functionality. And I think we will touch its technical details in the near future.

VijayQin commented 8 years ago

Yep~No.3 is relate to our ultimate goal. Dealing with it will be an import step definitely.