search

L4cache / kemono-dl

(personal fork of) A simple kemono.party downloader using python.
93 stars 14 forks source link

Duplicate file check #8

Closed slormo closed 5 months ago

slormo commented 1 year ago

Description

Kemono-DL sometimes duplicates attachments if any were previously downloaded. I think this is because the files change data servers and therefore get given a different name. A hash check might fix this but I'm not sure.

L4cache commented 1 year ago

I've noticed that too, but I gonna run dupe file scan for my drives occasionally, so I didn't look into that. Now I tried to fix it for us, hope it works well.

slormo commented 1 year ago

The commit seems to detect dupes well, but now it runs really slow. Making it an optional setting might be a good idea

L4cache commented 1 year ago

The commit seems to detect dupes well, but now it runs really slow. Making it an optional setting might be a good idea

Just to say that the hash compare only takes place when no file with same name found and files with similar filename detected, other wise there's just some filename compare, shouldn't be a huge slow down, there may be some other problems.