search

CjangCjengh / NovelpiaDownloader

노벨피아 소설을 EPUB로 다운로드
20 stars 3 forks source link

Thread number #6

Open YukinoshitaYukino123 opened 9 months ago

YukinoshitaYukino123 commented 9 months ago

Can you increase the maximum number of thread (more than 20) because it seems 20 threads maximum chapters is 500

CjangCjengh commented 9 months ago

In fact, I suggest keeping the number of threads at 2 or 3. Your issue is actually caused by having too many threads, as frequent access to Novelpia can result in your IP being banned. You can try opening the Novelpia webpage, where you will see information about the ban.

YukinoshitaYukino123 commented 9 months ago

I think 2 or 3 threads only works for novels with few chapters (bellow 200 or 300) because after that (downloading more than 300 chapters) it will give error messages (which where I got IP banned). For example this novel https://novelpia.com/novel/81527 I can't get past chapter 200ish with 2 threads and 500ish with 20 threads

CjangCjengh commented 9 months ago

I have added an "interval time" feature. You can reduce the number of threads while increasing the interval time to mimic human-like access speed, thus avoiding being banned. Additionally, I have raised the upper limit of the thread count to 100, but I still do not recommend using an increased thread count as a method to download it, as this significantly increases the likelihood of being banned.

P.S. I attempted to download it using 1 thread with a 1-second interval, and succeeded.

YukinoshitaYukino123 commented 9 months ago

Thanks for the addition of interval now I'm able to download it too, and also like you said adding more threads didn't do much but using the interval it succeeded.

Also I'm curious can you add feature like starting from which chapter to download (like from chapter 0, chapter 165 etc) and how many chapter to be downloaded.

Once again thanks for the effort.

CjangCjengh commented 9 months ago

OK, done.

YukinoshitaYukino123 commented 9 months ago

Thanks, I've tested it several times already and no issues encountered.

Also if you have time can you added some QoL like option to exclude illustrations from epub result (because some novel has huge size of illustration for a chapter) and adding "회 number or EP. number" in front of the title chapter

curllz commented 8 months ago

Can you add a list of optimal settings to use? For example, Chapters 0-200 use X threads and X internal, and Chapters 200-500 use X threads and X internal. Or, instead, can you give settings to download chapters ranging from 0-500+? Is it better to download chapters in increments or all in one go?

curllz commented 8 months ago

When the novel is done downloading can you add the chapter number in the epub and txt files? Some novels include chapter numbers in the title, and others don't. Is there a way where you can check whether it does have it? If not, then it adds the chapter number in the title of each chapter. If it does, nothing is done.