Want better control to customize app configs

[mhussaincov]

hi there, thanks for lnc!

i'm new to this issue creation thingy, my thoughts/requests could we have a config file for lnc in our user folder? e.g here on linux, ./config/lnc/ config.ini etc? format could be as folows: format= options could be web, txt, outputtdirectory= get_chapters=all and a setting to make one file for the novel?

could a setting be added where in urls from a file will be downloaded? maybe the -i switch?

supported ones that is? also could we have a sleep time setting? how long the app will wate before getting the next chapter, we would like to be polite to website providers write? :) also less chance of the ip adress being blacklisted, or previders requesting removal of the source? due to accessive get requests?

maybe have a setting where you are able to change useragent? make the app look like say the firefox browser? it would make it stand out alot less? also some sites have a mobile version of the site may help in adding sources? I hope some of this made some sence, Majid Hussain

[mhussaincov]

hey there again, would it be possible for lnc to have one file for the hole story? exammple one html file etc? currently lnc downloads html separately?

Majid

On 07/02/2020, mhussaincov notifications@github.com wrote:

hi there, thanks for lnc!

i'm new to this issue creation thingy, my thoughts/requests could we have a config file for lnc in our user folder? e.g here on linux, ./config/lnc/ config.ini etc? format could be as folows: format= options could be web, txt, outputtdirectory= get_chapters=all and a setting to make one file for the novel?

could a setting be added where in urls from a file will be downloaded? maybe the -i switch?

supported ones that is? also could we have a sleep time setting? how long the app will wate before getting the next chapter, we would like to be polite to website providers write? :) also less chance of the ip adress being blacklisted, or previders requesting removal of the source? due to accessive get requests?

maybe have a setting where you are able to change useragent? make the app look like say the firefox browser? it would make it stand out alot less? also some sites have a mobile version of the site may help in adding sources? I hope some of this made some sence, Majid Hussain

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dipu-bd/lightnovel-crawler/issues/331

-- kind regards, Majid Hussain

[dipu-bd]

Let me summarize what you want:

• Load configs from a file
• config to set concurrent request count
• config to set custom user agent
• improve html binder: create single point entry

I am currently in the middle of restructuring the project. It will be released as v3. Until it gets released v2 will not have new features. I will keep in mind to give control of as much config as possible to the user in v3.

[mhussaincov]

hi there, correct! I was not aware that there was a reconstruction in progress, sorry :( in the output formats, would it be possible to have a single html document for the hole novel? setting for v3? Majid

On 07/02/2020, mhussaincov notifications@github.com wrote:

hey there again, would it be possible for lnc to have one file for the hole story? exammple one html file etc? currently lnc downloads html separately?

Majid

On 07/02/2020, mhussaincov notifications@github.com wrote:

hi there, thanks for lnc!

i'm new to this issue creation thingy, my thoughts/requests could we have a config file for lnc in our user folder? e.g here on linux, ./config/lnc/ config.ini etc? format could be as folows: format= options could be web, txt, outputtdirectory= get_chapters=all and a setting to make one file for the novel?

could a setting be added where in urls from a file will be downloaded? maybe the -i switch?

supported ones that is? also could we have a sleep time setting? how long the app will wate before getting the next chapter, we would like to be polite to website providers write? :) also less chance of the ip adress being blacklisted, or previders requesting removal of the source? due to accessive get requests?

maybe have a setting where you are able to change useragent? make the app look like say the firefox browser? it would make it stand out alot less? also some sites have a mobile version of the site may help in adding sources? I hope some of this made some sence, Majid Hussain

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dipu-bd/lightnovel-crawler/issues/331

-- kind regards, Majid Hussain

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dipu-bd/lightnovel-crawler/issues/331#issuecomment-583398648

-- kind regards, Majid Hussain

[dipu-bd]

@mhussaincov oh, I didn't catch the single html request you have asked. I added it to the list above.

[mhussaincov]

hey there, my thanks :)

as they say in some of them web novels, your dao of programming is verry high sir! it is so high, that i'm sure i'm a frog in a well that can't see the imencities of heaven and earth, why your level of programming to me rezemble the towering hightes of mt ti!

On 08/02/2020, Sudipto Chandra notifications@github.com wrote:

@mhussaincov oh, I didn't catch the single html request. I added it to the list above.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/dipu-bd/lightnovel-crawler/issues/331#issuecomment-583732998

-- kind regards, Majid Hussain

[ghus3rz]

Hi there .. was looking at your project and wow I think you has been very responsive on the features. Putting this here first as you are already talking about customization and v3 but if need to be in new post, just tell me to move this.

Am wondering if the output file name could be further customized using flexible arguments? I saw in v2.18.0 release, there is now option to do below

[console] Add two more argument options:
    --filename NAME: Set the output file name
    --filename-only: Skip appending chapter range with file name

but I believe the specific NAME has to be manually inputted. Would this option can further be customized with something like

%a=Author %n=Storyname %c=numberofchapters %d=updatedDate %w=sitename

It would help with batch download.

Thanks a lot.

[dipu-bd]

@ghus3rz nice suggestion. I will try to implement it. If you have more suggestions, you can use this thread to discuss about v3: #336

[mhussaincov]

hey there, why not get filename from title of novel? or url of novel? quick question, does current light novel crawler update stories after download? Majid

On 08/02/2020, Majid Hussain mhussaincov93@gmail.com wrote:

hey there, my thanks :)

as they say in some of them web novels, your dao of programming is verry high sir! it is so high, that i'm sure i'm a frog in a well that can't see the imencities of heaven and earth, why your level of programming to me rezemble the towering hightes of mt ti!

On 08/02/2020, Sudipto Chandra notifications@github.com wrote:

@mhussaincov oh, I didn't catch the single html request. I added it to the list above.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/dipu-bd/lightnovel-crawler/issues/331#issuecomment-583732998

-- kind regards, Majid Hussain

-- kind regards, Majid Hussain

[dipu-bd]

by default, the app gets filename from title of novel. if you do not delete the existing output folder, it will not re-download already downloaded chapters

[lamahmud]

Just to add my 2 cents and +1 this request, what I'd love to see out of the config is the ability to specify in the config the root folder for downloads (example: ~/Lightnovels/) and have the novel download without the web path to that folder. Example, right now if I download a book "The Great Thief", the path shows as: "~/Lightnovels/Lightnovels/www.mywuxiaworld.com/The Great Thief" since I'm already in my root folder ~/LightNovels. What I'd like to happen is that the path for the download ends up as: "~/Lightnovels/The Great Thief" As it stands now, I have to adjust the path each time or append -o "~/Lightnovels/The Great Thief" to the download name. This is a pain for me because in order to make it easier to download using the console, I created an alias: alias Lnc='lncrawl --format epub --single --force --all --query'

So all I have to type is : Lnc "My Book Title" to start the query and download. I still have to choose the source and choose the path. I don't mind choosing the source from the menu since some sources aren't on the same chapters as others. It is a pain to have to correct the path each and every time though.

Thanks for the great work!

[dipu-bd]

Should be possible after implementing #1001