Some novels have multiple translator groups and the links for some of those groups are not crawlable. In particular, the crawler miserably fails and the process hangs for a long time whereas we could simply customize the groups that are known to have bad links or uncrawlable links.
As such, I'd suggest one of the following features:
[ ] Add a blacklist for links that are known to be dead. If the 'final' link (after resolution) matches one of the blacklisted patterns, we can ignore it (we don't even try to crawl anything). We can have a default blacklist and a user-defined one that would just contain regex patterns similar to a .gitignore.
[ ] Add the possibility to explicitly request a NU translator group. The group ID should be included in the AJAX request sent for the list of chapters using mygrpfilter=... parameter. The user should be able to provide the translation group name or ID directly by the command-line and the crawler should first find the ID of the group if needed.
[ ] Add a plugin logic where people could just filter chapter information before actually processing the chapter. Currently, both the link resolution and the crawling process appear to be done in the same future. However, they should probably be done separately so that we can decide whether the link is acceptable or not.
Some novels have multiple translator groups and the links for some of those groups are not crawlable. In particular, the crawler miserably fails and the process hangs for a long time whereas we could simply customize the groups that are known to have bad links or uncrawlable links.
As such, I'd suggest one of the following features:
.gitignore
.mygrpfilter=...
parameter. The user should be able to provide the translation group name or ID directly by the command-line and the crawler should first find the ID of the group if needed.