search

yasserg / crawler4j

Open Source Web Crawler for Java
Apache License 2.0
4.53k stars 1.93k forks source link

Implement WebCrawler as an abstract class #28

Open wjdhamilton opened 9 years ago

wjdhamilton commented 9 years ago

Mr Ganjisaffar,

I think that it would be better if WebCrawler was an abstract class rather than a concrete one, just so that it is then clear to the developer which methods can/should be overwritten by default.

Yours faithfully,

James Hamilton

laxika commented 9 years ago

+1

rzo1 commented 9 years ago

+1

Chaiavi commented 9 years ago

The thing is that there are many methods which could be overwritten In this class, many of which you don't want to override and don't want to force the user to override. On Jul 20, 2015 20:46, "rzo1" notifications@github.com wrote:

+1

— Reply to this email directly or view it on GitHub https://github.com/yasserg/crawler4j/issues/28#issuecomment-122965519.

rzo1 commented 9 years ago

I think the main problem is, that the end-user have to investigate the source code to find out, which methods he can probably override for his own purpose. Maybe providing some documentation outside the code to get an idea of what can be overridden would be of great value for a developer, who do not want to read the original source code.

Chaiavi commented 9 years ago

Good documentation is a very good thing.

This is an open source project.

I invite you to have a first stab for the documentation, others will continue the documenting work.

On Mon, Jul 20, 2015 at 9:28 PM, rz notifications@github.com wrote:

I think the main problem is, that the end-user have to investigate the source code to find out, which methods he can probably override for his own purpose. Maybe providing some documentation outside the code to get an idea of what can be overridden would be of great value for a developer, who do not want to read the original source code.

— Reply to this email directly or view it on GitHub https://github.com/yasserg/crawler4j/issues/28#issuecomment-122977290.

rzo1 commented 9 years ago

Thanks for the invite. I am currently using crawler4j in a distributed way (with hazelcast and JPPF).

So I think, I can do some documentation (in a few days over the summer term) and provide some more example of crawler4j usage as well.

Chaiavi commented 9 years ago

That will be truly great.

Thanks

On Mon, Jul 20, 2015 at 9:36 PM, rz notifications@github.com wrote:

Thanks for the invite. I am currently using crawler4j in a distributed way (with hazelcast and JPPF).

So I think, I can do some documentation (in a few days over the summer term) and provide some more example of crawler4j usage as well.

— Reply to this email directly or view it on GitHub https://github.com/yasserg/crawler4j/issues/28#issuecomment-122981284.