How to crawl web pages like *.do?

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. I actually have one web page which looks normal like a) 
http://www.website.com and it is redirected to another page, and it looks like 
b) http://www.anotherwebsite.com/sub1/sub2/newpage.do?p=72&ver=2. Could I crawl 
this kind of web page and its sub-pages? I tried put both a) and b) in the 
sample code but none of them can work.
2.
3.

What is the expected output? What do you see instead?
The webpages from *.do?XXX. What I see is zero.

What version of the product are you using?
V3.3

Please provide any additional information below.

Original issue reported on code.google.com by shawn.zh...@gmail.com on 11 May 2012 at 4:09

GoogleCodeExporter commented 9 years ago

I am facing the same issue. Please help.

Original comment by menakshi...@gmail.com on 10 Oct 2012 at 10:09

GoogleCodeExporter commented 9 years ago

Please provide an example URL so I can test this scenario

Original comment by avrah...@gmail.com on 11 Aug 2014 at 1:52

GoogleCodeExporter commented 9 years ago

Original comment by avrah...@gmail.com on 18 Aug 2014 at 3:19

Changed state: Accepted

GoogleCodeExporter commented 9 years ago

Closed due to inactivity and no good scenario

Original comment by avrah...@gmail.com on 23 Sep 2014 at 2:02

Changed state: Invalid

xrma / crawler4j

How to crawl web pages like *.do? #153