crawler fail due to http 303 see other

asepaprianto / crawler4j

Automatically exported from code.google.com/p/crawler4j

0 stars 0 forks source link

crawler fail due to http 303 see other #327

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?
the output that says successful, I see Failed: HTTP/1.1 303 See Other instead

What version of the product are you using?
3.5

Please provide any additional information below.

I downloaded the sample code and tried to run it,but all I get in the console 
was Failed: HTTP/1.1 303 See Other.

thanks for any help.

Original issue reported on code.google.com by alskdw...@gmail.com on 16 Dec 2014 at 4:09

GoogleCodeExporter commented 9 years ago

I will be happy to test it for you.

I just need a scenario

Which site did you crawl which caused this error ?

Original comment by avrah...@gmail.com on 16 Dec 2014 at 9:28

Changed state: Accepted

GoogleCodeExporter commented 9 years ago

hello

I tried to crawl https://www.google.com/ and https://www.nytimes.com.

Original comment by alskdw...@gmail.com on 16 Dec 2014 at 10:23

GoogleCodeExporter commented 9 years ago

Ok I checked it and I admit that I found a bug there.

This bug is already solved in v4.0, so please, just download the latest version 
and use it.

If you are using maven then it is super simple, just change the version from 
v3.5 to v4.0

As can be seen here:
https://code.google.com/p/crawler4j/wiki/MavenConfig

Original comment by avrah...@gmail.com on 16 Dec 2014 at 11:19

GoogleCodeExporter commented 9 years ago

Please check it out and report if it works/doesn't work on the latest version

Original comment by avrah...@gmail.com on 16 Dec 2014 at 11:20

GoogleCodeExporter commented 9 years ago

thanks for your help, but where can I find the download link of v4.0?

Original comment by alskdw...@gmail.com on 16 Dec 2014 at 2:42

GoogleCodeExporter commented 9 years ago

There is no download link for v4.0

You need to build it using Maven:
https://code.google.com/p/crawler4j/wiki/MavenConfig

Maven will pull automatically all of crawler4j's dependencies...

Some resources about Maven:
http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
http://www.tutorialspoint.com/maven/
http://www.slideshare.net/holograph/maven-for-dummies

Original comment by avrah...@gmail.com on 16 Dec 2014 at 2:47

GoogleCodeExporter commented 9 years ago

Thanks, that was a lot of help, I will let you know if I find any problems. :)

Original comment by alskdw...@gmail.com on 16 Dec 2014 at 3:29

GoogleCodeExporter commented 9 years ago

Already fixed in v4.0

Original comment by avrah...@gmail.com on 22 Jan 2015 at 11:42

Changed state: Invalid