-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
```
What steps will reproduce the problem?
1. Enable "Include Sitemap URL in robots.txt"
2. Verify robots.txt file
3. added line:
http://www.mysite.com/sitemap.xml/gz # Added by Google Sitemap Genera…
-
### Repro steps
When opening the application and attempting to crawl a site using ssl (HTTPS). I am getting this error in logs;
WARNING: QSslSocket: cannot resolve EVP_PKEY_base_id
WARNING: QSs…
-
Thanks for making the crawler4j application.
It is indeed complete and useful.
What i am looking for is a RSS crawler which can take out the links from RSS and get new ones in a periodic manner.
But t…