webwhiz-ai / webwhiz

WebWhiz allows you to create an AI chatbot that knows everything about your product and can instantly respond to your customer's queries.
https://www.webwhiz.ai/
GNU Affero General Public License v3.0
882 stars 148 forks source link

Content fails to be crawled #112

Open jekyll-one opened 8 months ago

jekyll-one commented 8 months ago

Hi there, for my new account (created on 2023-10-21), I've setup a first project:

The crawler for WebWhiz fails completely and no pages were added. A fist crawl seems to work as no issues were reported. When I check the data that has been crawled so far, I got the error message:

Page not found 404

The page you requested could not found. Sorry about that, this shoud not happen.

If I reopen my project, the error message:

Unable to create chatbot. Please try again or contact support

is issued.

I'm using ScreamingFrogSEOSpider to check my site. The crawler of this product reports no issues and all pages gets successfully crawled.

NOTE:: Directory listing is disabled for the Webserver to serve the site.

Thanks for any help, regards Juergen