Explore enabling Google to crawl the BugQuery website

[yonzarecki]

Adi insisted that enabling Google to crawl our site is very important.

In order to do that we'll need to create links to pages containing the question & answers based on the stack-trace. Several options for how to do that (discussed in our meeting) are:

• Create a menu of pages based on the exception type (can be tree structured x.y.z -> x then y then z)
• Sample different stack-traces from our db, and create pages in our site based on them.
• Save every request we receive to our site as a new page. We need to make sure we don't create the same page twice tho.

In this issue we should explore these options and find which ones we want to implement and when. We'll open new issues to options we want to implement.

[ZivIzhar]

Initial research results regarding google's crawlerbot with @rodedzats :

In order to help google crawl our site we should:

• After creating our static pages, we need to create a sitemap and upload it to Google Webmaster Tools

• Install Google Analytics

• submit our URL to google

• Share it via social medias (twitter, facebook, youtube, pinterest and especially Google+ (Ew))

• Advertise our site in comments and etc of existing blogs & sites

• Have a blog for our website and advertise our site in our blog (Google likes blogs)

Next up. research regarding the site's structure for optimal crawling.

[ZivIzhar]

Links for future references (will be updated during the research):

• Google's Search Engine Optimization's starter guide

• How to Create a Site Structure That Will Enhance SEO (by kissmetrics)

• Google's Steps to a Google-friendly site

• How to get Google to crawl your site faster

• GoogleBot Optimization (by kissmetrics)

[rodedzats]

How our site should look and what it should contain:

• Each html page of the site should contain a title inside a title tag within the head tag. Ideally there should be a unique, brief and descriptive title for each page.

• Each html page should contain a summarization of the page contents in a description meta tag. These descriptions should be unique. concise and accurate.

• The site urls should be simple to understand and contain only (or at least mostly) relevant English words. The urls should be organized in a simple directory structure.

• Make the site easy to navigate. The users should be able to get from the root page to all other pages easily, to do that we should use text for navigation and not pictures or drop down menus.

• The site should contain an HTML sitemap with links to the important pages.

• We should have a useful 404 not found page with link to the root page.

• Google may crawl our site slowly if our server is slow or if our site has too many errors. We should make sure the our server is fast and that all errors which occur, redirect to some valid url in our site.

[yonzarecki]

You should open issues regarding these steps so that they are not forgotten.

[rodedzats]

129 and #127 should suffice for now so I'm closing this issue. More issues about improving our site for Google may be opened in the future with this issue as reference.