search

Qwlouse / Findeco

GNU General Public License v3.0
7 stars 4 forks source link

Creating static Files for Searchengines #7

Open Qwlouse opened 11 years ago

Qwlouse commented 11 years ago

We really need to generate static pages to enable searchengines to scan the pages. At least we need to ensure that start page is well indexable.

justelex commented 11 years ago

pinae commented 10 days ago

I don't know how this should work since we don't want to generate files which are reachable for normal users.

Maybe we could use a api to supply the crawer with the information the user will get displayed if he enters a certain url.

pinae commented 11 years ago

We have RESTful URLs. Maybe that's enough for a crawler. We should test that first before diving into SE-APIs.

justelex commented 11 years ago

Our URLs do not work for indexing thus we are loading content only by ajax calls. This is not done by Crawlers. On the other Hand we need to talk about the fact that indexing may not be wanted....

pinae commented 11 years ago

If we don't want indexing we have to supply a robots.txt. I don't know the google crawler in detail but I think it might be capable of loading ajax.

Personally I think indexing would be a benefit.

Qwlouse commented 11 years ago

google actually offers a special instructions for webmasters to allow for crawling and indexing dynamic pages:

https://developers.google.com/webmasters/ajax-crawling/

This would require that we change our urls from 

hostname/#/foo.1

to 

hostname/#!/foo.1

and have our backend deliver some static pages with content if the crawler asks for the special url:

hostname/?_escaped_fragment_=/foo.1

I think this should be doable. So if we want this...

Qwlouse commented 11 years ago

As of yesterday we have changed the urls to hashbang in old browsers and without any # in new Browsers. So next we need to include the 

<meta name="fragment" content="!">

into our pages and make the backend return static pages for "the ugly url":

hostname/?_escaped_fragment_=/foo.1