search

veliovgroup / jazeee-meteor-spiderable

Fork of Meteor Spiderable with longer timeout, caching, better server handling
https://atmospherejs.com/jazeee/spiderable-longer-timeout
33 stars 9 forks source link

?_escaped_fragment_= in Google's view #16

Closed Buom01 closed 8 years ago

Buom01 commented 9 years ago

Hi,

I think that Spiderable should redirect crawlers from http://exemple.com/?_escaped_fragment_= to http://exemple.com/ with a 302 redirect. Why ? Else, google detect duplicate content, not SEO friendely :( If I say that, it's because google has also indexed my pages with ?_escaped_fragment_=

dr-dimitru commented 9 years ago

@jazeee What do you think? It is not problem to implement but do we have to? @Buom01 It is weird what Google indexes pages with ?_escaped_fragment_=, are you sure you are using urls with hashbang #!?

jazeee commented 9 years ago

I think it would be unusual to redirect unless someone like Google recommends this for this URL. You should also check your Google webmaster tools and site management settings. In that, you may have overridden the query suffix to be something other than the default escaped fragment. On Aug 21, 2015 11:54 AM, "dr.dimitru" notifications@github.com wrote:

@jazeee https://github.com/jazeee What do you think? It is not problem to implement but do we have to? @Buom01 https://github.com/Buom01 It is weird what Google indexes pages with ?_escapedfragment=, are you sure you are using urls with hashbang

!?

— Reply to this email directly or view it on GitHub https://github.com/jazeee/jazeee-meteor-spiderable/issues/16#issuecomment-133470535 .

Buom01 commented 9 years ago

Excuse me, I think to this yesterday, I would to say that my probleme was solved automaticaly Additiaonaly, I think that google want that webmasters keep _escaped_fragment_= in urls:

They explain that googlebot go on _escapedfragment= and automaticaly redirect users to #

And if google has indexed my home page in both version, it's because a directory of link has gived the wrong version to google. It's just that i'm impatient, google isn't stupid and now it's not in search results.

Sorry for inconvenience, now all is OK

javdl commented 8 years ago

And those two links are now deprecated: http://googlewebmastercentral.blogspot.nl/2015/10/deprecating-our-ajax-crawling-scheme.html

dr-dimitru commented 8 years ago

@Joostvanderlaan thank you for info. @jazeee due to this proposal - we shouldn't remove script tags from header. Everything else is same, as other crawlers still will use _escaped_fragment_=

jazeee commented 8 years ago

Agreed On Oct 22, 2015 8:09 AM, "dr.dimitru" notifications@github.com wrote:

@Joostvanderlaan https://github.com/Joostvanderlaan thank you for info. @jazeee https://github.com/jazeee due to this proposal - we shouldn't remove script tags from header. Everything else is same, as other crawlers still will use _escapedfragment=

— Reply to this email directly or view it on GitHub https://github.com/jazeee/jazeee-meteor-spiderable/issues/16#issuecomment-150215379 .

dr-dimitru commented 8 years ago

Can't implement this now, as page get rendered twice, - first on server then on client. So you will have on content duplicated. Anyone know the way to let Meteor know what page is already rendered?

javdl commented 8 years ago

@dr-dimitru even though other crawlers still will use _escapedfragment= I think it's a dead end street anyhow. For example; Twitter and Facebook can use Cards & OpenGraph spec. So for those you do not need escaped fragment either. Besides, @arunoda's Flow-Router will soon be moved into Meteor core. The coming version 4.0 of that router will support server side rendering (SSR) somewhere in January. SSR in turn will make the _escapedfragment= truly obsolete, cause with SSR also the crawlers which do not support JavaScript in the way Google now does will be able to crawl the pages.

jazeee commented 8 years ago

Agreed about Flow Router, etc. I don't think that we can eliminate escaped fragment until we truly have SSR. The reason is that there are other things that may depend on it, such as text readers for blind people. In the US, it is law that a website be accessible to blind readers. They would have to pass in this query to enable rendering.

Come January, it will go away, but still will need to support until people migrate to that version.

On Mon, Nov 2, 2015 at 2:45 AM, J notifications@github.com wrote:

@dr-dimitru https://github.com/dr-dimitru even though other crawlers still will use _escapedfragment= I think it's a dead end street anyhow. For example; Twitter and Facebook can use Cards & OpenGraph spec. So for those you do not need escaped fragment either. Besides, @arunoda https://github.com/arunoda's Flow-Router will soon be moved into Meteor core. The coming version 4.0 of that router will support server side rendering (SSR) somewhere in January. SSR in turn will make the _escapedfragment= truly obsolete, cause with SSR also the crawlers which do not support JavaScript in the way Google now does will be able to crawl the pages.

— Reply to this email directly or view it on GitHub https://github.com/jazeee/jazeee-meteor-spiderable/issues/16#issuecomment-152955419 .

javdl commented 8 years ago

@jazeee I agree :)

jazeee commented 8 years ago

I think this is probably closed.