search

WorldBrain / Legacy-Research-Engine

WorldBrain's Chrome Extension to full-text search through your browser history & bookmarks.
http://www.worldbrain.io
GNU General Public License v3.0
281 stars 65 forks source link

Indexing doesn't support AJAX requests #57

Open blackforestboi opened 7 years ago

blackforestboi commented 7 years ago

As this reddit comment suggests, so any modern single-page app or pagination won't be indexed.

This should change...

@obsidianart how can we do that?

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/40875971-indexing-doesn-t-support-ajax-requests?utm_campaign=plugin&utm_content=tracker%2F46814334&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F46814334&utm_medium=issues&utm_source=github).
blackforestboi commented 7 years ago

I saw this already for example with my asana, which I would have loved to see indexed, but doesnt work.

obsidianart commented 7 years ago

it is a little more complicated than that, since google have multiple engines and some do execute javascript and index your page: https://www.stephanboyer.com/post/122/does-google-execute-javascript I always advice against doing any ajax on a webpage because google penalise it, but it's not black and white and sometime you have no choice. I'm not sure about which page we are talking about and which server serves it (you normally render serverside the fetch, whatever is single-page app or not and react/angular have now universal compiler etc etc), but I would start from the basic question: did you register the website to google webmaster tool? does it say something there?