DWebYVR / localhost_vancouver_webring

Lieu-powered DWebYVR webring
4 stars 10 forks source link

Javascript sites aren't indexed #18

Open bmann opened 3 months ago

bmann commented 3 months ago

@graham-fleming added his site https://grahamfleming.com

It is included in the pre-crawl, but then doesn't appear to be in the search index.

bmann commented 3 months ago

I tried running everything locally and it also doesn't index locally

I messaged the lieu author to see if he has any tips https://toolsforthought.social/@boris/112832083740822099

bmann commented 3 months ago

https://merveilles.town/@cblgh/112832523291344860

look at the output from ’lieu crawl’, grepping it for the domains you think are missing. likely that they are js-generated, lieu doesn’t have a web browser embedded (like selenium does) and so can’t evaluate js statments atm