unyt-org / uix

The UIX Framework
https://uix.unyt.org
MIT License
54 stars 3 forks source link

Crawler issues #126

Closed jonasstrehle closed 3 months ago

jonasstrehle commented 4 months ago

After investigation it turns out that the googlebot and other search machine crawlers are getting back the invalid session content after our change to session initialization:

<html>
    <head>
        <noscript>Please activate JavaScript in your browser!</noscript>
        <script type="module" src="https://cdn.unyt.org/uix@0.2.x/src/session/init.ts"></script>
        <title></title>
    </head>
    <body></body>
</html>

Either we should consider to let the server generate the endpoint id in beforehand and make the client create its keys after loading or we should have an option for search machine to pass this check by using some basic regex such as

/bot|googlebot|crawler|spider|robot|crawling/i

Might also be an option to allow for title, description and og preview properties in app.dx used by the main page or search engine and add this information to the "Please activate JavaScript in your browser!" content.