search

cozydev-pink / protosearch

prototype search library in pure scala
https://cozydev-pink.github.io/protosearch/
Apache License 2.0
9 stars 5 forks source link

Refactor plugin with IndexRendererConfig #200

Closed valencik closed 3 months ago

valencik commented 3 months ago

This PR adds IndexRendererConfig which is a convenience builder as described by @jenshalm in https://github.com/typelevel/Laika/pull/588 This relies on new features added in that PR that are not yet released, so we switch to a Laika snapshot temporarily.

Additionally we bump to the latest Lucille version to avoid a cats-parse 0.3.10 vs 1.0.0 dependency issue.

We also update the contributing guide, resolving https://github.com/cozydev-pink/protosearch/issues/199

valencik commented 3 months ago

@jenshalm In https://github.com/cozydev-pink/protosearch/pull/200/commits/19ab55259862ea3dadf04104afad7bcbe7d6f0cf I've tried to simplify the plugin using the new extendWith(SearchUI) and IndexRendererConfig. The searchIndex.idx file is created when running docs/tlSitePreview and can be served from the preview server. However the remaining html and js files are not created. I'm guessing I'm doing the extendWith(SearchUI) bit incorrectly. Do you have any ideas?

My hope is that I can maintain a single line addition to a project's plugins.sbt file to test protosearch. This isn't how I imagine real users integrating protosearch into their build, it's just very convenient for testing on different projects without learning where they've setup their Laika config.

jenshalm commented 3 months ago

@jenshalm In 19ab552 I've tried to simplify the plugin using the new extendWith(SearchUI) and IndexRendererConfig. The searchIndex.idx file is created when running docs/tlSitePreview and can be served from the preview server. However the remaining html and js files are not created. I'm guessing I'm doing the extendWith(SearchUI) bit incorrectly. Do you have any ideas?

What you committed should not actually compile, as it misses the final .build step. Otherwise I never use the recursive foo := foo.value... syntax with sbt, but that's mostly because I don't know whether that's valid. What should definitely work is either laikaTheme ~= _.extendWith(SearchUI).build or laikaTheme := Helium.defaults.extendWith(SearchUI).build.

valencik commented 3 months ago

Hey @jenshalm, sorry, I had tried laikaTheme := Helium.defaults.extendWith(SearchUI).build as in https://github.com/cozydev-pink/protosearch/pull/200/commits/8acbeeb2f89b5f51c0d4b610cd32a95cf32724eb and still had the same issue. If I publish https://github.com/cozydev-pink/protosearch/pull/200/commits/8acbeeb2f89b5f51c0d4b610cd32a95cf32724eb locally, and add the following to http4s's plugins.sbt:

ThisBuild / resolvers +=
  "SonaType Snapshots".at("https://s01.oss.sonatype.org/content/repositories/snapshots/")
addSbtPlugin("pink.cozydev" % "protosearch-sbt" % "0.0-8acbeeb-SNAPSHOT")

when I run site/tlSite in that modified http4s setup, I see searchIndex.idx correctly built in site/target/docs/site/search but not the html or js files.

jenshalm commented 3 months ago

Hmm.. strange, what does laikaDescribe show in the Theme section near the top?

jenshalm commented 3 months ago

and what happens when you run laikaSite directly? This would help to rule out an integration issue with sbt-typelevel

valencik commented 3 months ago

Hmm.. strange, what does laikaDescribe show in the Theme section near the top?

I added the above project/plugins.sbt snippet to a simpler project (lucille) just to test if it was something strange about http4s, but the results were the same.

The Theme section from laikaDescribe:

[success] Theme:
[success]   Helium - with extensions:
[success]     sbt-typelevel-site Helium Extensions
Here's the docs/laikaDescribe output in full. 

sbt:root> docs/laikaDescribe
[success] 
[success] Parser(s):
[success]   Markdown
[success]   reStructuredText
[success] Renderer:
[success]   Depending on task
[success] Extension Bundles:
[success]   Laika's Default Extensions (supplied by library)
[success]   Laika's directive support (supplied by library)
[success]   Laika's built-in directives (supplied by library)
[success]   Document Type Matcher for Markdown (supplied by parser)
[success]   Default extensions for reStructuredText (supplied by parser)
[success]   Support for user-defined reStructuredText directives (supplied by parser)
[success]   Standard directives for reStructuredText (supplied by parser)
[success]   Document Type Matcher for reStructuredText (supplied by parser)
[success]   Directives for theme 'Helium' (supplied by theme)
[success]   Extensions for theme 'Helium' (supplied by theme)
[success]   Github-flavored Markdown (supplied by parser)
[success]   Default Syntax Highlighters for Code (supplied by library)
[success] Theme:
[success]   Helium - with extensions:
[success]     sbt-typelevel-site Helium Extensions
[success] Settings:
[success]   Strict Mode: false
[success]   Accept Raw Content: false
[success]   Compact Rendering: false
[success] Sources:
[success]   Markup File(s)
[success]     /index.md: file '/home/andrew/src/github.com/cozydev/lucille/site/target/mdoc/index.md'
[success]   Template(s)
[success]     -
[success]   Configuration Files(s)
[success]     -
[success]   CSS for PDF
[success]     -
[success]   Copied File(s)
[success]     /index.md~: file '/home/andrew/src/github.com/cozydev/lucille/site/target/mdoc/index.md~'
[success]   Root Directories
[success]     /home/andrew/src/github.com/cozydev/lucille/site/target/mdoc
[success] Target:
[success]   Directory '/home/andrew/src/github.com/cozydev/lucille/site/target/docs/site'
[success] Total time: 0 s, completed Mar 29, 2024, 5:15:06 PM
valencik commented 3 months ago

and what happens when you run laikaSite directly? This would help to rule out an integration issue with sbt-typelevel

Same story.

The output:

sbt:root> docs/laikaSite
[info] Delete API dir
[info] Parsing 1 markup document, 0 templates, 0 configurations, 0 stylesheets ...
[info] Generated index in /home/andrew/src/github.com/cozydev/lucille/site/target/docs/site/search/searchIndex.idx

And then all that's in site/target/docs/site/search/ is searchIndex.idx

jenshalm commented 3 months ago

The Theme section from laikaDescribe:

[success] Theme:
[success]   Helium - with extensions:
[success]     sbt-typelevel-site Helium Extensions

Wait, you get this with laikaTheme := Helium.defaults.extendWith...? Because that would mean sbt-typelevel overrides the theme after you've set it in your build.

valencik commented 3 months ago

Wait, you get this with laikaTheme := Helium.defaults.extendWith...? Because that would mean sbt-typelevel overrides the theme after you've set it in your build.

Yes? But keep in might, that laikaTheme := Helium.defaults.extendWith... is set in ProtosearchPlugin.scala, right here: https://github.com/cozydev-pink/protosearch/pull/200/commits/8acbeeb2f89b5f51c0d4b610cd32a95cf32724eb#diff-8afcff29810188a2facba42b13415688bd75ee91082aa43a77624a290657c5b9R34

So, I'm somewhat concerned I'm doing something wrong there. But yeah, perhaps sbt-typelevel is also doing something wrong? I definitely do not know.

jenshalm commented 3 months ago

I suspect this is an integration issue, as your plugin only depends on Laika standalone plugin and then is used via sbt-typelevel which does not honor existing theme setup: https://github.com/typelevel/sbt-typelevel/blob/main/site/src/main/scala/org/typelevel/sbt/TypelevelSitePlugin.scala#L130.

So I'd assume you need to change at least one aspect of the composition to prevent your theme extension from getting lost. You can a) try to depend on sbt-typelevel directly in your plugin or b) test with Laika standalone instead. Otherwise we'd need to change sbt-typelevel to build on top of existing theme definition, but I suspect this might get messy given how many aspects of the theme it adjusts. I think it's cleaner letting sbt-tl define the theme from scratch, but that's also how you lose your extension, I believe.

valencik commented 3 months ago

try to depend on sbt-typelevel directly in your plugin

Ahh, yes! I think this is a fine compromise for the very easy testing during prototyping I'm looking for. I've just tested this out in https://github.com/cozydev-pink/protosearch/pull/200/commits/4ada3a1b6899a18bb67fb00b2bd1c7b174ff52fc and laikaDescribe now gives:

[success] Theme:
[success]   Helium - with extensions:
[success]     sbt-typelevel-site Helium Extensions
[success]     protosearch UI

And search works in docs/tlSitePreview! Thanks so much for helping me work through this :heart: