search not finding generated pages (doesn't seem to be a renderOrder issue?)

hpfast commented 3 months ago

Version

2.2.2

Platform

Linux

What steps will reproduce the bug?

Using the following posts.page.ts and layout.vto, generated items are not found by search.pages().

posts.page.ts:

import Metalsmith from "npm:metalsmith";

export const layout = "layout.vto";
export const renderOrder = -1;

//this doesn't work ...
const sourcedir = '/path/to/other/dir/with/markdown-files';
const source = await Metalsmith('.')
  .source(sourcedir)
  .process();   //returns a promise
const articles = Object.values(source);

//but neither does this
const articles2 = [
  {
    title: 'foobar',
    publish: {
      slug: 'foobar'
    },
    contents: 'nothing to see here'
  },
  {
    title: 'barbaz',
    publish: {
      slug: 'barbaz'
    },
    contents: 'very interesting stuff here'
  }
]

export default function* () {
  //for (const article of articles) {
  for (const article of articles2) {
    yield {
      url: '/post/'+article.publish.slug,
      title: article.title,
      content: article.contents,
      type: "mytype"
    }
  }
}

template:

<html lang="en">
<!--other stuff ...->

  <h2>Articles</h2>
{{for item of search.pages("url^=/post")}}
<p class="flex stretch | article-list"><a href="{{item.url}}">{{item.title}}</a></p>
{{/for}}

</body>
</html>

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior?

posts should show up in search results.

What do you see instead?

I think I've confirmed that it doesn't have to do with the async nature of the datasource by replacing that with a literal array -- same result. But I'm not sure ...

I also think it's not renderOrder (for example, #275 ) because I've inspected the pages in both beforeRender and afterRender events and the renderOrder is as I set it in my page files.

I do think it has to do with the generated nature of the pages, because if I modify the search.pages() query -- for example, to !url^=/post then I get all my items with a direct file origin.

The generated pages do get rendered and have urls starting with /post.

I can't tell the difference between my setup and the examples in the docs ... using top-level await or a literal array doesn't seem to make a difference.

Additional information

I can't figure it out ... any ideas?

oscarotero commented 3 months ago

The url of the pages is '/post/'+article.publish.slug, which I assume it doesn't end with / or .html.

search.pages() only returns html pages, and Lume only consider HTML pages those which the .html extension in the url or ending with / (which is equivalent to /index.html). See this.

hpfast commented 3 months ago

Right, adding / to the end of the url makes the pages show up ...

So it is related to generating pages indirectly, in that when setting the url field yourself you need to be aware of the behaviour. Which is well-documented, but I lost sight of the connection. Good to have the two parts of the documentation linked together by this issue!

Thanks!

lumeland / lume