Offline search functionality

rogier-stegeman commented 1 year ago

ref #733

Is your feature request related to a problem? Please describe.

We are using VitePress for internal documentation. Especially for larger projects such as our entire knowledge base, it can be hard to find exactly what you're looking for. A search bar would greatly help out. An Algolia search bar is already available, but since our docs are hosted on a private network and for internal use only, this is not an option for us.

From Algolia DocSearch documentation:

Your website must be publicly available. We do not host search indices for websites that are only available after authentication or are hosted on a private network.

Describe the solution you'd like

A search bar that works offline, or at least on a private network. I believe VuePress has a plugin, but the serach bar shows "Powered by Algolia" so I'm not sure.

Describe alternatives you've considered

Hosting our docs publicly: not an option.
Using VuePress or a different static site generator altogether, but since our apps are built on top of Vite we'd rather not.
Using VuePress Next/V2 since it seems to have a local search option, and supports Vite as well.

Additional context

No response

Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Read the Contributing Guidelines.
[X] Check that there isn't already an issue that asks for the same feature to avoid creating a duplicate.

brc-dd commented 1 year ago

I think Algolia provides search solutions for enterprises too. Try contacting their sales. Also, I think someone from the community can make some package like emersonbottero/vitepress-plugin-search. You can try this one too, I am not sure if it works. I saw it on one of the discussions.

fawazahmed0 commented 1 year ago

It would be great to have a search functionality similar to docsify, it supports multiple languages and doesn't depend on algolia(i.e works offline)

jd-solanki commented 1 year ago

Also, algolia is paid for projects other than OSS.

ed-fruty commented 1 year ago

I think, that any static site generator must provide offline search out of the box. VuePress, Docusaurus etc. has this feature. And as an additional feature - search by algolia or another one engine.

emersonbottero commented 1 year ago

I think Algolia provides search solutions for enterprises too. Try contacting their sales. Also, I think someone from the community can make some package like emersonbottero/vitepress-plugin-search. You can try this one too, I am not sure if it works. I saw it on one of the discussions.

it works, I used it in a doc that clones 30 repo to extract the documentation. but the workflow is not good since I'm reading the generate html.

wakaloka-dev commented 1 year ago

How to use the regular Algolia service instead Docsearch? The document site I built is for commercial product, which is doesn't fulfill the submission requirement.

Your website must be a technical documentation of an open source project or a technical blog. We do not index commercial content.

alokVishu commented 1 year ago

Will it get Vuepress like search?

staghouse commented 1 year ago

Is there a status update on implementing local search?

brc-dd commented 1 year ago

but the workflow is not good since I'm reading the generate html.

@emersonbottero What if we could provide you a transformHtml hook that gives you access to stuff like: resolved site config/data, title, description, head config, content, html, filename? That should also prevent the running build twice thing IG?

staghouse commented 1 year ago

I did something kind of hacky last night based on vitepress-plugin-search - I wasnt happy with using lunr, needing to build twice and having it read the dist folder. So I did something like this:

Context: I was trying to figure out how VitePress could have Local Search like VuePress has, and it appears that VitePress doesnt export anything that combines all the data of each page. Basically it seems like theres a missing data point in siteData that should be pages, similiar to what Vuepress has with this.$site.pages.

Step 1: In my .vitepress config.js I import a script that runs through the docs folder and includes only markdown files. Then for each markdown file I run it through markdown-it to generate HTML with anchor links using markdown-it-anchor. For each anchor link I have cheerio read the HTML and I extract data about each anchor on each page and assign that to an object. I wrap those objects in an array and I end up with data that contains some meta data about each page, like url, title and anchors where anchors is an array of each header on the page containing more indexable information like header text, and hash. From there I just assign that to the themeConfig property of config.js.

Step 2: Created a Search.vue component that imports the theme config from vitepress useData and access that array. Then without going too much in to boring detail, steps through the array of pages on Search input and match results based on the input value and the array's objects (each page). Then just render the results that match.

Caveat: I have a very strict markdown file standard so the above is very catered toward what my files look like and how I wanted my search results to appear.

When all is said and done, I ended with a nifty local search strategy that didnt depend on lunr or the dist directory. It seems to just work at this time.

emersonbottero commented 1 year ago

but the workflow is not good since I'm reading the generate html.

@emersonbottero What if we could provide you a transformHtml hook that gives you access to stuff like: resolved site config/data, title, description, head config, content, html, filename? That should also prevent the running build twice thing IG?

All I need is in the readHtml parameters, so if we can have the same values as the output html in an hook it should work .👌

emersonbottero commented 1 year ago

I did something kind of hacky last night based on vitepress-plugin-search - I wasnt happy with using lunr, needing to build twice and having it read the dist folder. So I did something like this:

Context: I was trying to figure out how VitePress could have Local Search like VuePress has, and it appears that VitePress doesnt export anything that combines all the data of each page. Basically it seems like theres a missing data point in siteData that should be pages, similiar to what Vuepress has with this.$site.pages.

Step 1: In my .vitepress config.js I import a script that runs through the docs folder and includes only markdown files. Then for each markdown file I run it through markdown-it to generate HTML with anchor links using markdown-it-anchor. For each anchor link I have cheerio read the HTML and I extract data about each anchor on each page and assign that to an object. I wrap those objects in an array and I end up with data that contains some meta data about each page, like url, title and anchors where anchors is an array of each header on the page containing more indexable information like header text, and hash. From there I just assign that to the themeConfig property of config.js.

Step 2: Created a Search.vue component that imports the theme config from vitepress useData and access that array. Then without going too much in to boring detail, steps through the array of pages on Search input and match results based on the input value and the array's objects (each page). Then just render the results that match.

Caveat: I have a very strict markdown file standard so the above is very catered toward what my files look like and how I wanted my search results to appear.

When all is said and done, I ended with a nifty local search strategy that didnt depend on lunr or the dist directory. It seems to just work at this time.

I guess you are missing the nicest feature of lurn.. compression with preview , the way you are doing your search data will be big to be downloaded in the cliente. I already use

const SEARCH_FIELDS = ["title", "description", "keywords", "body", "anchor"];

to search and to give an idea .. this docs without the autogenerated part creates this index search and it can 'build all the previews based on that'.

const LUNR_DATA = {"version":"2.3.9","fields":["t","d","k","b","a"],"fieldVectors":[["t/0",[0,1.887,1,0.385,2,0.073]],["d/0",[3,2.505]],["k/0",[]],["b/0",[4,0.107,5,2.787]],["a/0",[]],["t/1",[1,0.347,2,0.065,6,0.839,7,0.584]],["d/1",[8,0.197,9,0.374,10,0.061,11,0.061,12,0.197,13,0.061]],["k/1",[]],["b/1",[1,0.57,4,0.063,6,0.801,7,0.96,9,0.659,11,0.087,13,0.063,14,0.201,15,0.279,16,0.201,17,0.201,18,0.201,19,1.621,20,1.621,21,1.621,22,1.126,23,0.801,24,1.621,25,1.621,26,1.621,27,1.621,28,1.564,29,0.657,30,2.251,31,1.621,32,1.621,33,1.621,34,1.621,35,1.621,36,1.621,37,1.621,38,1.621,39,1.621,40,2.251,41,0.557,42,1.126,43,1.621,44,1.621,45,1.621,46,2.586,47,2.251,48,1.621,49,1.621,50,2.251,51,1.621,52,1.126,53,1.621,54,0.801,55,2.251,56,1.621,57,1.621,58,1.621,59,1.564,60,1.621,61,1.621,62,2.251,63,1.621,64,1.126,65,1.621,66,1.126,67,1.126,68,0.363,69,0.363,70,0.801,71,1.126,72,1.621]],["a/1",[6,0.85,7,0.591]],["t/2",[1,0.289,2,0.055,41,0.486,73,0.699,74,0.486,75,0.699]],["d/2",[8,0.197,9,0.374,10,0.061,11,0.061,12,0.197,13,0.061]],["k/2",[]],["b/2",[1,0.595,4,0.061,9,0.604,10,0.061,14,0.197,15,0.276,16,0.197,17,0.197,18,0.197,22,1.106,29,0.497,41,0.764,69,0.62,70,0.786,73,1.531,74,0.88,75,1.265,76,1.591,77,0.953,78,1.265,79,2.222,80,1.591,81,1.544,82,1.591,83,1.544,84,1.591,85,1.106,86,1.591,87,1.591,88,1.591,89,1.591,90,1.591,91,1.591,92,1.544,93,1.591,94,1.591,95,1.591,96,1.591,97,0.547,98,1.591,99,1.591,100,1.106,101,1.106,102,1.106,103,1.106,104,1.591,105,1.591,106,1.591,107,1.591,108,2.222,109,1.591,110,1.591,111,1.591,112,1.591,113,1.591,114,1.591,115,1.591,116,1.591,117,1.106,118,1.591,119,1.591,120,0.547,121,1.591]],["a/2",[41,0.425,73,0.61,74,0.425,75,0.61]],["t/3",[1,0.347,2,0.065,122,0.38,123,0.839]],["d/3",[8,0.197,9,0.374,10,0.061,11,0.061,12,0.197,13,0.061]],["k/3",[]],["b/3",[1,0.665,4,0.074,14,0.238,15,0.313,16,0.238,17,0.238,18,0.238,23,0.948,29,0.429,42,1.333,54,0.948,68,0.429,69,0.429,77,0.66,78,0.948,97,0.867,120,0.66,122,0.67,123,0.948,124,1.919,125,1.333,126,1.333,127,1.919,128,1.919,129,1.919,130,1.333,131,1.333,132,1.333,133,1.333,134,1.333,135,1.333,136,1.333,137,1.333,138,1.333,139,1.333,140,1.333,141,1.333,142,0.948,143,1.333,144,1.333,145,0.948,146,1.333,147,1.333,148,1.333,149,1.333,150,1.919,151,1.919]],["a/3",[122,0.385,123,0.85]],["t/4",[1,0.347,2,0.065,152,0.839,153,0.584]],["d/4",[8,0.197,9,0.374,10,0.061,11,0.061,12,0.197,13,0.061]],["k/4",[]],["b/4",[1,0.701,4,0.035,7,0.315,9,0.495,14,0.114,15,0.182,16,0.319,17,0.114,18,0.114,28,1.457,29,0.47,52,0.636,59,0.636,64,0.636,66,0.636,67,0.636,68,0.514,69,0.328,70,0.724,74,0.315,77,0.919,78,0.452,81,0.636,83,0.636,85,0.636,92,0.636,97,0.63,100,1.702,101,0.636,102,0.636,103,0.636,117,1.275,120,0.315,122,0.47,126,0.636,142,1.036,145,1.27,152,0.724,153,0.315,154,0.915,155,0.915,156,0.915,157,0.915,158,0.915,159,0.915,160,0.915,161,0.915,162,0.915,163,0.915,164,0.915,165,0.915,166,0.915,167,0.915,168,0.915,169,0.915,170,0.915,171,0.915,172,0.915,173,0.915,174,0.915,175,0.915,176,0.915,177,0.915,178,0.915,179,1.466,180,0.915,181,0.915,182,1.466,183,1.466,184,0.915,185,1.466,186,1.466,187,1.466,188,0.915,189,1.466,190,0.915,191,0.915,192,0.915,193,0.915,194,0.915,195,0.915,196,0.915,197,0.915,198,0.915,199,0.915,200,0.915,201,0.915,202,1.466,203,0.915,204,0.915,205,0.915,206,0.915,207,0.915,208,2.673,209,0.915,210,0.915,211,0.915,212,0.915,213,0.915,214,0.915,215,0.915,216,0.915,217,0.915,218,0.915,219,0.915,220,0.915,221,0.915,222,0.915,223,0.915,224,0.915,225,0.915,226,0.915,227,0.915,228,0.915,229,0.915,230,0.915,231,1.466,232,0.915,233,0.915,234,0.636,235,0.915,236,0.915,237,0.915,238,0.915,239,0.915,240,0.915,241,0.915,242,0.915,243,0.915,244,0.915,245,1.466,246,0.915,247,0.915,248,2.295,249,0.915,250,0.915,251,0.915,252,1.466,253,0.915,254,0.915,255,0.915]],["a/4",[152,0.85,153,0.591]],["t/5",[1,0.347,2,0.065,9,0.4,256,1.18]],["d/5",[8,0.197,9,0.374,10,0.061,11,0.061,12,0.197,13,0.061]],["k/5",[]],["b/5",[1,0.661,4,0.072,9,0.657,14,0.233,15,0.308,16,0.233,17,0.233,18,0.233,23,0.927,29,0.557,54,0.927,68,0.42,69,0.42,77,0.645,97,0.645,120,0.645,122,0.557,125,1.304,130,1.304,131,1.304,132,1.304,133,1.304,134,1.304,135,1.304,136,1.304,137,1.304,138,1.304,139,1.304,140,1.304,141,1.304,142,0.927,143,1.304,144,1.304,145,0.927,146,1.304,147,1.304,148,1.304,149,1.304,234,1.304,256,1.304,257,1.877,258,1.877,259,1.877,260,1.877,261,1.877,262,1.877,263,1.877,264,1.877,265,1.877,266,1.877,267,1.877]],["a/5",[9,0.339,268,1.439,269,1.439]],["t/6",[1,0.347,2,0.065,9,0.4,270,0.839]],["d/6",[8,0.197,9,0.374,10,0.061,11,0.061,12,0.197,13,0.061]],["k/6",[]],["b/6",[1,0.473,4,0.089,9,0.547,14,0.288,15,0.353,16,0.288,17,0.288,18,0.288,68,0.519,71,1.611,153,0.797,270,1.145,271,2.319,272,2.319,273,2.319,274,2.319,275,3.079,276,2.319,277,2.319,278,2.319,279,2.319,280,2.319,281,2.319,282,2.319]],["a/6",[9,0.406,270,0.85]]],"invertedIndex":[["",{"_index":1,"t":{"0":{},"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["0",{"_index":189,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["1/1/0001",{"_index":87,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["10",{"_index":147,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["12:00:00",{"_index":88,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["2",{"_index":170,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["3",{"_index":167,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["404",{"_index":0,"t":{"0":{}},"d":{},"k":{},"b":{},"a":{}}],["access",{"_index":43,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["ad",{"_index":51,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["addcustom",{"_index":115,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["address",{"_index":62,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["admin",{"_index":34,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["aligned$1600col",{"_index":169,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["allow",{"_index":261,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["alway",{"_index":91,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["am}\"guid.emptythat",{"_index":89,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["api",{"_index":11,"t":{},"d":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"k":{},"b":{"1":{}},"a":{}}],["applic",{"_index":135,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["applicationsthi",{"_index":49,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["attribut",{"_index":200,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["authent",{"_index":7,"t":{"1":{}},"d":{},"k":{},"b":{"1":{},"4":{}},"a":{"1":{}}}],["authenticationazuread",{"_index":19,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["authenticationther",{"_index":20,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["automat",{"_index":193,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["await",{"_index":142,"t":{},"d":{},"k":{},"b":{"3":{},"4":{},"5":{}},"a":{}}],["azur",{"_index":50,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["azuread",{"_index":37,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["b",{"_index":275,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["ban",{"_index":277,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["base",{"_index":99,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["baseaddress",{"_index":30,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["basic",{"_index":6,"t":{"1":{}},"d":{},"k":{},"b":{"1":{}},"a":{"1":{}}}],["basicauthenticationconfig.readfromjsonfile(\"appsettings.json",{"_index":67,"t":{},"d":{},"k":{},"b":{"1":{},"4":{}},"a":{}}],["bla",{"_index":281,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["br",{"_index":183,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["branco",{"_index":181,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["c[fa:fa",{"_index":276,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["case",{"_index":94,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["catalogitem",{"_index":123,"t":{"3":{}},"d":{},"k":{},"b":{"3":{}},"a":{"3":{}}}],["catalogitemcr",{"_index":137,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["catalogitemy",{"_index":124,"t":{},"d":{},"k":{},"b":{"3":{}},"a":{}}],["chang",{"_index":248,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["check",{"_index":39,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["citycod",{"_index":213,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["citycode:{citycod",{"_index":221,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["class",{"_index":117,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["classse",{"_index":129,"t":{},"d":{},"k":{},"b":{"3":{}},"a":{}}],["client",{"_index":46,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["clientid",{"_index":60,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["clientsecret",{"_index":61,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["compani",{"_index":55,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["complex",{"_index":271,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["config",{"_index":66,"t":{},"d":{},"k":{},"b":{"1":{},"4":{}},"a":{}}],["configur",{"_index":159,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["configurations.custom",{"_index":104,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["console.writeline(\"incid",{"_index":252,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["constructorvar",{"_index":65,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["content",{"_index":17,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["convert",{"_index":105,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["countri",{"_index":182,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["cq7k8rfgpamg",{"_index":36,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["creat",{"_index":122,"t":{"3":{}},"d":{},"k":{},"b":{"3":{},"4":{},"5":{}},"a":{"3":{}}}],["credenti",{"_index":45,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["current",{"_index":18,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["custom",{"_index":78,"t":{},"d":{},"k":{},"b":{"2":{},"3":{},"4":{}},"a":{}}],["customenum",{"_index":108,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["customenumconverter<yourenum",{"_index":109,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["customrequeststateconvert",{"_index":119,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["d(fa:fa",{"_index":279,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["daemon",{"_index":48,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["danger",{"_index":165,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["data",{"_index":234,"t":{},"d":{},"k":{},"b":{"4":{},"5":{}},"a":{}}],["datetim",{"_index":75,"t":{"2":{}},"d":{},"k":{},"b":{"2":{}},"a":{"2":{}}}],["datetimey",{"_index":76,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["default",{"_index":41,"t":{"2":{}},"d":{},"k":{},"b":{"1":{},"2":{}},"a":{"2":{}}}],["defin",{"_index":42,"t":{},"d":{},"k":{},"b":{"1":{},"3":{}},"a":{}}],["descript",{"_index":249,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["deseri",{"_index":98,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["dev",{"_index":27,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["develop",{"_index":26,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["doc",{"_index":131,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["easi",{"_index":8,"t":{},"d":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"k":{},"b":{},"a":{}}],["element",{"_index":196,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["email",{"_index":214,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["email:{email",{"_index":222,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["empti",{"_index":95,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["enum",{"_index":110,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["enums.var",{"_index":107,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["exampl",{"_index":230,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["expandoobject",{"_index":244,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["export",{"_index":264,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["feed",{"_index":259,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["fenc",{"_index":270,"t":{"6":{}},"d":{},"k":{},"b":{"6":{}},"a":{"6":{}}}],["first",{"_index":134,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["flow",{"_index":125,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["flowvar",{"_index":138,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["forbidden",{"_index":278,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["found",{"_index":3,"t":{},"d":{"0":{}},"k":{},"b":{},"a":{}}],["get",{"_index":152,"t":{"4":{}},"d":{},"k":{},"b":{"4":{}},"a":{"4":{}}}],["graph",{"_index":273,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["guid",{"_index":74,"t":{"2":{}},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{"2":{}}}],["guid(\"catalogitemidher",{"_index":141,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["guid(incident.getproperty(\"sys_id\").tostr",{"_index":243,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["guid(sys_id",{"_index":149,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["handl",{"_index":52,"t":{},"d":{},"k":{},"b":{"1":{},"4":{}},"a":{}}],["happen",{"_index":90,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["https://dev77132.servic",{"_index":31,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["https://login.microsoftonline.com",{"_index":56,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["id",{"_index":59,"t":{},"d":{},"k":{},"b":{"1":{},"4":{}},"a":{}}],["import",{"_index":263,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["importset",{"_index":268,"t":{},"d":{},"k":{},"b":{},"a":{"5":{}}}],["importset'",{"_index":256,"t":{"5":{}},"d":{},"k":{},"b":{"5":{}},"a":{}}],["importset'sy",{"_index":257,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["inc",{"_index":245,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["inc.updateprop(\"short_descript",{"_index":247,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["incid",{"_index":242,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["incident.toobject",{"_index":246,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["incidentsnottyp",{"_index":235,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["incidentsnottyped.foreach(async",{"_index":241,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["incidentstablenottyp",{"_index":231,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["incidentstablenottyped.update(id",{"_index":251,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["inherit",{"_index":198,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["inject",{"_index":174,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["insid",{"_index":54,"t":{},"d":{},"k":{},"b":{"1":{},"3":{},"5":{}},"a":{}}],["instal",{"_index":156,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["instanc",{"_index":28,"t":{},"d":{},"k":{},"b":{"1":{},"4":{}},"a":{}}],["instancevar",{"_index":223,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["integr",{"_index":12,"t":{},"d":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"k":{},"b":{},"a":{}}],["invalid",{"_index":113,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["iscentered$12zebra",{"_index":171,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["isright",{"_index":168,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["json",{"_index":82,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["jsonconverteroptions.configurecustomserializers(new",{"_index":118,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["jsonpropertynam",{"_index":204,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["jsonpropertyname(\"u_city_cod",{"_index":212,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["jsonpropertyname(\"user_nam",{"_index":215,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["lanid",{"_index":216,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["learn",{"_index":132,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["librari",{"_index":13,"t":{},"d":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"k":{},"b":{"1":{}},"a":{}}],["limit(10",{"_index":233,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["limit(2",{"_index":179,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["lr",{"_index":274,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["made",{"_index":184,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["make",{"_index":127,"t":{},"d":{},"k":{},"b":{"3":{}},"a":{}}],["map",{"_index":201,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["model",{"_index":206,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["more",{"_index":229,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["morey",{"_index":133,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["mostli",{"_index":53,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["multipl",{"_index":262,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["name",{"_index":202,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["navigationappear",{"_index":5,"t":{},"d":{},"k":{},"b":{"0":{}},"a":{}}],["navigationappearanceon",{"_index":14,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["neat$1var",{"_index":173,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["need",{"_index":197,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["new",{"_index":69,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{}},"a":{}}],["next",{"_index":71,"t":{},"d":{},"k":{},"b":{"1":{},"6":{}},"a":{}}],["nice",{"_index":239,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["non",{"_index":250,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["now",{"_index":130,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["now.bas",{"_index":24,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["now.com/api",{"_index":32,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["nuget",{"_index":157,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["null",{"_index":93,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["nullabl",{"_index":84,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["number",{"_index":106,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["onc",{"_index":79,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["option",{"_index":22,"t":{},"d":{},"k":{},"b":{"1":{},"2":{}},"a":{}}],["optional)default",{"_index":80,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["orderby(\"sys_id",{"_index":240,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["overrid",{"_index":217,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["packagese",{"_index":158,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["page",{"_index":15,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["pageauthent",{"_index":121,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["pageconfigur",{"_index":255,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["pageconfigurationnext",{"_index":150,"t":{},"d":{},"k":{},"b":{"3":{}},"a":{}}],["pagecustom",{"_index":267,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["pageget",{"_index":282,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["pageimport",{"_index":151,"t":{},"d":{},"k":{},"b":{"3":{}},"a":{}}],["pageseri",{"_index":72,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["pagewhat",{"_index":253,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["pagin",{"_index":192,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["paramet",{"_index":64,"t":{},"d":{},"k":{},"b":{"1":{},"4":{}},"a":{}}],["parametersvar",{"_index":176,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["password",{"_index":35,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["pend",{"_index":114,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["possibl",{"_index":126,"t":{},"d":{},"k":{},"b":{"3":{},"4":{}},"a":{}}],["previou",{"_index":120,"t":{},"d":{},"k":{},"b":{"2":{},"3":{},"4":{},"5":{}},"a":{}}],["process",{"_index":260,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["properti",{"_index":101,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["public",{"_index":208,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["receiv",{"_index":195,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["request",{"_index":97,"t":{},"d":{},"k":{},"b":{"2":{},"3":{},"4":{},"5":{}},"a":{}}],["request.se",{"_index":266,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["requestcatalog",{"_index":139,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["requestcatalog.request(new",{"_index":143,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["reset",{"_index":194,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["respons",{"_index":83,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["rest",{"_index":10,"t":{},"d":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"k":{},"b":{"2":{}},"a":{}}],["return",{"_index":92,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["return:new",{"_index":86,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["s",{"_index":269,"t":{},"d":{},"k":{},"b":{},"a":{"5":{}}}],["scope",{"_index":40,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["secret",{"_index":47,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["select(new",{"_index":236,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["send",{"_index":258,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["serial",{"_index":73,"t":{"2":{}},"d":{},"k":{},"b":{"2":{}},"a":{"2":{}}}],["serializerscr",{"_index":162,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["servic",{"_index":23,"t":{},"d":{},"k":{},"b":{"1":{},"3":{},"5":{}},"a":{}}],["servicenow",{"_index":29,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{}},"a":{}}],["servicenow(config",{"_index":70,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"4":{}},"a":{}}],["servicenow.cor",{"_index":2,"t":{"0":{},"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"d":{},"k":{},"b":{},"a":{}}],["servicenow.core?next",{"_index":254,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["servicenow.coresearchmetakmain",{"_index":4,"t":{},"d":{},"k":{},"b":{"0":{},"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["servicenow.cr",{"_index":205,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["servicenow.usingcatalog<request>(new",{"_index":140,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["servicenow.warningthi",{"_index":163,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["servicenowbasemodel",{"_index":210,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["servicenowbasemodeltipus",{"_index":199,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["set",{"_index":77,"t":{},"d":{},"k":{},"b":{"2":{},"3":{},"4":{},"5":{}},"a":{}}],["setup",{"_index":160,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["sever",{"_index":21,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["short_descript",{"_index":237,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["singl",{"_index":265,"t":{},"d":{},"k":{},"b":{"5":{}},"a":{}}],["snowtabl",{"_index":203,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["snowtable(\"sys_us",{"_index":207,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["spinner",{"_index":280,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["start",{"_index":153,"t":{"4":{}},"d":{},"k":{},"b":{"4":{},"6":{}},"a":{"4":{}}}],["startednot",{"_index":154,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["state",{"_index":211,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["state:{st",{"_index":220,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["static",{"_index":116,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["string",{"_index":145,"t":{},"d":{},"k":{},"b":{"3":{},"4":{},"5":{}},"a":{}}],["string.th",{"_index":96,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["stripesar",{"_index":172,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["studio",{"_index":136,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["sys_id",{"_index":102,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["tabl",{"_index":16,"t":{},"d":{},"k":{},"b":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["tenant",{"_index":58,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["tenantid",{"_index":57,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["test",{"_index":272,"t":{},"d":{},"k":{},"b":{"6":{}},"a":{}}],["those",{"_index":63,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["tipth",{"_index":38,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["token",{"_index":44,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["tolistasync",{"_index":185,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["tostr",{"_index":218,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["type",{"_index":100,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["typedtypedget",{"_index":155,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["updat",{"_index":103,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["us",{"_index":9,"t":{"5":{},"6":{}},"d":{"1":{},"2":{},"3":{},"4":{},"5":{},"6":{}},"k":{},"b":{"1":{},"2":{},"4":{},"5":{},"6":{}},"a":{"5":{},"6":{}}}],["user",{"_index":209,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["user:{nam",{"_index":219,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usernam",{"_index":33,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["usernottyped.display",{"_index":191,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usersnottyp",{"_index":186,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usersnottyped.count",{"_index":188,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usersnottyped.foreach(usernottyp",{"_index":190,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["userst",{"_index":224,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["userstablenottyp",{"_index":177,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["userstablenottyped.tolistasync",{"_index":187,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usingcatalog",{"_index":128,"t":{},"d":{},"k":{},"b":{"3":{}},"a":{}}],["usingtable(\"incid",{"_index":232,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usingtable(\"sys_us",{"_index":178,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usingtable<user>(\"sys_us",{"_index":225,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["usual",{"_index":25,"t":{},"d":{},"k":{},"b":{"1":{}},"a":{}}],["valid",{"_index":112,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}],["valu",{"_index":81,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["var",{"_index":68,"t":{},"d":{},"k":{},"b":{"1":{},"3":{},"4":{},"5":{},"6":{}},"a":{}}],["varnameherenumb",{"_index":146,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["varnamehererefer",{"_index":148,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["varnameherestr",{"_index":144,"t":{},"d":{},"k":{},"b":{"3":{},"5":{}},"a":{}}],["version",{"_index":85,"t":{},"d":{},"k":{},"b":{"2":{},"4":{}},"a":{}}],["warningdangerthi",{"_index":164,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["warningtablesarecoolcol",{"_index":166,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["well",{"_index":161,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["wellservices.addsingleton<iservicenow>(new",{"_index":175,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["where(x",{"_index":226,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["withquery(\"nam",{"_index":180,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["withquery(\"short_descript",{"_index":238,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["x.countri",{"_index":228,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["x.name.contains(\"branco",{"_index":227,"t":{},"d":{},"k":{},"b":{"4":{}},"a":{}}],["yourenum",{"_index":111,"t":{},"d":{},"k":{},"b":{"2":{}},"a":{}}]],"pipeline":["stemmer"]};
const PREVIEW_LOOKUP = {"0":{"t":"404 | ServiceNow.Core","p":"ServiceNow.CoreSearchMetaKMain NavigationAppearance","l":"404.html","a":""},"1":{"t":"Basic Authentication | ServiceNow.Core","p":"ServiceNow.CoreSearchMetaKMain NavigationAppearanceOn this pag ...","l":"config/Authentication.html","a":"#basic-authentication"},"2":{"t":"Default Serializers for Guid and DateTime | ServiceNow.Core","p":"ServiceNow.CoreSearchMetaKMain NavigationAppearanceOn this pag ...","l":"config/Serializers.html","a":"#default-serializers-for-guid-and-datetime"},"3":{"t":"You can create CatalogItem | ServiceNow.Core","p":"ServiceNow.CoreSearchMetaKMain NavigationAppearanceOn this pag ...","l":"guide/catalog-item.html","a":"#you-can-create-catalogitem"},"4":{"t":"Getting Started | ServiceNow.Core","p":"ServiceNow.CoreSearchMetaKMain NavigationAppearanceOn this pag ...","l":"guide/getting-started.html","a":"#getting-started"},"5":{"t":"You can use ImportSet's | ServiceNow.Core","p":"ServiceNow.CoreSearchMetaKMain NavigationAppearanceOn this pag ...","l":"guide/import-set.html","a":"#you-can-use-importset-s"},"6":{"t":"Using fences | ServiceNow.Core","p":"ServiceNow.CoreSearchMetaKMain NavigationAppearanceOn this pag ...","l":"index.html","a":"#using-fences"}};
const data = { LUNR_DATA, PREVIEW_LOOKUP };
export default data;

staghouse commented 1 year ago

@emersonbottero I don't think that is lost on me. For my project that mapping is actually a bigger chunk of data than just the data I need.

I'm also implementing basically what VuePress has been doing so it's not like I'm breaking a mold. With that said, I don't mean to bash the original work - I just wanted something different that was friendlier to my flow.

brc-dd commented 1 year ago

So, IG if we provide you the transformHtml hook, it would ease things up for both of you. Like for @staghouse it will clean up step 1 (no need to manually run markdown-it), for @emersonbottero no need to manually find and read html files. Also there won't be any need to manually extract title, description, keywords, etc.

However, I have a concern regarding vitepress-plugin-search, there you are statically importing data from your index file (in Search.vue). But that file won't be present during first build. How will that work?

PS: We also have a buildEnd hook, so you can create a variable and add stuff to that from transformHtml, then on build end write it to a file inside siteConfig.outDir. From the component it can be dynamically imported. I am guessing this will also simplify the blog thing I wrote: https://github.com/vuejs/vitepress/issues/96#issuecomment-1146676893

emersonbottero commented 1 year ago

@brc-dd The way it works now it creates an empty index the first time, that's why we need a second build. that static import also prevent us from using dev script without at least one build.

the best solution would be creating an rollup plugin that in dev create an virtual 'module' that then would be used in the search component while in dev and would also create the file needed after the first build (since it would use the md files and not the html ones. I still have to learn that..

michaelkplai commented 1 year ago

Would it be possible to provide the an equivalent of this.$site.pages from VuePress through the useData function?

This would allow the following solution for VuePress to be implemented: https://medium.com/@z3by/build-a-better-search-in-vuepress-568680b6b8d4.

emersonbottero commented 1 year ago

I manage to create the Index in a vite plugin. With only one build I'm still trying to use a slot to use my search element in the plugin. So we only need to add the plugin and nothing else.

mrbrianevans commented 1 year ago

Would it be possible to create a search feature using https://github.com/nextapps-de/flexsearch, which can be used to index all your docs at build time, and then ship a static index file to the client which can be used to search the docs.

The benefits of this are that it doesn't depend on algolia (3rd party), it can be used completely offline once the webpage has loaded (would be good for PWA), it would work in the OPs scenario of private network, it could be opt-in if some prefer algolia, doesn't require a paid-for plan for non-open-source projects. Perhaps also a performance benefit of not having a HTTP request for every search.

With a little bit of guidance from someone more experienced with VitePress, I would be happy to have a go at making a PR to implement this.

mattjcowan commented 1 year ago

I manage to create the Index in a vite plugin. With only one build I'm still trying to use a slot to use my search element in the plugin. So we only need to add the plugin and nothing else.

Wanted to try out Vitepress yesterday, and the lack of a built-in search function felt like the only thing that would keep me from using it. Then I found your plugin, very helpful .... I altered it for my use-case and re-used the Algolia search UI instead, as I like the way it works. They have a transformSearchClient props property that can be used to bypass the xhr call back to Algolia and you can then just pass back your own results from wherever you want to get them from, in this case the Lunr index, but could be anything .... I just made a copy of the VPAlgoliaSearchBox.vue file and called it VPLunrSearchBox.vue ... The search function just needs to respond in a format that matches the Algolia search UI requirements. I changed the index on my side to use h1 through h6 tags instead of the ones in your plugin, but it's the same idea.

Then in the VPLunrSearchBox.vue file, I just changed the docsearch function to a lunrsearch function that calls docsearch internally.

import lunr from 'lunr';
import { LUNR_DATA, PREVIEW_LOOKUP } from './lunr_index.js';

function lunrsearch(props: DocSearchProps): void {
    props.disableUserPersonalization = true;
    props.transformSearchClient = (searchClient) => {
        searchClient.search = (n, r) => {
            const searchTerm = (n && n.length > 0 && n[0].query) ? n[0].query : '';
            return new Promise((resolve) => {
                // build response
                const response = {
                    results: [
                        {
                            "hits": [],
                            ... etc ...
                        }
                    ]
                };
                resolve(response);
            });
        }
        return searchClient;
    }
    docsearch(props);
};

The UI using the Vitepress docs site and Lunr as the index then looks like this. Sorry 'bout the blue theme customization, just me messing around.

I'd like to give props to Algolia for the search UI without the appearance that the search index is from Algolia, which is why I hid the Algolia logo for now, trying to think of best way to do that.

emersonbottero commented 1 year ago

I manage to create the Index in a vite plugin. With only one build I'm still trying to use a slot to use my search element in the plugin. So we only need to add the plugin and nothing else.

Wanted to try out Vitepress yesterday, and the lack of a built-in search function felt like the only thing that would keep me from using it. Then I found your plugin, very helpful .... I altered it for my use-case and re-used the Algolia search UI instead, as I like the way it works. They have a transformSearchClient props property that can be used to bypass the xhr call back to Algolia and you can then just pass back your own results from wherever you want to get them from, in this case the Lunr index, but could be anything .... I just made a copy of the VPAlgoliaSearchBox.vue file and called it VPLunrSearchBox.vue ... The search function just needs to respond in a format that matches the Algolia search UI requirements. I changed the index on my side to use h1 through h6 tags instead of the ones in your plugin, but it's the same idea.

Then in the VPLunrSearchBox.vue file, I just changed the docsearch function to a lunrsearch function that calls docsearch internally.
import lunr from 'lunr';
import { LUNR_DATA, PREVIEW_LOOKUP } from './lunr_index.js';

function lunrsearch(props: DocSearchProps): void {
    props.disableUserPersonalization = true;
    props.transformSearchClient = (searchClient) => {
        searchClient.search = (n, r) => {
            const searchTerm = (n && n.length > 0 && n[0].query) ? n[0].query : '';
            return new Promise((resolve) => {
                // build response
                const response = {
                    results: [
                        {
                            "hits": [],
                            ... etc ...
                        }
                    ]
                };
                resolve(response);
            });
        }
        return searchClient;
    }
    docsearch(props);
};
The UI using the Vitepress docs site and Lunr as the index then looks like this. Sorry 'bout the blue theme customization, just me messing around.

I'd like to give props to Algolia for the search UI without the appearance that the search index is from Algolia, which is why I hid the Algolia logo for now, trying to think of best way to do that.

Oh.. I can use that.. could you share your code?

gtg3vv commented 1 year ago

@emersonbottero I was poking at this same thing locally, but couldn't seem to get all the way to the solution above.

I added lunr index generation logic via the buildEnd hook and added a dynamic import so that I only need to trigger build logic once. (the index won't be present on initial live serve)
I added the custom component and overrode transformSearchClient and was able to get results back using the default algolia UI.

I was struggling to cleanly make a mapping from the current lunr search results to the algolia search format here: https://github.com/algolia/docsearch/blob/d96aac96b832c8aeaabb08eac72d348e0a0b2267/packages/docsearch-react/src/__tests__/api.test.tsx#L16. It's easy enough to get basic results, but I couldn't seem to get a proper hierarchy and text highlight working. It seemed like the algolia record extractor logic is what I needed, but I couldn't find any good implementations or examples of this.

mattjcowan commented 1 year ago

Here's a branch that adds lunr search to the vitepress docs (doesn't implement all the capabilities of Algolia search results, but it's a start). Did a few simple searches with it and was giving me the same results as the search on vitepress.vuejs.org ...

https://github.com/mattjcowan/vitepress/tree/with-lunr-search

Here are the files that you'll want to look at in the commit:

I just put all the files in one directory which should make it easy to copy from hopefully, and also simulates adding the edits to a vitepress docs repo.

cd docs && npm install

To build the index.

# build the docs at the root
npm run docs-build
# build the index (lunr_index.js file)
cd docs && npm run lunr-index-build

Please improve on it, all usual disclaimers apply ...

staghouse commented 1 year ago

This is something hacky I put together to get offline search working for me. The data can be accessed anytime using theme from useData();

VitePress config

import generatePages from './generatePages';

export const pages = generatePages({
  INCLUDE_DIR: 'docs',
  EXCLUDE_DIRS: ['public', '.vitepress'],
});

export default {
  themeConfig: {
    pages: pages,
  },
};

generatePages.js

/**
 * Generate pages data
 *
 * Iterate through the markdown files, convert them to HTML and
 * use Cheerio to walk through the anchors.
 *
 * This is an implementation based off of the package
 * vitepress-plugin-search: https://github.com/emersonbottero/vitepress-plugin-search
 */
import fs from 'fs';
import path from 'path';
import MarkdownIt from 'markdown-it';
import MarkdownItAnchor from 'markdown-it-anchor';
import { load as cheerioParse } from 'cheerio';

const slugify = (str) => {
  return (
    str
      // Remove control characters
      .replace(/[\u0000-\u001f]/g, '')
      // Replace special characters
      .replace(/[\s~`!@#$%^&*()\-_+=[\]{}|\\;:"'<>,.?/]+/g, '-')
      // Remove continuous separators
      .replace(/\-{2,}/g, '-')
      // Remove prefixing and trailing separators
      .replace(/^\-+|\-+$/g, '')
      // ensure it doesn't start with a number (#121)
      .replace(/^(\d)/, '_$1')
      // lowercase
      .toLowerCase()
  );
};

// Config options
const OPTIONS = {
  INCLUDE_DIR: '',
  EXCLUDE_DIRS: [],
};

const FRONTMATTER_REGEX = /---(.*?)---/s;
const MARKDOWN_FILE_INDEX = 'index.md';

/**
 * Check if the fileName has a Markdown extension.
 */
const isMarkdown = (fileName) => {
  return fileName.match(MARKDOWN_FILE_INDEX);
};

/**
 * Find each page and subdirectory in our src directory.
 */
const findMarkdown = (dir) => {
  /**
   * For each file/directory in our source directory reduce our content to an array.
   */
  return Array.from(
    new Set(
      fs.readdirSync(dir).reduce((reducer, file) => {
        /**
         * Join the file path to the src path to get a usuable path for getting information on.
         */
        const fileName = path.join(dir, file).toLowerCase();

        /**
         * Read the file.
         */
        const fileStats = fs.lstatSync(fileName);

        /**
         * If the file is a markdown file, push that to the reducer.
         */
        if (isMarkdown(fileName)) {
          reducer.push(fileName);

          return reducer;
        }

        /**
         * If the file is a directory
         */
        if (fileStats.isDirectory()) {
          /**
           * And it should be excluded, early return the reducer.
           */
          if (OPTIONS.EXCLUDE_DIRS.includes(file)) {
            return reducer;
          }

          /**
           * Get files in the current directory.
           */
          const dirFiles = fs.readdirSync(fileName);

          /**
           * Get the index markdown file of the current directory.
           */
          const dirIndex = dirFiles[dirFiles.indexOf(MARKDOWN_FILE_INDEX)];

          /**
           * If there is one (something like changelog just has more nested directories).
           */
          if (dirIndex) {
            /**
             * Store the markdown index file for thaat directory.
             */
            const dirIndexFile = `${fileName}/${dirIndex}`;

            /**
             * Push that file to the reducer.
             */
            reducer.push(dirIndexFile);
          }

          /**
           * For all the directories subdirectories.
           */
          dirFiles.forEach((dirFile) => {
            const subDirIndexFile = isMarkdown(dirFile)
              ? `${fileName}/${dirFile}`
              : `${fileName}/${dirFile}/${MARKDOWN_FILE_INDEX}`;

            /**
             * Push to the reducer
             */
            reducer.push(subDirIndexFile);
          });
        }

        /**
         * After all, return the reducer
         */
        return reducer;
      }, [])
    ).values()
  );
};

const readMarkdown = (fileName) => {
  /**
   * Title of the file.
   */
  let title = null;

  /**
   * Get usuable path of the read markdown file.
   *
   * This is the path that anchor use to navigate between pages in VitePress.
   */
  const basePath = fileName.split('index.md')[0];
  const [_, ...rest] = basePath.split('/');
  const path = `/${rest.join('/')}`;

  /**
   * Render our markdown in to MarkdownIt in order to get returned HTML to step through.
   *
   * We use markdown-it-anchor to in order to create anchors.
   */
  const markdown = fs.readFileSync(fileName).toString();
  const markdownStrippedOfFrontmatter = markdown.replace(FRONTMATTER_REGEX, '');
  const markdownItPermalink = MarkdownItAnchor.permalink.headerLink();
  const markdownIt = new MarkdownIt().use(MarkdownItAnchor, { permalink: markdownItPermalink, slugify });
  const markdownItHTML = markdownIt.render(markdownStrippedOfFrontmatter);

  /**
   * Use cheerio to parse the the HTML in order to access headers.
   */
  const cheerio = cheerioParse(markdownItHTML);

  /**
   * Map each header from the cheerio HTML to an object containg data about the header.
   */
  const headers = Array.from(cheerio('.header-anchor')).map((anchor) => {
    /**
     * Store if the heading is the h1 which should be the page name
     */
    const self = cheerio(anchor).parent()[0].name === 'h1';

    /**
     * Get the text of the anchor
     */
    const text = cheerio(anchor).text().split('<')[0].trim();

    /**
     * Create the hash of the anchor but split out a question mark for out FAQ headings
     */
    const hash = '#' + slugify(text);

    /**
     * Set the title of the current page if the header is an h1 heading.
     */
    title = self ? text : title;

    /**
     * Return the header itself with metadata.
     */
    return {
      hash,
      text,
    };
  });

  /**
   * Return the page itself with metadata.
   */
  return {
    path,
    headers,
    title,
  };
};

/**
 * Get pages data for each page in our source directory.
 *
 * This will find markdown files and directories of markdown files,
 * read the contents of those pages and return metadata-like information
 * about each index.md file nest within the source directory.
 */
const getPages = () => {
  /**
   * Find the markdown in our source directory and reducer that per file to
   * be read markdown with a metadata like array of pages
   */
  return findMarkdown(OPTIONS.INCLUDE_DIR).reduce((reducer, file) => {
    reducer.push(readMarkdown(file));

    return reducer;
  }, []);
};

/**
 * Export the generatePages function
 */
export default (config) => {
  if (!config) {
    throw new Error(`
You must provide a configuration object as an argument to the "generatePages" function...
Options includes:
  {
    INCLUDE_DIR: string = '',
    EXCLUDE_DIRS: string[] = []
  }
    `);
  }

  Object.assign(OPTIONS, config);

  return getPages();
};

emersonbottero commented 1 year ago

@brc-dd I couldn't make it work reading the out folder (because all plugins runs in parallel... 😅, know I got what you suggested before)

So I decided to parse the md files to html instead and I was struggling to generate html files by my own so I'm computing the md files directly and it seems to be working.. (@staghouse, just saw you did exactly that.. 🤣 )

This md version is looking really fast.. (maybe my sample is to small.. and I still had to get the titles and descriptions correctly) but I could export the data as an virtual module and import it in the search component..

see this

After publishing this small working version I'll try to see if we can use the shared code from @staghouse and @mattjcowan ! thanks for those by the way 😁

P.S.: I guess we will have a plugin working soon. 🚀

emersonbottero commented 1 year ago

New Version Publish.. 🚀

extract data directly from md files (seems fast ⚡)
It parse the anchors ⚓
It runs in dev or build (it uses the rollup hooks)
no complex setup to make it work.. just add the plugin and you are good to go.
need beta tester ⚗

Next Improvements!

use the algolia component passing transformSearchClient suggested by @mattjcowan! Can we use it @brc-dd ?
depending on the results maybe it is better to rally convert to html and then run the parser!
add options to the plugin like preview length
add page title to the search? (not sure is revelant)
exclude Frontmatter data from the index

Awesome Features (not sure if easily implemented)

Allow user to chose the indexer library (in this case we should require a parser and a reader to populate the founded items list

brc-dd commented 1 year ago

Awesome work @emersonbottero! 🎉 I'll try it out. Regarding transformSearchClient, I'll check that out too. If it's feasible, then we can expose some options from our side too so you don't have to write custom component or patch stuff.

Akryum commented 1 year ago

I've built an "offline" search into Histoire with full-text capabilities using flexsearch and it's working really well for now.

Here are some implementation references:

Creating indexes and other utils functions: https://github.com/histoire-dev/histoire/blob/main/packages/histoire/src/node/search.ts#L55-L67

In a vite plugin load: https://github.com/histoire-dev/histoire/blob/62b3fd5c8edd8201294dd8aa7b71b03874906e16/packages/histoire/src/node/vite.ts#L320-L326

Usage in the frontend: https://github.com/histoire-dev/histoire/blob/62b3fd5c8edd8201294dd8aa7b71b03874906e16/packages/histoire-app/src/app/components/search/search-docs-data.ts https://github.com/histoire-dev/histoire/blob/62b3fd5c8edd8201294dd8aa7b71b03874906e16/packages/histoire-app/src/app/components/search/SearchPane.vue

emersonbottero commented 1 year ago

Hi @Akryum , could you tried to use my plugin and send me some feedback? it also uses flexsearch and is a vite plugin!

jd-solanki commented 1 year ago

I'm just commenting for the visibility

Thanks to the new code group feature now we are just waiting for internal/offline search feature like VuePress to move our docs from VuePress to VitePress.

emersonbottero commented 1 year ago

I'm just commenting for the visibility

Thanks to the new code group feature now we are just waiting for internal/offline search feature like VuePress to move our docs from VuePress to VitePress.

But I already did the offline search! 😆 (I have to fix something that changed in the last vitepress version, but works great and has a lot of features)

jd-solanki commented 1 year ago

oh, Great! Let me give it a try.

Thanks for sharing 👍🏻

ATQQ commented 1 year ago

Recently I tried Pagefind (power by rust) in Vitepress，Demo Site => @sugarat/theme

search style like this

Pagefind GitHub

ATQQ commented 1 year ago

I think the algolia search window style is very beautiful

💡 Whether the component style of a theme can be reused in a custom way to process the search content entered by the user

brc-dd commented 1 year ago

Whether the component style of a theme can be reused in a custom way to process the search content entered by the user

@ATQQ You can probably use https://github.com/xiaoluoboding/vue-command-palette to get similar UI as Algolia. It's highly customizable and should be easily integrable with Pagefind's JS API.

ATQQ commented 1 year ago

Whether the component style of a theme can be reused in a custom way to process the search content entered by the user

@ATQQ You can probably use https://github.com/xiaoluoboding/vue-command-palette to get similar UI as Algolia. It's highly customizable and should be easily integrable with Pagefind's JS API.

Thank you for the advice. I spent over a day to complete the project transformation, and the effect is quite good.

However, the documentation quality of vue-command-palette is really poor. I had to look at its source code to understand how to transform and integrate it. 😅

emersonbottero commented 1 year ago

@ATQQ , why you didn't used vitepress-plugin-search? .🤔

ATQQ commented 1 year ago

@ATQQ , why you didn't used vitepress-plugin-search? .🤔

"I have experienced this plugin before and found that its search box UI is not particularly aesthetically pleasing (which is quite critical 😄). Additionally, it does not provide built-in support for Chinese and requires additional configuration.

When planning the development of the vitepress-plugin-pagefind plugin, I took reference from its implementation 😋."

ATQQ commented 1 year ago

vitepress-plugin-pagefind

Offline full-text search based on pagefind

vuejs / vitepress