Using `rust-docs-json` as the documentation source and deprecate `search-index.js`

blackgear / rustdoc_seeker

rustdoc-seeker parse the `search-index.js` generated by `rustdoc` and provide fast searching ability

MIT License

5 stars 4 forks source link

Using `rust-docs-json` as the documentation source and deprecate `search-index.js` #18

Open bdbai opened 1 month ago

bdbai commented 1 month ago

The latest nightly toolchain (as of 2024-05-23) ships a component called rust-docs-json that provides nicely structured JSON format of the Rust standard library documentations. In light of the deprecation of the old search-index.js format, I think it's time to migrate to using the new JSON format. Any suggestions or recommendations?

Unresolved questions

[ ] where do we store the json files? A total of 53.2 MB JSON files seems too big to be checked in to a repository.
Ask the user for the json files instead of vendoring by this crate?

cc @blackgear @upsuper

upsuper commented 1 month ago

I don't think we should check in the JSON file anywhere. It can continue to require consumer to provide the data to create the instance. It would be good to show how to fetch this data, though. Having to install nightly toolchain would be a bit unfortunate.

upsuper commented 1 month ago

That being said... it has a few files inside, so it's not fully clear how we can make it work in the same way. Maybe it can accept a list of json strings?

bdbai commented 1 month ago

@upsuper

It would be good to show how to fetch this data, though. Having to install nightly toolchain would be a bit unfortunate.

We might also mimic what rustup does i.e. retrieving a toolchain metadata from static.rust-lang.org, parse it and download the archive containing rust-docs-json. A few lines of shell scripts will do the trick, maybe?

it has a few files inside, so it's not fully clear how we can make it work in the same way. Maybe it can accept a list of json strings?

I haven't thoroughly read the implementation yet, but a breaking change of data structures (or a complete rewrite😱) may be inevitable. What about focusing on one instance per file, and let users decide how to deal with multiple instances?