search

fastrepl / canary

🐤 Canary provides UI primitives for building modern search-bar for docs with self-hostable infrastructure.
https://getcanary.dev
MIT License
43 stars 3 forks source link

Migrate to spider-rs #13

Open yujonglee opened 3 months ago

yujonglee commented 3 months ago

https://github.com/spider-rs/spider

This looks very mature project.

Our current implementation:

https://github.com/fastrepl/canary/blob/62346f2a89b915f4e62c1b8decfcee2e51c7e5d7/core/lib/canary/crawler.ex#L1-L3

yujonglee commented 3 months ago

We will use site map based crawler most of the time from c9a4a42894c22985e8ac5450dcba911ebfdb6d44. So this is not important.