search

kentcdodds / kentcdodds.com

My personal website
https://kentcdodds.com
Other
2.34k stars 624 forks source link

Search KCD page #5

Open kentcdodds opened 3 years ago

kentcdodds commented 3 years ago

I'd like a page where you can search through all my content everywhere. This has been discussed quite a bit: https://github.com/kentcdodds/kentcdodds.com/issues/107

The biggest challenge is creating a way to search through and index all that content. Not sure of the best way to go about it...

dane-stevens commented 2 years ago

redisearch would handle this nicely. Here's a blog post describing the basics behind it: https://redis.com/blog/building-real-time-full-text-site-search-with-redisearch/

Let me know if this is something I should look into further.

I do have question about the current redis implementation with primary and replica clients. Could you briefly explain the setup?

kentcdodds commented 2 years ago

Basically I have two connections, one to the primary region redis for writes and one to the replica region for reads. Fly handles automatic replication for me. That's it.

kentcdodds commented 2 years ago

The biggest challenge isn't text search. The biggest challenge is all the APIs (or non-APIs) you'd have to use across all my content distribution channels. Mostly YouTube, Egghead, Frontend Masters, podcasts I've been a guest on, and my own site.

dane-stevens commented 2 years ago

Interesting. Yeah, sounds like you would need a combination of cron-jobs, webhooks (if the APIs support them when new content is added) and some manual URL entry for scraping to get the data into Redis. Once the data is all ingested, the search part is easy. It is probably something that could be started with one data type and more added later.

Youtube push notifications: https://developers.google.com/youtube/v3/guides/push_notifications Not sure if Egghead or frontend masters have an API or not.