algolia / hn-search

Hacker News Search
http://hn.algolia.com
Other
550 stars 74 forks source link

`[dead]` posts not indexed #240

Closed PatMyron closed 1 year ago

PatMyron commented 1 year ago

[flagged] posts are still indexed, but [dead] posts are not https://news.ycombinator.com/newsfaq.html Examples: https://news.ycombinator.com/item?id=29220729 https://news.ycombinator.com/item?id=37790899

A setting to include [dead] posts could be useful

jeffrey-slentz-algolia commented 1 year ago

We avoided indexing [dead] posts for storage optimization, since there's not much information left except the author. What's the use case for searching for deleted posts or comments?

PatMyron commented 1 year ago

not much information left

As mentioned in https://news.ycombinator.com/newsfaq.html, there's a setting to show the rest of their information:

Dead posts aren't displayed by default, but you can see them all by turning on showdead in your profile

jeffrey-slentz-algolia commented 1 year ago

Ah, I missed the difference between dead and deleted. Can you share an example of a [dead] post?

PatMyron commented 1 year ago

Examples: https://news.ycombinator.com/item?id=29220729 https://news.ycombinator.com/item?id=37790899

These examples I linked are [dead] and only show for users who enabled showdead

jeffrey-slentz-algolia commented 1 year ago

Oh I see. We use the API to retrieve these posts, so the issue is that that information is not visible to us to index.

https://hacker-news.firebaseio.com/v0/item/29220729.json?print=pretty https://hacker-news.firebaseio.com/v0/item/37790899.json?print=pretty

Without a way to retrieve the original story text, we can't make it searchable, unless there'd be a great value in showing [dead] posts when searching for posts by a specific author.

For example, this search by:lalajollyton would show two results instead of the one result (and the dead item would have no title/link, just [dead]). https://hacker-news.firebaseio.com/v0/item/37785072.json?print=pretty https://hacker-news.firebaseio.com/v0/item/37790899.json?print=pretty

jeffrey-slentz-algolia commented 1 year ago

I'm going to close this one as won't do due to API restrictions. Thanks Pat!