Firebase API does not expose several "hidden" filters/lists of data.

[jperras]

I was attempting to make a contribution to Haxor-News by adding these missing filters/list types, but it seems as though the public Firebase API does not include them.

1. The so-called classic mode, which filters story results so that only submitted stories by longstanding members are present: https://news.ycombinator.com/classic
2. The bestcomments filter, which aggregates highest voted recent comments: https://news.ycombinator.com/bestcomments
3. The active filter, which aggregates the most active current discussions: https://news.ycombinator.com/active
4. The best filter, which simply lists the highest voted recent links: https://news.ycombinator.com/best

I believe the only way to add these in would be to manually parse the HTML of those pages to extract out JSON structures that could be parsed by the existing implementation, which is quite the departure from the current spirit of the codebase.

Is there any interest in this, or would it simply add more complexity than it's actual worth?

[donnemartin]

Thanks for the tip on classic, bestcomments, and active. I was only aware of best, which through trial and error I discovered was actually available, although not documented:

https://github.com/donnemartin/haxor-news/blob/master/haxor_news/lib/haxor/haxor.py#L181-L190

It's possible classic, bestcomments, and active might also be available, just not documented. Might be worth a shot :)

I believe the only way to add these in would be to manually parse the HTML of those pages to extract out JSON structures that could be parsed by the existing implementation, which is quite the departure from the current spirit of the codebase.

Curious about how popular these filter/list types are to try to gauge the tradeoff of supporting them vs the potential additional complexity. Do you feel you use these filters often?

The Hacker News API also doesn't support some key APIs for voting, commenting, and submitting. To knock these out of the TODO we might need to take the plunge and move away from relying solely on the API.

[jperras]

I was only aware of best, which through trial and error I discovered was actually available, although not documented.

Best way to find out about those ones is to read the (somewhat dated, somewhat incorrect) original Arc source code :wink: (note that the noobs filter no longer exists, hence the "dated" reference)

It's possible classic, bestcomments, and active might also be available, just not documented. Might be worth a shot :)

I thought as much, and attempted to hit the Firebase API endpoints with the variants that I could think of:

https://hacker-news.firebaseio.com/v0/classicstories
https://hacker-news.firebaseio.com/v0/classic
https://hacker-news.firebaseio.com/v0/classic_stories
https://hacker-news.firebaseio.com/v0/classic-stories

etc. but to no avail.

Curious about how popular these filter/list types are to try to gauge the tradeoff of supporting them vs the potential additional complexity. Do you feel you use these filters often?

I personally use the classic filter about 90% of the time. However, due to their inherent obscurity, I'm almost certain that only a very small subset of people that frequent HN use any of these "hidden" filters.

I guess the better question would be: What is the conditional probability of a person using a hidden filter given that they are a haxor-news user? Hard to tell, really. I'm certainly the first person to have brought it up for this project, at least :-).

The Hacker News API also doesn't support some key APIs for voting, commenting, and submitting. To knock these out of the TODO we might need to take the plunge and move away from relying solely on the API.

I feel that if a move away from pure API usage is forthcoming, then perhaps it's a worthwhile endeavour. The problem with such a process is, as always, the brittleness of the resulting parser due to its dependence on HTML structures that can change at any time.

[donnemartin]

Best way to find out about those ones is to read the (somewhat dated, somewhat incorrect) original Arc source code

Cool! Thanks for pointing this out, was fun to dig into the code.

I feel that if a move away from pure API usage is forthcoming, then perhaps it's a worthwhile endeavour. The problem with such a process is, as always, the brittleness of the resulting parser due to its dependence on HTML structures that can change at any time.

I agree. In the past I've used a Python non-official HN API--unfortunately it is now broken. The parsing code is also considerably more complex.

I'm certainly the first person to have brought it up for this project, at least :-).

I feel we could use some more input on the demand before diving in. I'll tag this issue as feedback-requested.