search

miku-nyan / Overchan-Android

Meta Imageboard Client
http://miku-nyan.github.io/Overchan-Android
GNU General Public License v3.0
113 stars 55 forks source link

8ch back pages #64

Open jacekmusial opened 8 years ago

jacekmusial commented 8 years ago

In 8ch.net back pages (page1.html, page2.html, ...) larger than 2 are disabled, due to save on I/O write.

So Overchan propose "catalog" option to browse threads on this imageboard.

"Catalog" option download file catalog.json, for example: 8ch.net/pol/catalog.json and fetch thread list from there.

Problem is: that catalog.json file contain threads only from page 1 and page 2. So you end up with ~20 threads, while in real there are much more threads.

What I propose is, of course, fetch thread list directly from catalog.html.

I'm writing here, because I want to ask, what tools are most suitable for that job. For http(s) request, use http-request by Kevin Sawicki?

Also, maybe I'm wrong and there is better option to fetch "real" thread list from 8ch.net imageboard?

miku-nyan commented 8 years ago

Problem is: that catalog.json file contain threads only from page 1 and page 2.

Sorry, I didn't check it at once.

What I propose is, of course, fetch thread list directly from catalog.html.

Right now I have no time to fix it.

I'm writing here, because I want to ask, what tools are most suitable for that job.

For example, look how it is made in krautchan module or in generic wakaba parser. https://github.com/miku-nyan/Overchan-Android/tree/master/src/nya/miku/wishmaster/chans/krautchan https://github.com/miku-nyan/Overchan-Android/blob/master/src/nya/miku/wishmaster/chans/krautchan/KrautReader.java https://github.com/miku-nyan/Overchan-Android/blob/master/src/nya/miku/wishmaster/api/util/WakabaReader.java

For http(s) request, use http-request by Kevin Sawicki?

Why? What do you mean?

miku-nyan commented 8 years ago

And whether it is necessary at all? As far as I know 8chan plans to switch to the new engine (infinity-next).

jacekmusial commented 8 years ago

Problem is: that catalog.json file contain threads only from page 1 and page 2.

Sorry, I didn't check it at once.

No problem, we all are human :smiley:

What I propose is, of course, fetch thread list directly from catalog.html.

Right now I have no time to fix it.

But I have. And I'm preparing to write it. Thanks for examples, this is what I was looking for.

For http(s) request, use http-request by Kevin Sawicki?

Why? What do you mean?

Nevermind. I forgot that you have better suited Http(s) class.

jacekmusial commented 8 years ago

As far as I know 8chan plans to switch to the new engine (infinity-next).

Yep, money-hungry inifnity-next. I forgot about that, sorry.

I can't find any information about when they would switch to new engine. And I have feeling that this will not be so quick (I don't have any proof for that).

miku-nyan commented 8 years ago

Maybe January 1, 2016, but I don't know exactly.

(from old infinity repo)

This software is DEPRECATED and UNMAINTAINED. Come January 1, 2016, no security fixes will be merged in.

TheGag96 commented 8 years ago

This is still a problem, and there aren't any plans to switch to Infinity any time soon. Do you think you could look into this sometime? Thanks.