Pagination for the regular fetch() on feeds.

[stevenlafl]

Right now here is the WIP: https://github.com/stevenlafl/threads-web-client/blob/master/src/pages/api/feed.ts

and associated Feed component: https://github.com/stevenlafl/threads-web-client/blob/master/src/app/Feed.tsx

I need to get it to display more. It looks like I'm limited to 8, and refreshing may or may not get me a similar result set. Is there pagination available?

[elijah-wright]

yeah I saw pagination in there but wanted to get the base stuff in before I worried about that stuff. there should be a token in there that you can use but idk how you use it. like I said, working on getting my setup back up

fwiw i'm also working on a web client so the stuff that you're bringing up is stuff i'm experiencing

[elijah-wright]

@stevenlafl the endpoint for the timeline is gzipped for some reason but look at line 7 of this file: https://github.com/threadsjs/threads.js/blob/main/src/managers/FeedManager.js you need to pass the max_id you get from the response of this endpoint to the body of the next one.

[stevenlafl]

Like this?

FeedManager.js

const RESTManager = require('./RESTManager');

class FeedManager extends RESTManager {
    async fetch(max_id) {
        return await this.request('/api/v1/feed/text_post_app_timeline/', {
            method: 'POST',
            body: 'pagination_source=text_post_feed_threads' + (max_id ? '&max_id=' + max_id : ''),
        })
    }

    ...
}

module.exports = FeedManager;

index.d.ts

...
declare module '@threadsjs/threads.js/src/managers/FeedManager.js' {
    import RESTManager from "@threadsjs/threads.js/src/managers/RESTManager.js";
    export default class FeedManager extends RESTManager {
        fetch(max_id?: number): Promise<any>;
        fetchThreads(user: string): Promise<any>;
        fetchReplies(user: string): Promise<any>;
        recommended(): Promise<any>;
    }
}
...

I went ahead and used the RESTManager in my implementation for now for infinite scroll. I am not sure if it works for certain, but it does get me a new result set without erroring.

[thisisdice]

I might be wrong but I don’t think that’s paginating. I think your fetching a new feed on each call which might eventually flag as it doesn’t follow normal usage. The app paginates as you scroll taking a gzipbody with values such as the below.

{"media id: "3143463776240999995 180569377", "version": 24 , "media pet": 1.0, "time info: {"10': 8920, "25: 8920, "50": 8920, "75": 8716}, 'was_share _tapped": false), {"media_id": "3143527582710965035403375573" "version": 24, "media pct": 0.9230769, "time info': {10: 865, "25: 865, "50: 865, "75": 254}, "was_share _tapped" :false}, {"media_id": "3143533596829412672 40730850" version": 24, "media pct":0.9594017, "time_ info": (10": 347, "25":347, "50": 347, "75" :3473, "was_share_tapped" false), {"media id': "3143519892437762445 403375573" "version" 24, "media pet" :1.0, "time info" (10" 1689, "25 1689, "50' 1079, "75":6197, "was_share_tapped" false), {"media_id": "3143528201681201769 403375573" "'version":24, "mediapct":1.0, "time info" ('10" :8920, 25":8514, "50": 8514, "75": 8514}, "was_share_tapped": false), ('media id":"3143523828357551367 180569377" "version": 24, "media_pct' :1.0, "time info": (™10": 1476, "25":1476, "50" :1476, "75" :1476), "was_share_tapped" false), {"media id': "3143528196759601588 27288745" "version": 24, "media pct": 1.0, "time info': (™10": 1274, "25":1274, "50": 1274, "75" :1274}, "was_share_tapped": false), "media id": "3143503512472939477 4053226" , "version": 24, "media pct: 1.0, "time info": (10" 206, "25": 206, "50" 206, 75":2067, "was_share_tapped": false), {"media id': "3143486442730059357 51091500" "version": 24, "media pct: 1.0, "time_info': {10":205, "25" :205, "50': 205, "75": 205}, "was_sharetapped": false}. {"media id": "3143468471916764278 395761959" "media pet":1.0, "time info": ("10":205, "25":205, "50": 205, "75": 205}, "was_share_tapped": false). 'media id" "3143530367872313738 4094723", "Version'" 24, 'media pet" 0.93615873, 'time info" ("10" 205, "25 - 205, 50": 205, "75" 143), "was_share_tapped" :false), ("media id': "31434591738320720963225079059* "version" :24, "media pet': 1.0, "time info': (*10": 8716, "25" :8716, "50":8716, "75" :8716), "was share_tapped":false) ]

[elijah-wright]

I'll take a look at this in the morning

[elijah-wright]

what thisisdice shared is the analytics that Meta is getting from your scrolling. so if you looked at a post for longer, it notes that down. idk what that stuff means but that's just it. there's a lot of analytics to threads lol, in that timeline endpoint Meta even gets your battery level

when you do a POST request to the timeline endpoint, what changes in the body when you pull to refresh and when you paginate is the reason (which changes to "pagination") and the max_id param, which you get from next_max_id from the previous response.

I'll see what I can do today about this

[stevenlafl]

One more thing to be aware of is that in implementing viewing replies with posts.fetch() we also need pagination. I see:

[thisisdice]

@elijah-wright @sooluh @stevenlafl I’m able to pull endpoints, requests and responses but I’m not the best at deciphering them into reusable components. Any suggestions for a collaborative effort to get as many of them in welcome! Should I start dumping them here for others to have a go?

[micheltucker]

@stevenlafl did you figure out how to get the proper feed? I am not even sure the meta engineers know. Every time I open the app I see something new 🤣

[elijah-wright]

@thisisdice sure

[stevenlafl]

@stevenlafl did you figure out how to get the proper feed? I am not even sure the meta engineers know. Every time I open the app I see something new 🤣

Hah, I don't see duplicates as I scroll down. Doesn't mean what I did was correct, I just have to trust @elijah-wright. But I have it to where I can refresh and see new stuff if I want. Otherwise just keep scrolling. Even if there are duplicates, I've got it to where it removes them before trying to display.

I managed to get it to display things like this (quote posts only showing links with no text in body of post), or even quotes of replies.

When I reply to a post, I get back the full renderable post, so that feeds right back into my component structure. A few quirks but it works well. @elijah-wright is a wizard for making any of this even possible.

@birobirobiro has been assisting me with some of the styling too.

[elijah-wright]

thank you lol!

[CryptaBean]

https://i.instagram.com/api/v1/feed/text_post_app_timeline/

[decoded gzip] URLEncoded form
has_camera_permission: 0
feed_view_info:        [{"media_id":"3143468282492089976_702310471","version":24,"media_pct":1.0,"time_info":{"10":37820642,"25":37820642,"50":37820440,"75":37820239},"was_share_tapped":false},{"media_id":"3143480192010578160_1626596167","version":24,"media_pct":0.37179488,"time_info":{"10":2481,"25":2073,"50":0,"75":0},"was_share_tapped":false}]
phone_id:              d9f9f6a3-****-4ac3-b4b1-0e6004702f82
reason:                cold_start_fetch **<---- this changes to pagination for subsequent** requests
battery_level:         75
timezone_offset:       3600
pagination_source:     text_post_feed_threads
device_id:            ****4c42-6663-****-884b-66418b4bb843
request_id:            ****8f5e-dea0-****-a3d6-68484e6faed1
is_pull_to_refresh:    0  **<---- this changes 1 for homepage refresh**
_uuid:                 ****4c42-6663-****-884b-66418b4bb843
is_charging:           0
is_dark_mode:          0
will_sound_on:         0
session_id:            ****8bd-4699-****-a42d-cf267108094b
bloks_versioning_id:   5f56efad68e1edec7801f630b5c122704ec5378adbee6609a448f105f34a9c73

https://i.instagram.com/api/v1/notifications/badge/

content-type: application/x-www-form-urlencoded; charset=UTF-8
phone_id:  d9f9f6a3-****-4ac3-b4b1-0e6004702f82
user_ids:  35048****
device_id: ****4c42-6663-****-884b-66418b4bb843
_uuid:     ****4c42-6663-****-884b-66418b4bb843

returns

json
{
    "badge_payload": {
        "35048****": {
            "badge_count_map": {
                "ac": 1
            },
            "total_count": 1
        }
    },
    "status": "ok"
}

[stevenlafl]

Confirmed that this is right, however the next_max_id ends with an equal sign which needs to be urlencoded (%3D)

So it becomes:

const RESTManager = require('./RESTManager');

class FeedManager extends RESTManager {
    async fetch(max_id) {
        return await this.request('/api/v1/feed/text_post_app_timeline/', {
            method: 'POST',
            body: 'pagination_source=text_post_feed_threads' + (max_id ? '&max_id=' + encodeURIComponent(max_id) : ''),
        })
    }

    ...
}

module.exports = FeedManager;

https://github.com/threadsjs/threads.js/pull/49

[elijah-wright]

should be fixed