junhoyeo / threads-api

Unofficial, Reverse-Engineered Node.js/TypeScript client for Meta's Threads. Web UI Included.
https://threads.junho.io
MIT License
1.59k stars 134 forks source link

Download thread replies with pagination | Proxy for authorization | Disable minification #208

Open ibotfather opened 1 year ago

ibotfather commented 1 year ago

1. New function for loading replies to a thread with pagination support:

Please, instead of: const post = await threadsAPI.getThreads(postID);

Add a new function:

getThreadsLoggedIn: PaginationPostIDQuerier<GetUserProfileThreadsPaginatedResponse> = async (
    postID,
    maxID = '',
    options = {},
  ): Promise<GetUserProfileThreadsPaginatedResponse> => {
    if (!this.token) {
      await this.getToken();
    }
    if (!this.token) {
      throw new Error('Token not found');
    }

    let data: GetUserProfileThreadsPaginatedResponse | ErrorResponse | undefined = undefined;
    try {
      const res = await axios.get<GetUserProfileThreadsPaginatedResponse | ErrorResponse>(
        `${BASE_API_URL}/api/v1/text_feed/${postID}/replies/${maxID && `?paging_token=${maxID}`}`,
        { ...options, httpsAgent: this.httpsAgent, headers: { ...this._getInstaHeaders(), ...options?.headers } },
      );
      data = res.data;
    } catch (error: any) {
      data = error.response?.data;
    }
    if (data?.status !== 'ok') {
      if (this.verbose) {
        console.log('[USER Threads] Failed to fetch', data);
      }
      throw new Error('Failed to fetch user Threads: ' + JSON.stringify(data));
    }
    return data;
  };
interface PaginationPostIDQuerier<T extends any> {
    (postID: string, maxID?: string, options?: AxiosRequestConfig): Promise<T>;
}

------------------------------------ 2. Remove minification during installation ..Please make sure that when installing with npm install threads-api , unminified data is loaded. I can add a new feature to TS , but I can't submit to github . ------------------------------------ 3. Add proxy support to login() and syncLoginExperiments(): ....Yesterday I spent several hours to add proxy support for the authorization function in the minified .js - it was crazy .. but I did it. And now, every time I update the library, I will need to change the JS file again. So if you can, add "httpsAgent: this.httpsAgent" to the login() method in:

const requestConfig: AxiosRequestConfig = {
         method: 'POST',
         httpsAgent: this.httpsAgent,
         headers: this._getAppHeaders(),
         ..........

And also in the syncLoginExperiments method, "httpsAgent: this.httpsAgent" in:

const res = await axios.post(`${BASE_API_URL}/api/v1/qe/sync/`, this.sign(data), {
         headers: {
           ...this._getAppHeaders(),
           Authorization: undefined
           'Sec-Fetch-Site': 'same-origin',
         ...........
         httpsAgent: this.httpsAgent,

------------------------------------

P.S. I write code in PHP , Python , so TS is not very comfortable for me. For some reason, I can’t install from source (package errors, despite the fact that they are already installed)

Thanks!

ibotfather commented 1 year ago

UPD: I was able to figure out how to compile TS , so I attached the finished function in the previous post.

Also, I ask in every function where there is headers: add also httpsAgent: this.httpsAgent,

So that the proxy settings are applied to all requests (I already did it locally)

junhoyeo commented 1 year ago

..Please make sure that when installing with npm install threads-api , unminified data is loaded. I can add a new feature to TS , but I can't submit to github .

Hey @ibotfather I appreciate your effort! Can you try forking the repo and adding the changes there to make a pull request?

Remove minification during installation

Reasonable πŸ‘

Add proxy support to login() and syncLoginExperiments():

Totally missed it, proxy support to login was added in #204, but I think we still need to fix syncLoginExperiments, right?

....Yesterday I spent several hours to add proxy support for the authorization function in the minified .js - it was crazy .. but I did it. And now, every time I update the library, I will need to change the JS file again.

By the way, modifying the node_modules directly is bad practice. If you need to instantly fix the package yourself use something like patch-package or yarn patch.

ibotfather commented 1 year ago

@junhoyeo I write new function for get notifications:

getNotifications = async (filter, pagination) => {
    let params: Record<string, any> = {
        feed_type: 'all',
        mark_as_seen: false,
        timezone_offset: -25200,
        timezone_name: "America%2FLos_Angeles"
    };

    if (filter) {
        params.selected_filters = filter;
    }

    if (pagination) {
        params.max_id = pagination.max_id;
        params.pagination_first_record_timestamp = pagination.pagination_first_record_timestamp;
    }

    const queryString = Object.keys(params).map(key => key + '=' + params[key]).join('&');
    const url = `${BASE_API_URL}/api/v1/text_feed/text_app_notifications/?${queryString}`;

    let data: any | ErrorResponse | undefined = undefined;
    try {
        const res = await axios.get<any | ErrorResponse>(url, {
            httpsAgent: this.httpsAgent,
            headers: { ...this._getInstaHeaders(), ...options?.headers }
        });
        data = res.data;
    } catch (error: any) {
        data = error.response?.data;
    }

    if (data?.status !== 'ok') {
        if (this.verbose) {
            console.log('[Notifications] Failed to fetch', data);
        }
        throw new Error('Failed to fetch notifications: ' + JSON.stringify(data));
    }

    return data;
};

Hey @ibotfather I appreciate your effort! Can you try forking the repo and adding the changes there to make a pull request?

sorry, but really I don't know: how...

junhoyeo commented 1 year ago

@ibotfather Sure! Here's the steps if you want to contribute yourself:

  1. Click Fork in junhoyeo/threads-api.

    Screenshot 2023-07-19 at 3 10 51 AM
  2. You'll have your own repository in https://github.com/ibotfather/threads-api

  3. Use git to sync files to your local machine and edit files there β€” GitHub have a guide here!

  4. If you're new I would recommend using GitHub Desktop. It has a GUI & comes battery included so you only have to care about the code. πŸ˜†

ibotfather commented 1 year ago

And mark "seen":

notificationSeen = async () => {
      if (!this.token) {
        await this.getToken();
      }
      if (!this.token) {
        throw new Error('Token not found');
      }

      let data: any | ErrorResponse | undefined = undefined;
      try {
        const res = await axios.post<any | ErrorResponse>(
          `${BASE_API_URL}/api/v1/text_feed/text_app_inbox_seen/`,
          {},
          { httpsAgent: this.httpsAgent, headers: this._getInstaHeaders() },
        );
        data = res.data;
      } catch (error: any) {
        data = error.response?.data;
      }
      if (data?.status !== 'ok') {
        if (this.verbose) {
          console.log('[Notification Seen] Failed to perform', data);
        }
        throw new Error('Failed to perform notification seen: ' + JSON.stringify(data));
      }
      return data;
    };
ibotfather commented 1 year ago

@ibotfather Sure! Here's the steps if you want to contribute yourself:

My friend, I will definitely deal with git , but later. I just want to help :)

I have to urgently deal with my main development

stevenhao commented 1 year ago

+1 to getThreadsLoggedIn! Otherwise it seems there's a limit (around 15) that the graphql endpoint returns. Happy to help w/ a PR later