Rishikant181 / Rettiwt-API

A CLI tool and an API for fetching data from Twitter for free!
https://rishikant181.github.io/Rettiwt-API/
MIT License
319 stars 32 forks source link

getTweets does not return tweets in order of descending date #280

Closed flashmercurymcfly closed 1 year ago

flashmercurymcfly commented 1 year ago

Unsure if this is intentional, but using the getTweets method where I specify a user with fromUsers doesn't always return x number of tweets in descending order of date.

Usage of getTweets

getTweets({'fromUsers' : ['arknights_fan'], 'links': 'true'}, 10)

Here is the response I got as of writing

[
  {
    id: '1636639456232108032',
    createdAt: 'Fri Mar 17 08:03:48 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: '[New Outfit].Astesia - Fantastic Crafts (CC reward) https://t.co/pl9gHun0Pv.',
    replyTo: null,
    lang: 'en',
    quoteCount: 6,
    replyCount: 1,
    retweetCount: 27,
    likeCount: 48
  },
  {
    id: '1632695293010649088',
    createdAt: 'Mon Mar 06 10:51:06 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: 'https://t.co/VQMULUgN6l.',
    replyTo: '1632695264568852480',
    lang: 'zxx',
    quoteCount: 0,
    replyCount: 0,
    retweetCount: 2,
    likeCount: 5
  },
  {
    id: '1633017019934339073',
    createdAt: 'Tue Mar 07 08:09:32 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: 'Arknights × Monster Hunter collab event is now live on CN! https://t.co/FwEbIZ4OVl.',
    replyTo: null,
    lang: 'en',
    quoteCount: 5,
    replyCount: 0,
    retweetCount: 60,
    likeCount: 245
  },
  {
    id: '1632638692824334338',
    createdAt: 'Mon Mar 06 07:06:11 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: 'https://t.co/HPxik40HkG.',
    replyTo: '1632638623488303105',
    lang: 'zxx',
    quoteCount: 1,
    replyCount: 1,
    retweetCount: 11,
    likeCount: 30
  },
  {
    id: '1636639503116017665',
    createdAt: 'Fri Mar 17 08:03:59 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: 'https://t.co/wuiokUNUQ2.',
    replyTo: '1636639456232108032',
    lang: 'zxx',
    quoteCount: 1,
    replyCount: 0,
    retweetCount: 12,
    likeCount: 11
  },
  {
    id: '1632638752681254914',
    createdAt: 'Mon Mar 06 07:06:26 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: 'https://t.co/PtzbbXc4Up.',
    replyTo: '1632638692824334338',
    lang: 'zxx',
    quoteCount: 0,
    replyCount: 0,
    retweetCount: 12,
    likeCount: 24
  },
  {
    id: '1636205822979231744',
    createdAt: 'Thu Mar 16 03:20:42 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: '[PV] Contingency Contract #12 Operation Basepoint https://t.co/VBsKFFXkSm.',
    replyTo: null,
    lang: 'en',
    quoteCount: 66,
    replyCount: 2,
    retweetCount: 204,
    likeCount: 363
  },
  {
    id: '1636205827370414081',
    createdAt: 'Thu Mar 16 03:20:43 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: 'https://t.co/P5pAyiLT20.',
    replyTo: '1636205822979231744',
    lang: 'zxx',
    quoteCount: 6,
    replyCount: 0,
    retweetCount: 32,
    likeCount: 71
  },
  {
    id: '1632695264568852480',
    createdAt: 'Mon Mar 06 10:50:59 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: 'New furniture set https://t.co/DTY833ZTuh.',
    replyTo: null,
    lang: 'en',
    quoteCount: 0,
    replyCount: 1,
    retweetCount: 8,
    likeCount: 31
  },
  {
    id: '1632638623488303105',
    createdAt: 'Mon Mar 06 07:05:55 +0000 2023',
    tweetBy: '1143748652193607680',
    entities: { mentionedUsers: [], urls: [], media: [Array], hashtags: [] },
    quoted: undefined,
    fullText: '[New Outfit].Scene - Betsushi https://t.co/BhaYWnITOO.',
    replyTo: null,
    lang: 'en',
    quoteCount: 3,
    replyCount: 1,
    retweetCount: 34,
    likeCount: 145
  }
]

As you can see, the list is definitely not sorted by the tweet's created date (I'm not sure how it's sorted actually). Is there an existing way to fix this that I'm unaware of or would this require a change? There's a URL parameter "f=live" that appears when using twitter's basic search and selecting "Latest" but I don't know if that'd be helpful here.

Rishikant181 commented 1 year ago

That's just how data is fetched from Twitter, unfortunately.

The data that is returned from Twitter is always unsorted. It can be sorted on our side, at the cost of a bit higher (although unnoticeable) latency.

Would it be better if an option was added to get sorted data?

flashmercurymcfly commented 1 year ago

To confirm, in the example I gave the above list would be sorted by date as opposed to the data from twitter being sorted then returning count number of tweets correct?

Rishikant181 commented 1 year ago

Correct sir

flashmercurymcfly commented 1 year ago

If it's not too much trouble for you to implement I'd definitely find it useful.

Rishikant181 commented 1 year ago

I will implement it after discussing with @dvnasutosh .

Owing to my exams, any feature updates will come after a week from now.

Thanks for sharing your thoughts. Will work on it for sure!

flashmercurymcfly commented 1 year ago

Thanks for considering my suggestion. Really appreciate it especially with the mess twitter's API has been the past month or so.

Rishikant181 commented 1 year ago

Yeah, lots of migrations had to be done on our end too, but it has been dealt with.

Have a good day!

dvnasutosh commented 1 year ago

the problem is with the exclusion of "include_ext_views" param and direct fetch of data from "globalobjects". What the request does is sends all the required data under "globalobjects" and send the sort format under timelines. `