Pagination doesn't work well

[Boby024]

When putting "searchAllTweets" in the "do-while" it stop after the first call, i.e when the second is built with just "next_token" and/or "sinceId" it does work (output always null". Because I want to build my own "getTweetsRecursively" with some extra process in connection with my database. can you @redouane59 please explain to me how to solve it?

[redouane59]

Hey, I'm not sure to understand it well. Do you want to make only one call or make a recursive call. Did you just use the AdditionnalParameter argument ? You can set recursiveCall to true if you want it to loop.

[Boby024]

I want to build my own recursive method using "searchAllTweets" because I have more than two Twitter developer credentials. But when I call the method "searchAllTweets" and AdditionnalParameter with just "sinceId" and/or "next_toiken", it doesn't work

[redouane59]

Can you please leave here some code and logs if you have them.

[redouane59]

@Boby024 is it fixed ? If not please give me an update :)

[Boby024]

String newestId = null;
        String next;
        AdditionalParameters additionalParameters = AdditionalParameters.builder().recursiveCall(false).maxResults(100).startTime(searchTerm.getSearchFrom()).endTime(searchTerm.getSearchUntil()).build();
        do {
            TweetList tweetList = twitterClient.searchAllTweets(searchTerm.getTerm(), additionalParameters);
            if (tweetList == null || tweetList.getData() == null) {
                break;
            }

            result.add(tweetList);
            if (newestId == null) {
                newestId = tweetList.getMeta().getNewestId();
            }
            next = tweetList.getMeta().getNextToken();
            additionalParameters = AdditionalParameters.builder().recursiveCall(false).maxResults(100).nextToken(next).sinceId(newestId).build();

        } while(next != null);

[Boby024]

sorry for the late reply.

[redouane59]

Hmm I'm actually having the same behaviour. Did you try to make the same calls from postman ? Is it working ? I'm not sure that next_token can work with sinceId at the same time.

[Boby024]

I also tested it without "sinceId" (just with "next_token") but it just stopped after the first request.

[redouane59]

Strange, on my side it was working. Please write here your full code with the values to let me run it directly in a unit test. On parallel please try it in Postman and confirm to me that the behaviour is different. Thanks !

[Boby024]

private List getTweets(TwitterClient twitterClient, SearchTerm searchTerm) { List result = new ArrayList<>(); String newestId = null; String next; AdditionalParameters additionalParameters = AdditionalParameters.builder().recursiveCall(false).maxResults(100).startTime(searchTerm.getSearchFrom()).endTime(searchTerm.getSearchUntil()).build(); do { TweetList tweetList = twitterClient.searchAllTweets(searchTerm.getTerm(), additionalParameters); if (tweetList == null || tweetList.getData() == null) { break; }

        result.add(tweetList);
        if (newestId == null) {
            newestId = tweetList.getMeta().getNewestId();
        }
        next = tweetList.getMeta().getNextToken();
        additionalParameters = AdditionalParameters.builder().recursiveCall(false).maxResults(100).sinceId(newestId).nextToken(next).build();
    } while(next != null);

    return result;

}

[Boby024]

here is the part where I try to get the tweets. I also tested it with postman and it worked

[redouane59]

Please give me a full example without variable but concrete values instead, and correctly formatted. Thanks :)

[Boby024]

what do you mean ? however I used these values for the test { "term": "#covid lang:en", "search_from": "2021-06-21 11:30:00", "search_until": "2021-06-21 11:36:00" }

[redouane59]

I'm sorry but to be efficient, I am just asking you to post here a full testable code. Something I could just copy / paste and run inside a unit test without having to replace or change some lines. Thanks !

[Boby024]

it works now the parameter request wasn't gut here: how to perform it: https://developer.twitter.com/en/docs/twitter-api/pagination

[redouane59]

So there is no bug ? If so, could you close the issue ?

[Boby024]

we can close this issue now :)