TikTokScraper.user return always empty collector

[ambigos1]

Hi. Using latest version 1.4.33 using high quality residential proxy.

No matter how many attempts I am trying with different public profiles. I am always getting empty collector when invoking TikTokScraper.user(...) function.

Any help?

[vanities]

I'm experiencing the same thing... the cli appears to be working?

The CLI isn't working either.

[smachs]

Same issue!

[andrekerygma]

Same here, I'm trying to collect the data but nothing comes in return

[lu3do]

+1 The issue first occurred about 3 hours ago, and it now working since.

[BennyTheDev]

same (empty results)

[lu3do]

music() is also having the same issue, so maybe all the multi posts scrape functions are affected by this same issue.

[BennyTheDev]

I think the parser needs to be updated as it seems they updated something. the returned body looks fine and no 403 or stuff like that.

edit: hm, no, seems some requests return something, some are empty.

[tblanks1980]

not sure if this is what is affecting other people, but I wanted to share the response for my issue.

the following is response.body I received from a TikTokScraper.hashtag scrape request (i added console.log(response.body) on L171 of node_modules/tiktok-scraper/build/core/TikTok.js to get this value)

{
    statusCode: 10000,
    verifyConfig: {
        code: 10000,
        type: 'verify',
        subtype: 'slide',
        fp: 'verify_334b645b4b3b6af8b6378ced7bbb5583',
        region: 'va',
        detail: 'lUY*IO07axSjFFRhxXFR7Wn*rIF4ABSJA5KCuvyVewAjyVYihbLdSasDVBL-N0na6MUrN5OZ-mZumWqra16WOm2giqt6QUZ-tU84iS4ObEVyZIWgFY2-wj05CducfiBU4Q7H21TcKiuzmVorC6rMyJJz09xVCipH39p0FqY2lCxf9TPJ9oyr99x71RNJQud3cFgRs8WJQr3d5B5VJAIjW0Z6OauCa1DYVJDZm7svG9OWzO7sql32Dhnp0Hhv6XgmzMzdH0MKBVPAfGcF3Zp3z3M4mxkfAbPPU-LEJO1YZTCIPabR4NnNUQEJfyDP2T*Ic0J6A6ZS3vpf1SlT04HOUPCdekmHS*McTp3XZ*ZFcynnO9C35xKlrhnQPxHQ3odHDJgWoQ..'
    }
}

i don't know if this means there is some new behavior from TikTok where a captcha-like verification is required...

The only request type that works for me right now is TikTokScraper.getUserProfileInfo(). The others are all getting this verify problem.

Sorry to tag you @drawrowfly but I think this is possibly a breaking change for the library. By the way, thanks for the awesome work!

[crispmama]

I use TikTokScraper.user() and starting today it returns an empty collection, I also tried to use signUrl() to generate the signatures but still, it would return an empty response.

It seems like Tiktok is somehow able to detect the difference between the tokens generated by them vs by the library.

[amirdezhkam]

Same issue for me. I think it's for User-Agent request header. I found it by trying to get posts by tiktok frontend APIs and when I send real user agent in headers everything works good, but I couldn't fix it in the library.

[crispmama]

Same issue for me. I think it's for User-Agent request header. I found it by trying to get posts by tiktok frontend APIs and when I send real user agent in headers everything works good, but I couldn't fix it in the library.

For me, I tried generating the signature via library and used real user agent and sent request via got but still got empty response back. I was under impression that tiktok is somehow able to detect the signature but I will also try inspecting this further.

[BennyTheDev]

They seem to use akamai now. I see an _abck entry in the cookies.

[asportnoy]

Even if I try to run the function directly on the site (which worked before) the request has an empty array, so maybe they added another parameter or changed which function is used altogether?

@drawrowfly are you aware of this issue?

[BennyTheDev]

Even if I try to run the function directly on the site (which worked before) the request has an empty array, so maybe they added another parameter or changed which function is used altogether?

@drawrowfly are you aware of this issue?

it's akamai

[drawrowfly]

Even if I try to run the function directly on the site (which worked before) the request has an empty array, so maybe they added another parameter or changed which function is used altogether?

@drawrowfly are you aware of this issue?

Looking into it now

[asportnoy]

Here's the fix @drawrowfly In signature: 1) Replace JSDOM with https://npmjs.com/package/window 2) Keep the same options as with JSDOM, but add the userAgent option. This must be the same user agent as is used for the actual request. In main code: 3) Re-implement verifyFp. The existing code for that works fine.

Edit: I think the x-secsdk-csrf-token might not be needed anymore, you should see if you are able to remove it.

[asportnoy]

I implemented the above fix for myself and can confirm it works. My system has recovered and is back to a usual amount of functioning proxies.

[youngvo]

@asportnoy Could you please make a pull request for your hotfix?

[drawrowfly]

Here's the fix @drawrowfly In signature:

1. Replace JSDOM with https://npmjs.com/package/window
2. Keep the same options as with JSDOM, but add the userAgent option. This must be the same user agent as is used for the actual request. In main code:
3. Re-implement verifyFp. The existing code for that works fine.

Edit: I think the x-secsdk-csrf-token might not be needed anymore, you should see if you are able to remove it.

no need for extra package, we only need to set navigator UA as it was before few month ago , in case of jsdom it is done through the ResourceLoader

[drawrowfly]

@1.4.34 and @asportnoy regarding 'x-secsdk-csrf-token' decided to leave them in case they will change their minds