search

carcabot / tiktok-signature

Generate tiktok signature token using node
733 stars 283 forks source link

Cannot run with Docker on Linux + is it possible to dynamically get hardcoded values? #188

Closed foxt451 closed 1 year ago

foxt451 commented 1 year ago

In examples, there are different hardcoded values like msToken, userAgent, deviceId, x_bogus etc. They are great when I test my scraper locally on mac, but when I deploy it to a remote platform (that runs in Docker, Linux), all I get are empty responses. I assume it's because of a mismatch between platform/user-agent or something like that. And I have no idea how to change user-agent without breaking all other hardcoded values.

Would it be possible to somehow explain in the examples how to dynamically get all other values, or perhaps there is some available Dockerfile/source code example that works on Linux?

fluorescentoast commented 1 year ago

I suspect your issue is with your deployed docker container and not the hardcoded values. I ran into a similar problem.

Does the docker container run locally as expected?

foxt451 commented 1 year ago

@fluorescentoast Hi, thanks for reply! I build it with --platform linux/amd64, and when I run it locally with the same --platform linux/amd64 I can't reproduce the exact error that happened remotely on the platform, but it still doesn't work (for another reason, navigation times out). If i remove --platform flag locally (I'm on Mac M1) it runs somewhat fine, navigation times out just in some cases. On remote platform (which runs Ubuntu) navigations never time out, I just get empty API responses.

You've faced a similar problem, did you manage to solve it?

I see that the problem is with docker container, but I guess it would be much easier to know how to modify user-agent value to Ubuntu

fluorescentoast commented 1 year ago

What region is your platform hosted on? could be a similar issue to https://github.com/carcabot/tiktok-signature/issues/172

foxt451 commented 1 year ago

@fluorescentoast Hmm, I doubt it. I used US proxy locally and non-US proxy on platform and locally, and the result was the same (worked locally, didn't work on platform). I guess I'll try to run the scraper in some ubuntu virtual machine to see if it's problem with linux or with platform specifically

foxt451 commented 1 year ago

Ok, so I've finally got my hands to run it on my Ubuntu machine. And it works, which means it's not a linux issue

foxt451 commented 1 year ago

Oh, so now I tried to run the same axios request (i.e. with same headers and all) locally, and it works... Which points that the problem is not with playwright but with the last request. Very strange. Will try using simple fetch. Maybe it's indeed IP address thing, but that's strange, since proxies are the same

UPD: turns out I forgot to add proxy to axios request