benjaminguinaudeau / tiktokr

An R Scraper for Tiktok
Other
49 stars 7 forks source link

User not found... #13

Closed stevesteve2 closed 3 years ago

stevesteve2 commented 4 years ago

Hi! So this was working last night with the exact same code and is not working anymore? Wondering if you're making more changes (or maybe I changed something on my end)...

This is my code:

devtools::install_github("benjaminguinaudeau/tiktokr", force = TRUE) library(tiktokr) library(reticulate) use_python(py_config()$python) py_install("requests", pip = TRUE) tk_install() tk_init()

user_posts <- tk_posts(scope = "user", query = "willsmith", n = 1000)

And it's sharing a user not found error for all users I use:

! [2020-11-20 11:28:54] u-willsmith (not found)

Thanks!

benjaminguinaudeau commented 4 years ago

I'm sorry for the issues. Ok, I had some issue to reproduce the error, because it's not systematic.

I understand now, that this is caused by Captcha detection, which is only sometimes triggered and can be overcome by adding identification cookies to the request.

With the cookie, I have not experienced any blocking issue. In case it should come back, I also add a stop(), that would break if the request failed because of an authentication problem.

I'll update the readme with an explanation of how to get the cookie and push in a moment

adamkthecoder commented 4 years ago

Hi! Thanks for fixing the previous error. I am wondering if you know why I might be getting the following error when I try to run the code:

adamkthecoder commented 4 years ago

Warning message: In get_data(url, ...) : empty response ; something was probably wrong with the request

benjaminguinaudeau commented 4 years ago

Hi,

What code are you running exactly? The error you get indicates an empty response from tiktok, which can indicate that you're blocked or that the query was wrongly specified.

Did you add the tiktok cookie using tk_auth()? We are not fully done with the maintenance of the documentation, though the captcha problem seems to be solved now.

adamkthecoder commented 4 years ago

Hi Benjamin, thanks for the quick response. This is when I am running this:

user_posts <- tk_posts(scope = "user", query = "willsmith", n = 50, cookie = cookie)

adamkthecoder commented 4 years ago

To give you a broader overview, here is what I have done:

install.packages("devtools")
library("devtools")
install_github("benjaminguinaudeau/tiktokr")

library(tiktokr)
library(reticulate)
use_python(py_config()$python)

tk_install()

tk_init()

cookie <- "<tt_csrf_token=wIkwTagDouAU-GjV82ecHtKP; s_v_web_id=verify_khmml1i9_eqEbiFWj_vmq2_4bdV_9C7b_FaaNBeWPiWze; passport_csrf_token=2bfb41cb61285346b641c1e2a5a44994; MONITOR_WEB_ID=994851e9-0dd1-47b5-b21b-b861d10b6753>"

trends <- tk_posts(scope = "trends", n = 200, cookie = cookie)

use_python("/Users/adamkanwal/miniconda3/bin/python3")

tk_init()

user_posts <- tk_posts(scope = "user", query = "willsmith", n = 50, cookie = cookie)
adamkthecoder commented 4 years ago

Never mind! I resolved the issue. However I am also wondering what one should use as a "music id" for songs?

benjaminguinaudeau commented 4 years ago

You can get music_ids from the results of: tk_posts()

For instance:

user_post <- tk_posts(scope = "user", query = "willsmith", n = 40)
user_post$music_id %>% purrr::map(~{tk_posts(scope = "music", query = .x, n = 25)})

Sorry for the issue, we've been maintaining the package and pushing the update live. In the course of maintaining, we slightly changed the way cookies were handled. We now solved this using the function tk_auth() and environment variables, so that no cookie argument needs to be passed.

With the last version of tiktokr, you can run:

cookie <- "<tt_csrf_token=wIkwTagDouAU-GjV82ecHtKP; s_v_web_id=verify_khmml1i9_eqEbiFWj_vmq2_4bdV_9C7b_FaaNBeWPiWze; passport_csrf_token=2bfb41cb61285346b641c1e2a5a44994; MONITOR_WEB_ID=994851e9-0dd1-47b5-b21b-b861d10b6753>"

tk_auth(cookie = cookie)

user_posts <- tk_posts(scope = "user", query = "willsmith", n = 50)