How to handle the __signature field in tiktok share link

elvisyjlin commented 5 years ago

Hi, the _signature field seems to be a new verification mechanism. I'm working on it.

toyinstark commented 5 years ago

Hi, the _signature field seems to be a new verification mechanism. I'm working on it.

Any luck with how the _signature is generated?

Also, How long does it take for a _signature to expire

elvisyjlin commented 5 years ago

If you read Chinese, this article (https://blog.csdn.net/swukong_/article/details/80887940) illustrates how to dig into the douyin signature, which is similar to the tiktok signature. The idea is to find out a javascript code which performs sign(). But I observed that the signature mechanism behaves different from the way the article says. I.e., the javascript seems different.

toyinstark commented 5 years ago

If you read Chinese, this article (https://blog.csdn.net/swukong_/article/details/80887940) illustrates how to dig into the douyin signature, which is similar to the tiktok signature. The idea is to find out a javascript code which performs sign(). But I observed that the signature mechanism behaves different from the way the article says. I.e., the javascript seems different.

I think this solves the problem? https://github.com/loadchange/amemv-crawler/blob/master/fuck-byted-acrawler.js

elvisyjlin commented 5 years ago

It seems to utilize the same method as the article says. I will be grateful If you can try it and see if it is feasible. I haven't got time to survey it.

carcabot commented 5 years ago

If you follow this you will get the correct signature.

def generate_signature(value):
    root_dir = os.path.dirname(os.path.abspath(__file__))

    response = muterun_js(root_dir + '/scripts/byted-acrawler.js', value)
    if response.exitcode == 0:
        # the command was successful, handle the standard output
        result = response.stdout.rstrip()
    else:
        # the command failed or the executable was not present, handle the standard error
        standard_err = response.stderr
        exit_code = response.exitcode
        print('Exit Status ' + exit_code + ': ' + standard_err)
        result = None
    return result

using this byted crawler

dacopan commented 5 years ago

If you follow this you will get the correct signature.

def generate_signature(value):
    root_dir = os.path.dirname(os.path.abspath(__file__))

    response = muterun_js(root_dir + '/scripts/byted-acrawler.js', value)
    if response.exitcode == 0:
        # the command was successful, handle the standard output
        result = response.stdout.rstrip()
    else:
        # the command failed or the executable was not present, handle the standard error
        standard_err = response.stderr
        exit_code = response.exitcode
        print('Exit Status ' + exit_code + ': ' + standard_err)
        result = None
    return result

using this byted crawler

this not work for me. any updates?

carl-jin commented 5 years ago

If you follow this you will get the correct signature.

def generate_signature(value):
    root_dir = os.path.dirname(os.path.abspath(__file__))

    response = muterun_js(root_dir + '/scripts/byted-acrawler.js', value)
    if response.exitcode == 0:
        # the command was successful, handle the standard output
        result = response.stdout.rstrip()
    else:
        # the command failed or the executable was not present, handle the standard error
        standard_err = response.stderr
        exit_code = response.exitcode
        print('Exit Status ' + exit_code + ': ' + standard_err)
        result = None
    return result

using this byted crawler

this not work for me. any updates?

the only i found is use google puppeteer, did you figure it out other way?

dacopan commented 5 years ago

If you follow this you will get the correct signature.

def generate_signature(value):
    root_dir = os.path.dirname(os.path.abspath(__file__))

    response = muterun_js(root_dir + '/scripts/byted-acrawler.js', value)
    if response.exitcode == 0:
        # the command was successful, handle the standard output
        result = response.stdout.rstrip()
    else:
        # the command failed or the executable was not present, handle the standard error
        standard_err = response.stderr
        exit_code = response.exitcode
        print('Exit Status ' + exit_code + ': ' + standard_err)
        result = None
    return result

using this byted crawler

this not work for me. any updates?

the only i found is use google puppeteer, did you figure it out other way?

Yes, use this awesome project by scrapping hub team https://github.com/scrapy-plugins/scrapy-splash

rubik commented 5 years ago

@elvisyjlin Did you solve this? I'm using the byted crawler tool, but I always get the same response:

{
    "content": "",
    "contentType": "text/html",
    "statusCode": 200
}

I am passing the same headers that I see in my browser, but still it does not work.

carcabot commented 5 years ago

The signature was changed again.

Take a look to my solution tiktok-signature