search

isaackogan / TikTokLive

Python library to receive live stream events (comments, gifts, etc.) in realtime from TikTok LIVE.
https://isaackogan.github.io/TikTokLive/
MIT License
876 stars 175 forks source link

Consistent broken payloads with `WebcastLinkMicBattle` #232

Closed shaiyon closed 2 weeks ago

shaiyon commented 3 months ago

Describe the bug

Consistent broken payloads with WebcastLinkMicBattle events. Sometimes it goes through normally, but around half the time this broken payload decoding varint error comes up. Could this be a new schema that hasn't been accounted for yet, maybe the battle ending? Don't mind tackling this myself if you can point me in the right direction.

Expected Behaviour

Tracking a battle beginning or ending.

Actual Behaviour

[TikTokLive] ERROR from U.s.D.H.T.T.c.client.py:406 — Traceback (most recent call last):
  File "/Users/shaiyon/Documents/project_name/TikTokLive/TikTokLive/client/client.py", line 404, in _parse_webcast_response
    proto_event: ProtoEvent = event_type().parse(response.payload)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 1015, in parse
    value = self._postprocess_single(
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 955, in _postprocess_single
    value = cls().parse(value)
            ^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 1015, in parse
    value = self._postprocess_single(
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 955, in _postprocess_single
    value = cls().parse(value)
            ^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 1008, in parse
    decoded, pos = decode_varint(parsed.value, pos)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 487, in decode_varint
    raise ValueError("Too many bytes when decoding varint.")
ValueError: Too many bytes when decoding varint.

Broken Payload:
b'\n6\n\x14WebcastLinkMicBattle\x10\x91\x96\x8a\xe8\x8a\xd1\xfa\xbaf\x18\x9f\x96\x88\xc4\xd4\xeb\xf7\xbaf \xf9\x81\xf1\xe4\x832(\x01\xd0\x01\x01\x10\x90\x96\xa9\x96\xce\xaa\xfa\xbaf\x1a"\x08\x90\x96\xa9\x96\xce\xaa\xfa\xbaf\x10\xc9\xb5\xde\xe4\x832\x18\xad\x02 \x90\x96\x86\xdc\xad\xa3\xfa\xbaf(\x03@\x01 \x05*\x1a\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\x0e\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x10\x01\x18g*\x19\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\r\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x18\xd4\tJ\xd2\x0f\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\xc5\x0f\n\x8f\x05\x08\x86\x88\xbf\xbe\xe6\x8a\xe9\x9aa\x10"\x1a\x0bWeedman 902"\xde\x04\n\x8c\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/612ef28ac1850f3be59cc872f5ddc952~c5_100x100.webp?lk3s=a5d48078&nonce=84167&refresh_token=374e5f3f0b9a24d454b8401ff9b53ea3&x-expires=1719176400&x-signature=0Qxc%2FC5VdRsIHgWy1SnS5clhUIA%3D&shp=a5d48078&shcp=d2876c74\n\x8e\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/612ef28ac1850f3be59cc872f5ddc952~c5_100x100.jpeg?lk3s=a5d48078&nonce=35952&refresh_token=ba58d331c490adf86203d6c51cbdb0e2&x-expires=1719176400&x-signature=wRPa4W%2FxNevnV1Md%2BraqaFttUYU%3D&shp=a5d48078&shcp=d2876c74\x12<100x100/tos-maliva-avt-0068/612ef28ac1850f3be59cc872f5ddc9522\x137004685486460486662\n\x95\x05\x08\x85\x88\xd8\x84\xc0\xb0\xe3\xfa^\x10\x13\x1a\x11McDaddy Rico \xf0\x9f\x98\x9c"\xde\x04\n\x90\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/d7eda1280decc4753f4379f4cf9c8a28~c5_100x100.webp?lk3s=a5d48078&nonce=45835&refresh_token=78f8bfddbcaafea70c9f2096015e1c0f&x-expires=1719176400&x-signature=XnQi%2BCjk%2Bau8pn36VirHu9h3%2Bm4%3D&shp=a5d48078&shcp=d2876c74\n\x8a\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/d7eda1280decc4753f4379f4cf9c8a28~c5_100x100.jpeg?lk3s=a5d48078&nonce=85970&refresh_token=167fa817f6a7ab8add1492ee3dfef286&x-expires=1719176400&x-signature=WCdnp0TD8Bk2XQP7Dgi67Oe9Q6Q%3D&shp=a5d48078&shcp=d2876c74\x12<100x100/tos-maliva-avt-0068/d7eda1280decc4753f4379f4cf9c8a282\x136842530806944367621\n\x82\x05\x08\x86\x88\xd9\xa0\xe9\xca\xfe\xc4_\x10\x04\x1a\x04Matt"\xd8\x04\n\x8a\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/c8bfdd9fb5dd79db486a6002275271d8~c5_100x100.webp?lk3s=a5d48078&nonce=91314&refresh_token=025cfb9b99037eb70e2a4412a1f27639&x-expires=1719176400&x-signature=FD1r9vjXJ7LED7EFm9mZqOtqJTE%3D&shp=a5d48078&shcp=d2876c74\n\x8a\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/c8bfdd9fb5dd79db486a6002275271d8~c5_100x100.jpeg?lk3s=a5d48078&nonce=58945&refresh_token=5aac7f50d07c95c5fb988536acd0c3ae&x-expires=1719176400&x-signature=O0o5KcDxBSWSLTkKs4tlY4QtYpU%3D&shp=a5d48078&shcp=d2876c74\x12<100x100/tos-maliva-avt-0068/c8bfdd9fb5dd79db486a6002275271d82\x136884308755171132422\x10g\x1a\x12191139995515252736J\xe0\x10\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\xd3\x10\n\xbb\x05\x08\x9a\x88\xb0\xac\x83\xaf\xcf\xc1d\x10\x84\x01\x1a\x17\xe2\x9c\x8c\xef\xb8\x8f only pind \xe2\x9c\x8c\xef\xb8\x8f"\xfd\x04\n\x9a\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/dfdb58b99833dc3bf6f28876c31e20a7~c5_100x100.webp?lk3s=a5d48078&nonce=49965&refresh_token=3c437998130aceeda15c044634a4510d&x-expires=1719176400&x-signature=tkRA%2BTMJmTGRHJnosiXSlt382p4%3D&shp=a5d48078&shcp=d2876c74\n\x97\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/dfdb58b99833dc3bf6f28876c31e20a7~c5_100x100.jpeg?lk3s=a5d48078&nonce=3339&refresh_token=70c01f3aff4ba2562ba535e411a178fe&x-expires=1719176400&x-signature=gdZQkrFPsNqu2YyrGGVmRDyw1nI%3D&shp=a5d48078&shcp=d2876c74\x12D100x100/tos-useast2a-avt-0068-euttp/dfdb58b99833dc3bf6f28876c31e20a72\x137242700212245627930\n\xca\x05\x08\x9a\x88\x81\xf6\xc5\xf8\xe2\xdad\x10+\x1a \xd8\xb4\xdb\x81\xd8\xb1\xdb\x8c\xd8\xa7\xd8\xb1\xf0\x9f\x96\xa4\xf0\x9f\x87\xb5\xf0\x9f\x87\xb0\xf0\x9f\x87\xae\xf0\x9f\x87\xb9"\x84\x05\n\x9a\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/ec3a871ab0731dec13e6f49dcef0952a~c5_100x100.webp?lk3s=a5d48078&nonce=45406&refresh_token=98efd4b9a64574d4474925d709fc206e&x-expires=1719176400&x-signature=zJjOUAMlJdA%2FBhU6mk7ubMGQh2w%3D&shp=a5d48078&shcp=d2876c74\n\x9e\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/ec3a871ab0731dec13e6f49dcef0952a~c5_100x100.jpeg?lk3s=a5d48078&nonce=43494&refresh_token=674b71e9c15cf1fa21822b19067767a7&x-expires=1719176400&x-signature=3iMC3k8NSzFVa%2BWtLnF%2FJr%2FVqEE%3D&shp=a5d48078&shcp=d2876c74\x12D100x100/tos-useast2a-avt-0068-euttp/ec3a871ab0731dec13e6f49dcef0952a2\x137256860050096931866\n\xad\x05\x08\x82\x88\xb8\xde\x9e\xf6\x9b\xf3d\x10\x15\x1a+\xf0\x9d\x95\xaf\xf0\x9d\x96\x8a\xf0\x9d\x96\x8b\xf0\x9d\x96\x86\xf0\x9d\x96\x9a\xf0\x9d\x96\x91\xf0\x9d\x96\x99\xf0\x9d\x96\x8a\xf0\x9d\x96\x97 \xe2\x9a\x94\xef\xb8\x8f"\xdc\x04\n\x8e\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/880a4e347cba416093dca474eb3cbb90.webp?lk3s=a5d48078&nonce=97307&refresh_token=06350644f79b0267366fa174dd92212f&x-expires=1719176400&x-signature=sKsq9pdRkFUEoHm96H%2BatUfLXIo%3D&shp=a5d48078&shcp=d2876c74\n\x8b\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/880a4e347cba416093dca474eb3cbb90.jpeg?lk3s=a5d48078&nonce=4507&refresh_token=d2b3d9b541d6d8ed21f9b9809832970a&x-expires=1719176400&x-signature=feSYTqEpzF6ZKPvjLeqp5tfy8ZI%3D&shp=a5d48078&shcp=d2876c74\x12;100x100/tos-alisg-avt-0068/880a4e347cba416093dca474eb3cbb902\x137270621458392286210\x10\xd4\t\x1a\x137146125739820598273R\x8d\x05\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\x80\x05\n\xfd\x04\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\x08Ali Jutt\x1a\xd8\x04\n\x8d\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/26c7da7443e82dc7520571415689543a.webp?lk3s=a5d48078&nonce=5851&refresh_token=ed9c80d14d540205f33c6765fc2a0e25&x-expires=1719176400&x-signature=ZPBwd6o9qEccH5K8ax7A%2BDMxgpI%3D&shp=a5d48078&shcp=d2876c74\n\x90\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/26c7da7443e82dc7520571415689543a.jpeg?lk3s=a5d48078&nonce=36074&refresh_token=0a27140104c3f0844eedf0d37f679b39&x-expires=1719176400&x-signature=m6RFfrjBL%2F8DlG%2Fz71b0Xr2FBL4%3D&shp=a5d48078&shcp=d2876c74\x123tos-alisg-avt-0068/26c7da7443e82dc7520571415689543a"\x0cali_jutt1982R\x89\x05\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\xfc\x04\n\xf9\x04\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\x06Tori V\x1a\xd5\x04\n\x8d\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/285745437d005f2527db37af8f0b2579~c5_100x100.webp?lk3s=a5d48078&nonce=2537&refresh_token=e139ec435edd7cbc07cc99838f08736e&x-expires=1719176400&x-signature=B5VNzKSI3WzL3Xi07%2F38%2F3XgYPk%3D&shp=a5d48078&shcp=d2876c74\n\x8c\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/285745437d005f2527db37af8f0b2579~c5_100x100.jpeg?lk3s=a5d48078&nonce=37294&refresh_token=4ba4c2710ae7f9698b6476b10c1823c7&x-expires=1719176400&x-signature=rIqT4trBOabq4S84dcPBywmHi%2BI%3D&shp=a5d48078&shcp=d2876c74\x124tos-maliva-avt-0068/285745437d005f2527db37af8f0b2579"\rtorivofficialj\x9f\x03\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\x92\x03\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x10\x01\x18\x01(\x012\xff\x02sslocal://webcast_lynxview_popup?url=https%3A%2F%2Flf16-gecko-source.tiktokcdn.com%2Fobj%2Fbyte-gurd-source-sg%2Ftiktok%2Ffe%2Flive%2Ftiktok_live_revenue_match%2Fpages%2Flive_match_streak%2Ftemplate.js&height=832rpx&radius=16rpx&container_bg_color=FFFFFF&bdhm_bid=tiktok_live_revenue_match&bdhm_pid=tiktok_live_revenue_match_pages_live_match_streak&server_room_id=7383053092951968513j\x9d\x03\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\x90\x03\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x10\x02(\x012\xff\x02sslocal://webcast_lynxview_popup?url=https%3A%2F%2Flf16-gecko-source.tiktokcdn.com%2Fobj%2Fbyte-gurd-source-sg%2Ftiktok%2Ffe%2Flive%2Ftiktok_live_revenue_match%2Fpages%2Flive_match_streak%2Ftemplate.js&height=832rpx&radius=16rpx&container_bg_color=FFFFFF&bdhm_bid=tiktok_live_revenue_match&bdhm_pid=tiktok_live_revenue_match_pages_live_match_streak&server_room_id=7383052755885230879'

Package Version

Latest code from main.

Operating System

MacOS

Anything Else

Thank you!

isaackogan commented 3 months ago

Hi Shaiyon,

Yes it's likely precisely that. @jwdeveloper @kholerpop1 do you have any updated proto files? I'd like to sift through and update the proto.

Shaiyon, if you would like to tackle this yourself you should likely fork the library and intercept the payload before it becomes a dictionary. Encode this in base64 and use https://protobuf-decoder.netlify.app/ to read the raw structure. You can look for the difference in the proto structure and update it manually if you would like. I'll be happy to merge that until we get a 'real' proto update.

kohlerpop1 commented 3 months ago

@isaackogan https://github.com/jwdeveloper/TikTokLiveJava/tree/master/API/src/main/proto has our most up to date proto. I have made some changes recently regarding WebcastLinkMicBattle however I do not claim it to be perfect.

P.S. You spelled my name wrong! Lol

isaackogan commented 3 months ago

@isaackogan https://github.com/jwdeveloper/TikTokLiveJava/tree/master/API/src/main/proto has our most up to date proto. I have made some changes recently regarding WebcastLinkMicBattle however I do not claim it to be perfect.

P.S. You spelled my name wrong! Lol

No one is perfect. For example, you got your name wrong. I got it right. I name you kholer as of today.

shaiyon commented 3 months ago

@isaackogan seems like the .proto files are a bit outdated compared to the Java version, so I updated those to match. Having some trouble identifying which scripts to run to build the docs/python/ts files, where should I start?

isaackogan commented 3 months ago

@shaiyon dont worry about the docs, make a PR and I'll build them. It's a shitshow from 2y ago...