调用/api获取抖音视频节点出现错误

[jqsl2012]

发生错误的平台？

如：抖音/TikTok 抖音

发生错误的端点？

如：API-V1/API-V2/Web APP v1

提交的输入值？

如：短视频链接 短视频

是否有再次尝试？ 是

如：是，发生错误后X时间后错误依旧存在。

你有查看本项目的自述文件或接口文档吗？

如：有，并且很确定该问题是程序导致的。 有

正在进行混合解析... 正在解析douyin视频链接... 正在通过抖音分享链接获取原始链接... 获取原始链接成功, 原始链接为: https://www.iesdouyin.com/share/video/6918273131559881997/ 获取到的douyin视频ID是6918273131559881997

ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/ubuntu/python_api/notebooks/dy/Douyin_TikTok_Download_API-main/scraper.py", line 280, in get_douyin_video_data api_url = self.generate_x_bogus_url(api_url) File "/home/ubuntu/python_api/notebooks/dy/Douyin_TikTok_Download_API-main/scraper.py", line 226, in generate_x_bogus_url xbogus = execjs.compile(open(self.relpath('./X-Bogus.js')).read()).call('sign', query, File "/home/ubuntu/.local/lib/python3.8/site-packages/execjs/_abstract_runtime_context.py", line 37, in call return self._call(name, *args) File "/home/ubuntu/.local/lib/python3.8/site-packages/execjs/_external_runtime.py", line 92, in _call return self._eval("{identifier}.apply(this, {args})".format(identifier=identifier, args=args)) File "/home/ubuntu/.local/lib/python3.8/site-packages/execjs/_external_runtime.py", line 78, in eval return self.exec(code) File "/home/ubuntu/.local/lib/python3.8/site-packages/execjs/_abstract_runtimecontext.py", line 18, in exec return self.exec(source) File "/home/ubuntu/.local/lib/python3.8/site-packages/execjs/_external_runtime.py", line 85, in exec output = self._exec_with_tempfile(source) File "/home/ubuntu/.local/lib/python3.8/site-packages/execjs/_external_runtime.py", line 127, in _exec_with_tempfile self._fail_on_non_zero_status(ret, stdoutdata, stderrdata) File "/home/ubuntu/.local/lib/python3.8/site-packages/execjs/_external_runtime.py", line 134, in _fail_on_non_zero_status raise ProcessExitedWithNonZeroStatus(status=status, stdout=stdoutdata, stderr=stderrdata) execjs._exceptions.ProcessExitedWithNonZeroStatus: (101, '', '/tmp/execjs599yclqr.js:483:16 Expected ; but found a\n let a = !1;\n ^\n/tmp/execjs599yclqr.js:524:8 Expected an operand but found )\n }, () => 0, () => "03v", {\n ^\n/tmp/execjs599yclqr.js:526:5 Expected an operand but found ,\n }, {\n ^\n/tmp/execjs599yclqr.js:528:22 Expected ; but found :\n msNewTokenList: [],\n ^\n/tmp/execjs599yclqr.js:530:17 Expected ; but found :\n clickList: [],\n ^\n/tmp/execjs599yclqr.js:532:19 Expected ; but found :\n activeState: [],\n ^\n/tmp/execjs599yclqr.js:534:15 Expected ; but found :\n envcode: 0,\n ^\n/tmp/execjs599yclqr.js:536:16 Expected ; but found :\n msStatus: 0,\n ^\n/tmp/execjs599yclqr.js:538:13 Expected ; but found :\n ttwid: "",\n ^\n/tmp/execjs599yclqr.js:540:19 Expected ; but found :\n tt_webid_v2: ""\n ^\n/tmp/execjs599yclqr.js:544:12 Expected : but found a\n let a = new Uint8Array(3);\n ^\n/tmp/execjs599yclqr.js:545:8 Invalid return statement\n return a[0] = e / 256, a[1] = e % 256, a[2] = b % 256, String.fromCharCode.apply(null, a)\n ^\n/tmp/execjs599yclqr.js:546:4 Expected eof but found }\n }, (e, b) => {\n ^\n')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 50, in call result = await fn(*args, **kwargs) File "/home/ubuntu/python_api/notebooks/dy/Douyin_TikTok_Download_API-main/scraper.py", line 293, in get_douyin_video_data raise ValueError(f"获取抖音视频数据出错了: {e}") ValueError: 获取抖音视频数据出错了: (101, '', '/tmp/execjs599yclqr.js:483:16 Expected ; but found a\n let a = !1;\n ^\n/tmp/execjs599yclqr.js:524:8 Expected an operand but found )\n }, () => 0, () => "03v", {\n ^\n/tmp/execjs599yclqr.js:526:5 Expected an operand but found ,\n }, {\n ^\n/tmp/execjs599yclqr.js:528:22 Expected ; but found :\n msNewTokenList: [],\n ^\n/tmp/execjs599yclqr.js:530:17 Expected ; but found :\n clickList: [],\n ^\n/tmp/execjs599yclqr.js:532:19 Expected ; but found :\n activeState: [],\n ^\n/tmp/execjs599yclqr.js:534:15 Expected ; but found :\n envcode: 0,\n ^\n/tmp/execjs599yclqr.js:536:16 Expected ; but found :\n msStatus: 0,\n ^\n/tmp/execjs599yclqr.js:538:13 Expected ; but found :\n ttwid: "",\n ^\n/tmp/execjs599yclqr.js:540:19 Expected ; but found :\n tt_webid_v2: ""\n ^\n/tmp/execjs599yclqr.js:544:12 Expected : but found a\n let a = new Uint8Array(3);\n ^\n/tmp/execjs599yclqr.js:545:8 Invalid return statement\n return a[0] = e / 256, a[1] = e % 256, a[2] = b % 256, String.fromCharCode.apply(null, a)\n ^\n/tmp/execjs599yclqr.js:546:4 Expected eof but found }\n }, (e, b) => {\n ^\n')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/ubuntu/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call return await self.app(scope, receive, send) File "/home/ubuntu/.local/lib/python3.8/site-packages/fastapi/applications.py", line 271, in call await super().call(scope, receive, send) File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/applications.py", line 118, in call await self.middleware_stack(scope, receive, send) File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in call raise exc File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in call await self.app(scope, receive, _send) File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in call raise exc File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in call await self.app(scope, receive, sender) File "/home/ubuntu/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call raise e File "/home/ubuntu/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call await self.app(scope, receive, send) File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/routing.py", line 706, in call await route.handle(scope, receive, send) File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/home/ubuntu/.local/lib/python3.8/site-packages/starlette/routing.py", line 66, in app response = await func(request) File "/home/ubuntu/.local/lib/python3.8/site-packages/fastapi/routing.py", line 237, in app raw_response = await run_endpoint_function( File "/home/ubuntu/.local/lib/python3.8/site-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(values) File "/home/ubuntu/.local/lib/python3.8/site-packages/slowapi/extension.py", line 729, in async_wrapper response = await func(*args, *kwargs) # type: ignore File "/home/ubuntu/python_api/notebooks/dy/Douyin_TikTok_Download_API-main/web_api.py", line 267, in hybrid_parsing data = await api.hybrid_parsing(url) File "/home/ubuntu/python_api/notebooks/dy/Douyin_TikTok_Download_API-main/scraper.py", line 467, in hybrid_parsing data = await self.get_douyin_video_data(video_id) if url_platform == 'douyin' \ File "/home/ubuntu/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped return await fn(args, kwargs) File "/home/ubuntu/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 47, in call do = self.iter(retry_state=retry_state) File "/home/ubuntu/.local/lib/python3.8/site-packages/tenacity/init.py", line 326, in iter raise retry_exc from fut.exception() tenacity.RetryError: RetryError[<Future at 0x7f9160ce4160 state=finished raised ValueError>]

[Evil0ctal]

更换IP后再次尝试看看。

[samly1220]

更换IP一样失败

[jqsl2012]

@Evil0ctal 为什么换了ip，换了7-8个代理，有美国，日本，英国的，都报错，全都是：ValueError: 获取抖音视频数据出错了: 0, message='Attempt to decode JSON with unexpected mimetype:

[befantasy]

哎，本来docker部署好好的，昨天升了一下级，报错了。现在镜像删了重新用老版本，同样报错。。。

[befantasy]

退回到v3.1.6，不行。去docker hub上翻了翻， 按照tag的release时间，感觉之前正常使用的版本是0b42677，还了回去，好了。。。

但是同样0b42677，在我另外一台ARM的服务器上却又不行。

以下是报错：

douyin_tiktok_download_api | INFO: Will watch for changes in these directories: ['/app'] douyin_tiktok_download_api | INFO: Uvicorn running on http://0.0.0.0:7000 (Press CTRL+C to quit) douyin_tiktok_download_api | INFO: Started reloader process [7] using StatReload douyin_tiktok_download_api | INFO: Started server process [10] douyin_tiktok_download_api | INFO: Waiting for application startup. douyin_tiktok_download_api | INFO: Application startup complete. douyin_tiktok_download_api | exception calling callback for <Future at 0xfffbdd505120 state=finished raised RetryError> douyin_tiktok_download_api | Traceback (most recent call last): douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/tenacity/_asyncio.py", line 50, in call douyin_tiktok_download_api | result = await fn(*args, kwargs) douyin_tiktok_download_api | File "/app/scraper.py", line 277, in get_douyin_video_data douyin_tiktok_download_api | raise e douyin_tiktok_download_api | File "/app/scraper.py", line 262, in get_douyin_video_data douyin_tiktok_download_api | api_url = self.generate_x_bogus_url(api_url) douyin_tiktok_download_api | File "/app/scraper.py", line 208, in generate_x_bogus_url douyin_tiktok_download_api | xbogus = execjs.compile(open(self.relpath('./X-Bogus.js')).read()).call('sign', query, self.headers['User-Agent']) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/execjs/_abstract_runtime_context.py", line 37, in call douyin_tiktok_download_api | return self._call(name, args) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/execjs/_external_runtime.py", line 92, in _call douyin_tiktok_download_api | return self._eval("{identifier}.apply(this, {args})".format(identifier=identifier, args=args)) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/execjs/_external_runtime.py", line 78, in _eval douyin_tiktok_downloadapi | return self.exec(code) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/execjs/_abstract_runtimecontext.py", line 18, in exec douyin_tiktok_download_api | return self.exec(source) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/execjs/_external_runtime.py", line 87, in exec douyin_tiktok_download_api | output = self._exec_with_pipe(source) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/execjs/_external_runtime.py", line 108, in _exec_with_pipe douyin_tiktok_download_api | self._fail_on_non_zero_status(ret, stdoutdata, stderrdata) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/execjs/_external_runtime.py", line 134, in _fail_on_non_zero_status douyin_tiktok_download_api | raise ProcessExitedWithNonZeroStatus(status=status, stdout=stdoutdata, stderr=stderrdata) douyin_tiktok_download_api | execjs._exceptions.ProcessExitedWithNonZeroStatus: (-5, '\n<--- Last few GCs --->\n\n\n<--- JS stacktrace --->\n\n', '\n#\n# Fatal process OOM in insufficient memory to create an Isolate\n#\n\n') douyin_tiktok_download_api | douyin_tiktok_download_api | The above exception was the direct cause of the following exception: douyin_tiktok_download_api | douyin_tiktok_download_api | Traceback (most recent call last): douyin_tiktok_download_api | File "/usr/lib/python3.10/concurrent/futures/_base.py", line 342, in _invoke_callbacks douyin_tiktok_download_api | callback(self) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/pywebio/session/coroutinebased.py", line 347, in _wakeup douyin_tiktok_download_api | self.step(future.result()) douyin_tiktok_download_api | File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result douyin_tiktok_download_api | return self.__get_result() douyin_tiktok_download_api | File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result douyin_tiktok_download_api | raise self._exception douyin_tiktok_download_api | File "/app/scraper.py", line 441, in hybrid_parsing douyin_tiktok_download_api | data = await self.get_douyin_video_data( douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped douyin_tiktok_download_api | return await fn(args, kwargs) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/tenacity/_asyncio.py", line 47, in call douyin_tiktok_download_api | do = self.iter(retry_state=retry_state) douyin_tiktok_download_api | File "/root/.local/lib/python3.10/site-packages/tenacity/init.py", line 326, in iter douyin_tiktok_download_api | raise retry_exc from fut.exception() douyin_tiktok_download_api | tenacity.RetryError: RetryError[<Future at 0xfffbdd505b70 state=finished raised ProcessExitedWithNonZeroStatus>]

[Evil0ctal]

检查是否安装nodejs运行环境。

[befantasy]

检查是否安装nodejs运行环境。

我是docker部署的，镜像里环境不完整吗？

[Evil0ctal]

可能是的，你检查一下docker配置文件里有没有nodejs，我现在没有电脑。

[befantasy]

进到容器中执行node --version是没问题的，版本号12.22.9。

报错给GPT分析了一下：

根据报错信息可以看出,这个程序在调用execjs模块执行JavaScript代码时发生了错误。

关键的报错信息有:

1. douyin_tiktok_download_api调用execjs.compile执行X-Bogus.js脚本时报错

2. 最终抛出了execjs._exceptions.ProcessExitedWithNonZeroStatus异常

3. 异常信息中有"Fatal process OOM in insufficient memory to create an Isolate"

4. 调用栈显示是在获取抖音视频数据的时候出错的

所以很有可能是:

1. 程序使用execjs模块调用Node.js环境来执行JavaScript代码

2. X-Bogus.js这个脚本可能比较占内存,触发了JavaScript运行时的内存不足错误

3. Node.js无法为执行该脚本创建新的Isolate实例,导致脚本执行失败

4. 最后抛出了一个非0退出码的异常,被Python端的execjs模块捕获并封装成了ProcessExitedWithNonZeroStatus异常

所以解决方法是:

1. 检查X-Bogus.js脚本,是否存在内存泄露或占用过多内存的问题

2. 调整Node.js环境的内存限制,允许分配更多内存给JavaScript执行

3. 尝试升级Node.js版本,以确保不是版本问题导致的内存不足

4. 如果条件允许,可以增加系统内存,或将程序部署到内存更充足的环境

[befantasy]

但是我这台机器的内存12G，另外一台1G的VPS上缺没问题。 有一个特殊的是这台有问题的CPU是ARM的。

[jqsl2012]

@Evil0ctal 兄弟，我为你带来了一单生意，但是 https://tikhub.io/ 怎么付费？发了邮件也没人回应？

[Evil0ctal]

加我微信吧 evil0ctal

[jqsl2012]

已加

[jqsl2012]

@Evil0ctal 兄弟麻烦看下微信