PaddlePaddle / Paddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (『飞桨』核心框架,深度学习&机器学习高性能单机、分布式训练和跨平台部署)
http://www.paddlepaddle.org/
Apache License 2.0
22.18k stars 5.57k forks source link

Support for `gast` with Python 3.12 #68889

Open eee4017 opened 3 hours ago

eee4017 commented 3 hours ago

bug描述 Describe the Bug

PaddlePaddle utilizes gast for handling Python Abstract Syntax Trees (AST). With the release of Python 3.12, changes to Python’s grammar have introduced new syntax, such as type_param, which is now part of Python's AST. The gast has updated with this grammar. However, since PaddlePaddle does not use gast as a submodule, it has not kept pace with the latest updates in gast.

The test test_gast_with_compatibility fails when executed in a Python 3.12 environment.

Error Log:

1382: ======================================================================
1382: FAIL: test_param_of_func (test_gast_with_compatibility.TestPythonCompatibility.test_param_of_func)
1382: Param in func is ast.Name in PY2, but ast.arg in PY3.
1382: ----------------------------------------------------------------------
1382: Traceback (most recent call last):
1382:   File "/home/fralin/workspace/paddle/paddle-develop/build/test/legacy_test/test_gast_with_compatibility.py", line 143, in test_param_of_func
1382:     self._check_compatibility(source, target)
1382:   File "/home/fralin/workspace/paddle/paddle-develop/build/test/legacy_test/test_gast_with_compatibility.py", line 128, in _check_compatibility
1382:     self.assertEqual(source_dump, target_dump)
1382: AssertionError: "Modu[244 chars]d())))], decorator_list=[])], type_ignores=[])" != "Modu[244 chars]d())))], decorator_list=[], type_params=[])], type_ignores=[])"
1382: Diff is 1187 characters long. Set self.maxDiff to None to see it.
1382:
1382: ----------------------------------------------------------------------
1382: Ran 5 tests in 0.003s

The error arises because the type_params syntax introduced in Python 3.12 is not recognized by the existing gast version in PaddlePaddle, causing mismatches in AST comparisons during testing.

Although updating gast to a newer version could potentially resolve the issue, there are additional code adjustments required within PaddlePaddle’s JIT modules. For example, the following locations need to be updated to handle grammar changes in Python 3.12: utils.py origin_info.py.

其他补充信息 Additional Supplementary Information

No response

SigureMo commented 3 hours ago

Thanks for the report, we'll be upgrading the gast source code soon.

jeng1220 commented 42 minutes ago

@SigureMo , 能麻煩將 PR#68892 cherry-pick 到分支 release/3.0-beta2 嗎?

SigureMo commented 21 minutes ago

PR 验证还需要周期,问题还没完全解决,cherry-pick 可能来不及

现在有哪些场景可能会受到影响呢?Python 3.12 是完全不可用么?还是只是单测场景有问题?

SigureMo commented 12 minutes ago

@jeng1220

刚刚问了下,根据发版计划,v3.0.0-beta2 tag 应该是赶不上了,但这个 PR 完成后,并且打完 tag 后可以 cherry-pick 到 release/3.0-beta2 下

也就是说发布的 wheel 包里没有这个 PR,但是 release/3.0-beta2 里会有,请问这样可以解决问题么?