如果bug是由PaddleX API 2.0导致,且该bug在develop分支里已修复,参考FAQ Q4替换内置PaddleX API
描述问题
win10系统,在图像分类训练时,报错
复现
错误日志
This log file path is /home/rao/Code/SunPeiDiQinMao/paddleX/projects/P0001/T0001/err.log
注意:标志为WARNING/INFO类的仅为警告或提示类信息,非错误信息
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/nn/functional/loss.py:1281: DeprecationWarning: invalid escape sequence \s
"""
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntuamd64/paddle/nn/functional/pooling.py:740: DeprecationWarning: invalid escape sequence \
"""
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntuamd64/paddle/nn/layer/pooling.py:1187: DeprecationWarning: invalid escape sequence \
"""
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/incubate/nn/functional/fusedtransformer.py:111: DeprecationWarning: invalid escape sequence \
"""
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/incubate/nn/functional/fusedtransformer.py:339: DeprecationWarning: invalid escape sequence \
"""
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/vision/ops.py:952: SyntaxWarning: assertion is always true, perhaps remove parentheses?
assert (len(x.shape) == 4,
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/signal.py:307: DeprecationWarning: invalid escape sequence \o
"""
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/signal.py:472: DeprecationWarning: invalid escape sequence \s
"""
/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/tensor/creation.py:130: DeprecationWarning: np.object is a deprecated alias for the builtin object. To silence this warning, use object by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
Process Process-1:3:
Traceback (most recent call last):
File "multiprocessing/process.py", line 297, in _bootstrap
File "multiprocessing/process.py", line 99, in run
File "pms/model_tasks/tasks.py", line 73, in _call_paddlex_train
File "pms/model_tasks/train/classification.py", line 118, in train
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/cv/models/classifier.py", line 703, in init
model_name=model_name, num_classes=num_classes, params)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/cv/models/classifier.py", line 70, in init
self.net = self.build_net(params)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/cv/models/classifier.py", line 75, in build_net
params)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/ppcls/arch/backbone/legendary_models/mobilenet_v3.py", line 438, in MobileNetV3_small_x1_0
kwargs)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/ppcls/arch/backbone/legendary_models/mobilenet_v3.py", line 164, in init
act="hardswish")
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/ppcls/arch/backbone/legendary_models/mobilenet_v3.py", line 240, in init
bias_attr=False)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/nn/layer/conv.py", line 656, in init
data_format=data_format)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/nn/layer/conv.py", line 135, in init
default_initializer=_get_default_param_initializer())
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/dygraph/layers.py", line 424, in create_parameter
default_initializer)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/layer_helper_base.py", line 378, in create_parameter
**attr._to_kwargs(with_initializer=True))
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/framework.py", line 3137, in create_parameter
initializer(param, self)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/initializer.py", line 366, in call
stop_gradient=True)
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/framework.py", line 3167, in append_op
kwargs.get("stop_gradient", False))
File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/dygraph/tracer.py", line 45, in trace_op
not stop_gradient)
OSError: (External) CUSOLVER error(7).
[Hint: 'CUSOLVER_STATUS_INTERNAL_ERROR'. An internal cuSolver operation failed. This error is usually caused by a cudaMemcpyAsync() failure.To correct: check that the hardware, an appropriate version of the driver, and the cuSolver library are correctly installed. Also, check that the memory passed as a parameter to the routine is not being deallocated prior to the routine’s completion.] (at /paddle/paddle/fluid/platform/device_context.h:418)
[operator < gaussian_random > error]
Checklist:
描述问题
win10系统,在图像分类训练时,报错
复现
错误日志
This log file path is /home/rao/Code/SunPeiDiQinMao/paddleX/projects/P0001/T0001/err.log 注意:标志为WARNING/INFO类的仅为警告或提示类信息,非错误信息 /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/nn/functional/loss.py:1281: DeprecationWarning: invalid escape sequence \s """ /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntuamd64/paddle/nn/functional/pooling.py:740: DeprecationWarning: invalid escape sequence \ """ /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntuamd64/paddle/nn/layer/pooling.py:1187: DeprecationWarning: invalid escape sequence \ """ /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/incubate/nn/functional/fusedtransformer.py:111: DeprecationWarning: invalid escape sequence \ """ /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/incubate/nn/functional/fusedtransformer.py:339: DeprecationWarning: invalid escape sequence \ """ /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/vision/ops.py:952: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert (len(x.shape) == 4, /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/signal.py:307: DeprecationWarning: invalid escape sequence \o """ /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/signal.py:472: DeprecationWarning: invalid escape sequence \s """ /home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/tensor/creation.py:130: DeprecationWarning:
np.object
is a deprecated alias for the builtinobject
. To silence this warning, useobject
by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations if data.dtype == np.object: Process Process-1:3: Traceback (most recent call last): File "multiprocessing/process.py", line 297, in _bootstrap File "multiprocessing/process.py", line 99, in run File "pms/model_tasks/tasks.py", line 73, in _call_paddlex_train File "pms/model_tasks/train/classification.py", line 118, in train File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/cv/models/classifier.py", line 703, in init model_name=model_name, num_classes=num_classes, params) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/cv/models/classifier.py", line 70, in init self.net = self.build_net(params) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/cv/models/classifier.py", line 75, in build_net params) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/ppcls/arch/backbone/legendary_models/mobilenet_v3.py", line 438, in MobileNetV3_small_x1_0 kwargs) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/ppcls/arch/backbone/legendary_models/mobilenet_v3.py", line 164, in init act="hardswish") File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddlex/ppcls/arch/backbone/legendary_models/mobilenet_v3.py", line 240, in init bias_attr=False) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/nn/layer/conv.py", line 656, in init data_format=data_format) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/nn/layer/conv.py", line 135, in init default_initializer=_get_default_param_initializer()) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/dygraph/layers.py", line 424, in create_parameter default_initializer) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/layer_helper_base.py", line 378, in create_parameter **attr._to_kwargs(with_initializer=True)) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/framework.py", line 3137, in create_parameter initializer(param, self) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/initializer.py", line 366, in call stop_gradient=True) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/framework.py", line 3167, in append_op kwargs.get("stop_gradient", False)) File "/home/rao/Downloads/PaddleX_GUI_2.1.0_ubuntu_amd64/paddle/fluid/dygraph/tracer.py", line 45, in trace_op not stop_gradient) OSError: (External) CUSOLVER error(7). [Hint: 'CUSOLVER_STATUS_INTERNAL_ERROR'. An internal cuSolver operation failed. This error is usually caused by a cudaMemcpyAsync() failure.To correct: check that the hardware, an appropriate version of the driver, and the cuSolver library are correctly installed. Also, check that the memory passed as a parameter to the routine is not being deallocated prior to the routine’s completion.] (at /paddle/paddle/fluid/platform/device_context.h:418) [operator < gaussian_random > error]