paddlehub 2.0 文本分类返回各类别概率

[dedex1994]

paddlehub 2.0 文本分类任务怎么返回各类别概率 ，我看新的predict接口已经取消了return_result参数，那应该怎么使predict接口返回概率呢

[KPatr1ck]

你好，具体使用的paddlehub和module的版本是多少？ 对应的版本可以通过以下命令查询：

pip list | grep paddlehub
hub list

[dedex1994]

2.0.4

[dedex1994]

module是用的roberta-wwm-ext经过finetune后得到的模型

[KPatr1ck]

目前predict返回的结果都是概率最大的标签，如果需要查看各类的概率，可以在此处获得。 https://github.com/PaddlePaddle/PaddleHub/blob/0c53f545f40b53a36f0ebd5543b147b8b6f04ca2/paddlehub/module/nlp_module.py#L602

[dedex1994]

这个地方我试过，在return的结果里加入prob会在预测时报显存不足的错 `Traceback (most recent call last): File "pred.py", line 18, in model_results = model.predict(data, batch_size=16) File "/usr/local/lib/python3.7/site-packages/paddlehub/compat/paddle_utils.py", line 220, in runner return func(*args, kwargs) File "/usr/local/lib/python3.7/site-packages/paddlehub/module/module.py", line 63, in _wrapper return func(*args, *kwargs) File "/root/.paddlehub/modules/useful_model/module.py", line 112, in predict model_results = self.module.predict(data, batch_size=batch_size, use_gpu=True) File "/usr/local/lib/python3.7/site-packages/paddlehub/module/nlp_module.py", line 565, in predict probs = self(input_ids, segment_ids) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(inputs, kwargs) File "/root/.paddlehub/modules/roberta_wwm_ext/module.py", line 120, in forward result = self.model(input_ids, token_type_ids, position_ids, attention_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(*inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddlenlp/transformers/roberta/modeling.py", line 293, in forward attention_mask=attention_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(*inputs, *kwargs) File "/usr/local/lib/python3.7/site-packages/paddlenlp/transformers/roberta/modeling.py", line 236, in forward encoder_outputs = self.encoder(embedding_output, attention_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/transformer.py", line 648, in forward output = mod(output, src_mask=src_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(*inputs, *kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/transformer.py", line 536, in forward src = self.self_attn(src, src, src, src_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(inputs, **kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/transformer.py", line 387, in forward out = tensor.matmul(weights, v) File "/usr/local/lib/python3.7/site-packages/paddle/tensor/linalg.py", line 148, in matmul return op(x, y, 'trans_x', transpose_x, 'trans_y', transpose_y) SystemError: (Fatal) Operator matmul_v2 raises an paddle::memory::allocation::BadAlloc exception. The exception content is :ResourceExhaustedError:

Out of memory error on GPU 0. Cannot allocate 6.000244MB memory on GPU 0, available memory is only 7.500000MB.

Please check whether there is any other process using GPU 0.

1. If yes, please stop them, or start PaddlePaddle on another GPU.

2. If no, please decrease the batch size of your model.

   (at /paddle/paddle/fluid/memory/allocation/cuda_allocator.cc:69) . (at /paddle/paddle/fluid/imperative/tracer.cc:172)`

[KPatr1ck]

这个地方我试过，在return的结果里加入prob会在预测时报显存不足的错 `Traceback (most recent call last): File "pred.py", line 18, in model_results = model.predict(data, batch_size=16) File "/usr/local/lib/python3.7/site-packages/paddlehub/compat/paddle_utils.py", line 220, in runner return func(*args, kwargs) File "/usr/local/lib/python3.7/site-packages/paddlehub/module/module.py", line 63, in _wrapper return func(*args, kwargs) File "/root/.paddlehub/modules/useful_model/module.py", line 112, in predict model_results = self.module.predict(data, batch_size=batch_size, use_gpu=True) File "/usr/local/lib/python3.7/site-packages/paddlehub/module/nlp_module.py", line 565, in predict probs = self(input_ids, segment_ids) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call* outputs = self.forward(inputs, kwargs) File "/root/.paddlehub/modules/roberta_wwm_ext/module.py", line 120, in forward result = self.model(input_ids, token_type_ids, position_ids, attention_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(*inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddlenlp/transformers/roberta/modeling.py", line 293, in forward attention_mask=attention_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(*inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddlenlp/transformers/roberta/modeling.py", line 236, in forward encoder_outputs = self.encoder(embedding_output, attention_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call* outputs = self.forward(inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/transformer.py", line 648, in forward output = mod(output, src_mask=src_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call outputs = self.forward(*inputs, kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/transformer.py", line 536, in forward src = self.self_attn(src, src, src, src_mask) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in call* outputs = self.forward(inputs, **kwargs) File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/transformer.py", line 387, in forward out = tensor.matmul(weights, v) File "/usr/local/lib/python3.7/site-packages/paddle/tensor/linalg.py", line 148, in matmul return op(x, y, 'trans_x', transpose_x, 'trans_y', transpose_y) SystemError: (Fatal) Operator matmul_v2 raises an paddle::memory::allocation::BadAlloc exception. The exception content is :ResourceExhaustedError:

Out of memory error on GPU 0. Cannot allocate 6.000244MB memory on GPU 0, available memory is only 7.500000MB.

Please check whether there is any other process using GPU 0.

1. If yes, please stop them, or start PaddlePaddle on another GPU.
2. If no, please decrease the batch size of your model.

(at /paddle/paddle/fluid/memory/allocation/cuda_allocator.cc:69) . (at /paddle/paddle/fluid/imperative/tracer.cc:172)`

理论上不会影响显存，可能需要你排查一下GPU的使用情况。

[dedex1994]

找到问题了，是我这边输出代码写的有问题，已经解决了，多谢

[godsoul]

这个代码是怎么改的。@dedex1994

[dedex1994]

改下PaddleHub/paddlehub/module/nlp_module.py 里的predict接口，把probs也一起return @godsoul

[wj-Mcat]

There will be different result/prob/attention result which should be the the output. Refer to: transformers model forward output

And this will be the great solution for paddlehub.