PaddlePaddle / PaddleOCR

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)
Apache License 2.0
40.34k stars 7.47k forks source link

关于使用ppstructure版面分析模型3个模型(都是210M大小的模型:)报错,3个没有达到满意的结果 #13270

Closed freezehe closed 21 hours ago

freezehe commented 4 days ago

问题描述 / Problem Description

如题:我是在centos7服务器上通过hub serving 方式部署服务,java业务代码调用该服务,控制台报了一些奇怪的源码错误。测试了6种版面模型都是不行的,轻量级9M大小的 模型返回的只有标记table,没有table里面的内容,另外3个210M的模型就是直接报错了,现在贴的就是最后一个模型ppyolov2_r50vd_dcn_365e_tableBank_latex。

我的需求是:我想识别这个图片的内容,包括表格和其他元素。 p1

运行环境 / Runtime Environment

复现代码 / Reproduction Code

我的java业务代码如下: package com.freeze;

import org.apache.commons.codec.binary.Base64; import org.apache.http.HttpEntity; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpPost; import org.apache.http.entity.StringEntity; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils; import org.json.JSONArray; import org.json.JSONObject;

import java.io.File; import java.io.FileInputStream; import java.io.IOException;

public class PaddleHubTableClientV3 {

public static void main(String[] args) {
    String serverUrl = "http://172.16.12.229:8868/predict/structure_layout"; // PaddleHub Serving默认URL
    File imageFile = new File("D:\\1\\test-ocr\\test-ocr-0629\\src\\main\\resources\\img\\p1.png");

    try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
        HttpPost uploadFile = new HttpPost(serverUrl);

        // Read image file and encode to Base64
        FileInputStream fileInputStream = new FileInputStream(imageFile);
        byte[] imageBytes = new byte[(int) imageFile.length()];
        fileInputStream.read(imageBytes);
        String encodedImage = Base64.encodeBase64String(imageBytes);
        fileInputStream.close();

        // Create JSON object with the Base64 encoded image
        JSONObject json = new JSONObject();
        json.put("images", new String[]{encodedImage});

        // Print the request JSON for debugging
        System.out.println("Request JSON:");
        System.out.println(json.toString());

        // Set the request content type to application/json
        StringEntity requestEntity = new StringEntity(json.toString(), "UTF-8");
        uploadFile.setEntity(requestEntity);
        uploadFile.setHeader("Content-Type", "application/json");

        // Execute the request
        CloseableHttpResponse response = httpClient.execute(uploadFile);
        HttpEntity responseEntity = response.getEntity();
        String responseString = EntityUtils.toString(responseEntity, "UTF-8");
        response.close();

        // Print the original response for debugging
        System.out.println("Original Response:");
        System.out.println(responseString);

        // Parse the response
        JSONObject jsonResponse = new JSONObject(responseString);
        if (jsonResponse.has("status") && !jsonResponse.getString("status").equals("000")) {
            System.out.println("Error Status: " + jsonResponse.getString("status"));
            System.out.println("Error Message: " + jsonResponse.getString("msg"));
        } else if (jsonResponse.has("results")) {
            JSONArray resultsArray = jsonResponse.getJSONArray("results");
            for (int i = 0; i < resultsArray.length(); i++) {
                JSONObject result = resultsArray.getJSONObject(i);
                String html = result.getString("html");

                // Print the HTML content for debugging
                System.out.println("HTML Content:");
                System.out.println(html);

                // Optionally parse the HTML content to extract table data
                // (Implement HTML parsing logic if needed)
            }
        } else {
            System.out.println("Unexpected response structure.");
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
}

}

完整报错 / Complete Error Message

Original Response: {"msg":"In user code:\n\n File \"tools/export_model.py\", line 107, in \n main()\n File \"tools/export_model.py\", line 103, in main\n run(FLAGS, cfg)\n File \"tools/export_model.py\", line 69, in run\n trainer.export(FLAGS.output_dir)\n File \"/Users/anmengtao/Desktop/model_ED/PaddleDetection-release-2.0/ppdet/engine/trainer.py\", line 460, in export\n input_spec, static_model.forward.main_program,\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py\", line 534, in main_program\n concrete_program = self.concrete_program\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py\", line 454, in concrete_program\n return self.concrete_program_specify_input_spec(input_spec=None)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py\", line 487, in concrete_program_specify_input_spec\n concreteprogram, = self.get_concrete_program(\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py\", line 402, in get_concrete_program\n concrete_program, partial_program_layer = self._program_cache[cache_key]\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py\", line 711, in getitem\n self._caches[item] = self._build_once(item)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py\", line 698, in _build_once\n concrete_program = ConcreteProgram.from_func_spec(\n File \"\", line 2, in from_func_spec\n \n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py\", line 25, in impl\n return wrapped_func(*args, kwargs)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py\", line 39, in impl\n return func(*args, kwargs)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py\", line 652, in from_func_spec\n outputs = static_func(inputs)\n File \"/var/folders/0t/0lmgjgfj7012tggtgbwtqby80000l2/T/tmp0hu0he2y.py\", line 26, in forward\n out = paddle.jit.dy2static.convert_ifelse(self.training, true_fn_1,\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py\", line 210, in convert_ifelse\n return _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py\", line 235, in _run_py_ifelse\n return true_fn(true_args) if pred else false_fn(false_args)\n File \"/Users/anmengtao/Desktop/model_ED/PaddleDetection-release-2.0/ppdet/modeling/architectures/meta_arch.py\", line 29, in forward\n out = self.get_pred()\n File \"/Users/anmengtao/Desktop/model_ED/PaddleDetection-release-2.0/ppdet/modeling/architectures/yolo.py\", line 75, in get_pred\n bbox_pred, bbox_num = self._forward()\n File \"/Users/anmengtao/Desktop/model_ED/PaddleDetection-release-2.0/ppdet/modeling/architectures/yolo.py\", line 59, in _forward\n body_feats = self.backbone(self.inputs)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py\", line 891, in call\n outputs = self.forward(inputs, kwargs)\n File \"/Users/anmengtao/Desktop/model_ED/PaddleDetection-release-2.0/ppdet/modeling/backbones/resnet.py\", line 573, in forward\n conv1 = self.conv1(x)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py\", line 891, in call\n outputs = self.forward(*inputs, kwargs)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/container.py\", line 86, in forward\n input = layer(input)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py\", line 891, in call\n outputs = self.forward(*inputs, kwargs)\n File \"/Users/anmengtao/Desktop/model_ED/PaddleDetection-release-2.0/ppdet/modeling/backbones/resnet.py\", line 122, in forward\n out = self.conv(inputs)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py\", line 891, in call\n outputs = self.forward(*inputs, *kwargs)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/nn/layer/conv.py\", line 631, in forward\n out = F.conv._conv_nd(\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/nn/functional/conv.py\", line 143, in _conv_nd\n helper.append_op(\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/layer_helper.py\", line 43, in append_op\n return self.main_program.current_block().append_op(args, kwargs)\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/framework.py\", line 3012, in append_op\n op = Operator(\n File \"/Users/anmengtao/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/framework.py\", line 2102, in init\n for frame in traceback.extract_stack():\n\n InvalidArgumentError: The size of Op(Conv) inputs should not be 0.\n [Hint: Expected in_dims[i] != 0, but received in_dims[i]:0 == 0:0.] (at /paddle/paddle/phi/infermeta/binary.cc:494)\n [operator < conv2d > error]","results":"","status":"101"}

可能解决方案 / Possible solutions

附件 / Appendix

我贴一下我的这个目录下的param.py配置:PaddleOCR/deploy/hubserving/structure_layout image

image

freezehe commented 4 days ago

补充一下,我贴一下我hub serving 部署的脚本是:nohup hub serving start -m ocr_system structure_layout -p 8868 &

然后再补充一下其中一个轻量级模型返回的结果:如下 image

freezehe commented 4 days ago

我想要的是table里面的内容,而不是告知我这个是table的置信度。是否有模型支持?