Stuck in function "testDatabase"

[hmchung]

Hi guys, I have issue running the demo_general. The compilation shows no issue, but the execution just hangs after producing this message: Creating a small database... After some simple investigation, I found that the execution seems to stuck in this loop: (File Vocabulary.cpp, line 1396-1411)

for(unsigned int i = 0; i < fn.size(); ++i)
  {
    NodeId nid = (int)fn[i]["nodeId"];
    NodeId pid = (int)fn[i]["parentId"];
    WordValue weight = (WordValue)fn[i]["weight"];
    std::string d = (std::string)fn[i]["descriptor"];

    m_nodes[nid].id = nid;
    m_nodes[nid].parent = pid;
    m_nodes[nid].weight = weight;
    m_nodes[pid].children.push_back(nid);

    DescManip::fromString(m_nodes[nid].descriptor, d);
  }

Does any one has the same issue or has any idea why? Thank you very much

Chung

[hmchung]

Latest update: I think the issue is pretty strange now... What I did is adding a debug message into that loop, simple as this: cout << "[Debugging] nid: " << nid << " - pid: " << pid << " - weights: " << weight << endl; So the for loop now look like this

  for(unsigned int i = 0; i < fn.size(); ++i)
  {
    NodeId nid = (int)fn[i]["nodeId"];
    NodeId pid = (int)fn[i]["parentId"];
    WordValue weight = (WordValue)fn[i]["weight"];
    std::string d = (std::string)fn[i]["descriptor"];

    m_nodes[nid].id = nid;
    m_nodes[nid].parent = pid;
    m_nodes[nid].weight = weight;
    m_nodes[pid].children.push_back(nid);

    DescManip::fromString(m_nodes[nid].descriptor, d);

    std::cout << "[Debugging] nid: " << nid << " - pid: " << pid  << " - weights: " << weight << std::endl;
  }

Then it runs, but not consisitently. 50% of the times, it would give this error message:

OpenCV Error: Assertion failed (s >= 0) in setSize, file /home/chung/ws/opencv_ws/opencv/modules/core/src/matrix.cpp, line 310
/home/chung/ws/opencv_ws/opencv/modules/core/src/matrix.cpp:310: error: (-215) s >= 0 in function setSize

Even when it runs, it would stuck again at the subsequent testDatabase() function... I am really confused now. Is it a compiler bug?

[junzhang2016]

I came across the same problem, and this link solved it: https://github.com/rmsalinas/DBow3/issues/6

[carlin314]

Hi, I came across this issue - "stuck" too in an embed platform, is this just the computation power too slow, or this is due to some inefficiency of program? I test in x86, it is ok, and can process 1w/s of my database, my database is 11w, so 11s will be ok, but in embed one, it is 0.7w/min, and it is to say it won't finish until 15min later, I didn't wait, but I am really confused, is this really need so much compute power? How you solve this, @hmchung I see in your github fork, I see just "if (i==0) usleep(1000)", and emmm, this doesn't really make ....that much sense, and I test it, nothing changed but wait 1s, so what's really going on with this issue, can I conclude it is just due to computation power not enough?

[carlin314]

It seems turn out to be the string stream is very slow on some platform without speed up: https://stackoverflow.com/questions/5830868/c-stringstream-is-too-slow-how-to-speed-up

and in essence, the database is not binary and that function is manual reverse serialization, which is not optimized.

If someone come across the same problem, the database can change to something like protobuf defined binary format.

[carlin314]

Forget what I said, in my case, it is due to OpenCV 4.0 reimplement of cv::FileNode data access

Persistence (storing and loading structured data to/from XML, YAML or JSON) in the core module has been completely reimplemented in C++ and lost the C API as well. For now base64 support is not complete (only loading base64-encoded XML and YAML is supported, encoding is not supported at all). Also, the random access of sequences stored in a FileNode is now a slow O(N) operation vs. fast O(1) in the previous implementation; use cv::FileNodeIterator for much faster sequential access. On the positive side, the loaded FileStorage's take 3-6x less memory than in the previous implementation.

refer to: https://github.com/opencv/opencv/wiki/ChangeLog#version400

Change the source code, use cv:;FIleNodeIterator will be ok.