search

BlueBrain / HighFive

HighFive - Header-only C++ HDF5 interface
https://bluebrain.github.io/HighFive/
Boost Software License 1.0
680 stars 160 forks source link

Exception Triggered #224

Closed nyckmaia closed 5 years ago

nyckmaia commented 5 years ago

I did a simple "copy and paste" test with Qt 5 and HighFive library and it works good! I tested the H5Easy C++ example...

So, now I would like to use this library in my Qt C++ project.

In the middle of my code, I typed:

H5Easy::File file("example.h5", H5Easy::File::Overwrite);
std::vector<double> B  = { 1., 2., 3. };
H5Easy::dump(file, "/path/to/B", B, H5Easy::DumpMode::Overwrite);

All compiles Ok...no errors.

It creates the example.h5 file... it creates the B vector...all ok. But when the debugger hits the H5::dump() I got this error message:

error

The Qt console points to this file:

highfive\bits\H5Exception_misc.hpp:63: 
warning: Debugger encountered an exception: Exception at 0x7ffc2c9e9129, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) (first chance)

My questions: 1- What can I do to solve this exception? 2- Is there a way to know if the H5::dump() wrote all data with no errors? Like a return bool?

My system is:

ferdonline commented 5 years ago

Hi @nyckmaia If you start it from a command line, do you see any additional error messages?

nyckmaia commented 5 years ago

@ferdonline sorry...my application is a Qt GUI application.... I updated my question pointing to the H5Exception_misc.hpp:63 file... is it help you?

ferdonline commented 5 years ago

Indeed, from your last update it seems not... We had already reports that MSVC creates bad code in debug mode. Please compile in release mode and let me know if it solves the problem. Unfortunately we don't have manpower to debug these Windows cases. To check if the dataset was written, give a check at h5ls and h5dump

nyckmaia commented 5 years ago

Thank you @ferdonline ... Yes, I know that I can check the dataset using the h5dump command on Terminal/Console... But I would like to get a return value from the H5Easy::dump() C++ method. Is there any way to get some "return value" from this function?

alkino commented 5 years ago

In fact it already return the created DataSet or throw on error. Do you need more informations?

nyckmaia commented 5 years ago

I'm trying to catch the throw exception:

try {
    // Write density vector to a HDF5 file:
    H5Easy::dump(file, "/my/path/to/density", _density, H5Easy::DumpMode::Overwrite);
} catch (HighFive::Exception e) {
    // Print error on console:
    qDebug() << e.what();
}

But when the debugger hits the H5Easy::dump() I got the expection GUI window like my first post here (image above).

Questions: 1 - So, how can I hold the exception using HighFive? 2- Could you please give a short example about the return of H5Easy::dump()? I tried something like:

H5Easy::DataSet = H5Easy::dump(file, "/my/path/to/density", _density, H5Easy::DumpMode::Overwrite);

But I got a compilation error. Could you help me?

nyckmaia commented 5 years ago

Well, debugging my application I found a weird thing. Please, look this code:

H5Easy::File file("example.h5", H5Easy::File::Overwrite);

// Writing a simple test data:
std::vector<double> B = {1.0, 2.0, 3.0};
H5Easy::dump(file, "/my/path/to/B", B, H5Easy::DumpMode::Overwrite);

// Writing my data:
I'm passing the H5File file by reference to my 'save' method.
my_obj.save(file, "/my/path/to/density");

Here is the save method:

void Obj::save(H5Easy::File& file, const QString& path) const
{
    try {
        // Write density vector to a HDF5 file:
        H5Easy::dump(file, path.toStdString(), _density, H5Easy::DumpMode::Overwrite);
    } catch (HighFive::Exception e) {
        qDebug() << e.what();
    }
}

This code works good and I got no errors or exceptions.

Now the WEIRD thing:

If I remove these two lines:

std::vector<double> B = {1.0, 2.0, 3.0};
H5Easy::dump(file, "/my/path/to/B", B, H5Easy::DumpMode::Overwrite);

When the debugger hits the:

H5Easy::dump(file, path.toStdString(), _density, H5Easy::DumpMode::Overwrite);

I got the GUI exception error above in the first post here.

Why is this happening?

tdegeus commented 5 years ago

@nyckmaia Also I don't have windows nor windows experience. However looking at H5Easy_misc.hpp:63 it seems that you get an exception from file.createGroup(...). Could you check the group name, manually call the function, and confirm that you hdf5 was empty?

tdegeus commented 5 years ago

Also, your attempt to get the dataset missed a variable name

nyckmaia commented 5 years ago

@tdegeus Yes, I missed the H5Easy::DataSet variable name. So, to check if I wrote all data I'm doing something like:

H5Easy::DataSet ds = H5Easy::dump(file, "/my/path/to/density", _density, H5Easy::DumpMode::Overwrite);

if (ds.getElementCount() == 0) {
    // Error
}

It was my mistake...sorry about that...

nyckmaia commented 5 years ago

The Weird question

H5Easy::File file("example.h5", H5Easy::File::Overwrite);
std::vector<double> B = {1.0, 2.0, 3.0};
file.createGroup("/my/path/to/B");
H5Easy::dump(file, "/my/path/to/B", B, H5Easy::DumpMode::Overwrite);

When the debugger hits the H5Easy::dump(), I got the exception below. Please, look my screen and see the debugger variable names: debugger

What I have to do now?

alkino commented 5 years ago

Try it:

H5Easy::File file("example.h5", H5Easy::File::Overwrite);
std::vector<double> B = {1.0, 2.0, 3.0};
file.createGroup("/my/path/to");
H5Easy::dump(file, "/my/path/to/B", B, H5Easy::DumpMode::Overwrite);
nyckmaia commented 5 years ago

Great! It works! Thank you @alkino

So, only at the first time when my main application calls the H5Easy::dump() it needs first to create a HDF5 group without the data name in the group path. Like:

file.createGroup("/my/path/to");
H5Easy::dump(file, "/my/path/to/data_name", B, H5Easy::DumpMode::Overwrite);

After this first time call, I can call the H5Easy::dump() directly, without create the group before it and passing the full path + data name. Like:

H5Easy::dump(file, "/other_path/new_data", data, H5Easy::DumpMode::Overwrite);`

That's it? If yes, This post can be closed.

Thank you again! You are great!

alkino commented 5 years ago

You need to create all group chain 'my', 'my/path', 'my/path/to' only one time, after opening the file. After that you can create the dataset B inside this group with dump.

What you did before is creating a group named B inside my/path/to and so you cannot create a dataset on this group.

nyckmaia commented 5 years ago

The Last Thing

To solve this questions (create group before dump data) I modified my C++ method to this:

size_t Step_Density::save(H5Easy::File& file, const QString& group_path, const QString ds_name) const
{
    // Verify if the group exists:
    if (!file.exist(group_path.toStdString())) {
        // Create the HDF5 group path:
        file.createGroup(group_path.toStdString());
    }

    // Concatenate the group path with dataset name:
    QString full_path = group_path + "/" + ds_name;

    // Write data to HDF5 file:
    H5Easy::DataSet ds = H5Easy::dump(file, full_path.toStdString(), _density, H5Easy::DumpMode::Overwrite);

    // Get the number of written  elements:
    size_t el = ds.getElementCount();

    // Error checking:
    if(el == 0) {
        qDebug() << "Zero elements were written  to the HDF5 file";
    }

    return el;
}

But when the debugger hits the:

if (!file.exist(group_path.toStdString())) {

I got the same exception:

Exception at 0x7ffc2c9e9129, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) (first chance) at C:\Users\lamar\Desktop\rotortest\Core\third-party\highfive\bits\H5Exception_misc.hpp:63

Why this is happening?

alkino commented 5 years ago

In fact what help me to understand what happend in your Exception is the red circle in the previous capture you sent. Please, sent it again.

nyckmaia commented 5 years ago

error

alkino commented 5 years ago

This it not the information from the Exception call. The debugger need to be on the toException.

nyckmaia commented 5 years ago

Now the Qt Creator IDE Debugger is not going to the highfive\bits\H5Exception_misc.hpp:63 file. I got the exception message on the IDE console, but in the debugger I can go until here H5Node_traits_misc.hpp and not inside the H5Exception_misc.hpp file;

File: H5Node_traits_misc.hpp

image

When I enter in this bool NodeTraits<Derivate>::_exist(const std::string& node_name) function, the value of val variable is equal to 2.

After running this line:

htri_t val = H5Lexists(static_cast<const Derivate*>(this)->getId(),
                           node_name.c_str(), H5P_DEFAULT);

The variable val is equal to -1 and I got the exception error in the Qt console:

Exception at 0x7ffc2c9e9129, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) (first chance) at C:\Users\lamar\Desktop\rotortest\Core\third-party\highfive\bits\H5Exception_misc.hpp:63

So, after that, I press the F10 key and the debugger enter inside the if and hits:

HDF5ErrMapper::ToException<GroupException>(
            std::string("Invalid link for exist() "));

The aplication still running but I loose the "yellow debugger arrow" and I have to break/stop the debug session.

Please, tell me how could I help you to understand better this problem? I'm online all the time...

nyckmaia commented 5 years ago

Sorry, now I got it: File: highfive\bits\H5Exception_misc.hpp:63 image

alkino commented 5 years ago

No problem. Remove the if it's not really needed, I will discuss with my team how to improve interface. In fact, you test if "/my/path/to" exists and it crash because "/my" doesn't exist. We should add a recursive version of exist. Sorry for the inconvenience.

nyckmaia commented 5 years ago

Thank you @alkino for your patience and quick responses...

Please, let me know the updates and news about that... I'm developing a commercial software where this feature is essential...

Thank you again,

tdegeus commented 5 years ago

@alkino @nyckmaia Sorry I don't understand your bug (or in fact I'm not entirely sure that it is highfive that is giving you problems). H5Easy was designed such that it creates all groups that you need if needed. I.e. the following works:

#include <highfive/H5Easy.hpp>

int main()
{
  std::vector<size_t> a = {1, 2, 3};

  H5Easy::File file("test.h5", H5Easy::File::Overwrite);

  H5Easy::dump(file, "/path/a", a);
  H5Easy::dump(file, "/path/to/my/data", a);

  return 0;
}

Maybe you can try this code in isolation with your Qt compiler?

alkino commented 5 years ago

Oh, ok. So the createGroup is not needed.

The problem here was exist("/path/to/") before path exists. The underlying hdf5 lib don't return no but error.

nyckmaia commented 5 years ago

This is a completly new Qt Console project, and I got the same exception: image

Detail: In the right side, I have a debugger values. I expand the File variable and inside of it I have others 3 fields to expand. When I try to expand the HighFive::AnnotateTraits I loose all informations about the File variable. image

My system is:

Qt version details: image

Visual Studio version details image

Just to be clear, I'm not using Visual Studio, but the Qt C++ compiler is MSVC.

nyckmaia commented 5 years ago

Oh, ok. So the createGroup is not needed.

The problem here was exist("/path/to/") before path exists. The underlying hdf5 lib don't return no but error.

Hummm...so this exception is a HDF5 library internal bug? Is there any solution?

nyckmaia commented 5 years ago

For the new/clean source-code example from @tdegeus, here is the H5Exception_misc.hpp:63 file in debug mode: image

Important: Even that I got this exception errors, after stop the debug session and go to the Windows Console and type h5dump.exe test.h5 I got the correct data std::vector<size_t> a inside this file. image

tdegeus commented 5 years ago

@nyckmaia To me it sounds a lot like a bug of the underlying library. Can you try the library calls as in createGroup in isolation to zoom in?

nyckmaia commented 5 years ago

Of course @tdegeus, but I’m newer in HDF5 format.

Could you please give me a source-code example to do this?

ferdonline commented 5 years ago

There are several things at play here. To avoid turning this conversation in some blog-like thing, allow me to give some insight.

Internally exist() will call a lower level _exist() that indeed throws an exception in case h5Lexist returned a -1. This is just to be conforming to the generic HighFive way of doing things: If an error is produced we need to gather the error information with the ErrorMapper into the Highfive exception.

So even if you indeed get an exception at that point, the high level exist() (without underscore) will catch the exception and return a false. That's why apparently your program finishes and writes data.

See tests_high_five_base.cpp line 1069 for a test where that is working.

What can be wrong:

What you can do to debug

nyckmaia commented 5 years ago

@ferdonline thank you...

So, I tested this simple example on both IDE's:

include "Eigen/Dense"

include "highfive/H5Easy.hpp"

include

using Eigen::MatrixXd;

int main(int argc, char *argv[]) { // Only in Qt source-code version: QCoreApplication app(argc, argv);

H5Easy::File file("test.h5", H5Easy::File::Overwrite);

std::vector<double> a = { 1, 2, 3, 4, 5 };

MatrixXd m(2, 2);
m(0, 0) = 3;
m(1, 0) = 2.5;
m(0, 1) = -1;
m(1, 1) = m(1, 0) + m(0, 1);

H5Easy::dump(file, "/path/a", a);
H5Easy::dump(file, "/path/to/my/data", a);
H5Easy::dump(file, "/my/matrix", m);

// Only in Qt source-code version:
return app.exec();

}

I got **NO exception erros using Visual Studio**, and I got **3 exception** (one for each `dump()` line) using **Qt Creator IDE.**

The problem is that my developed **cross-plataform GUI is already done in Qt**, and I can't re-write it.
**Is there any possibility to mute/silence this exception using Qt Creator IDE?**
I tryed to do something like:

try { H5Easy::dump(file, "/path/a", a); } catch (H5Easy::Exception e) { // Do something to hold the exception... const char* msg = e.what(); }


But when the debugger hits the `dump()` line I got the exception **without** catch it.
nyckmaia commented 5 years ago

To me, this difference between Qt and Visual Studio is weird because Qt is using the same MSVC compiler and CDB debugger of Visual Studio.

Here is the Qt IDE list of auto detected Windows debuggers installed in my system: image

Here is the Compiler and Debugger that I'm using in this example project: image

So, in my understanding, the only difference is the IDE text editor...

Well, in the end of the day, to me, the most important NOW is to find a way to continue using Qt Creator IDE on Windows without getting the exceptions erros, but if any one would like to explain to me why Qt results are different of VS results, I will gratefull.

ferdonline commented 5 years ago

I got 3 exception (one for each dump() line)

If it was a real unhandled exception the program would immediately abort. What seems to me you have is some weird configuration in Qt creator that shows you a GUI message with the info about exceptions which are thrown, independently or not of the fact they are correctly caught. Actually more people complaining: https://forum.qt.io/topic/95563/flood-of-exception-warnings-is-there-any-way-to-disable-this

I'm very sorry but this is not something we can fix for you. Please look if there is something you can disable or contact Qt support. In the very last resort you can always fork and patch Highfive to never throw the exception but pass an error code among the functions instead.