LLNL / conduit

Simplified Data Exchange for HPC Simulations
https://software.llnl.gov/conduit/
Other
213 stars 65 forks source link

json base64 parse doesn't do what it is supposed to #1326

Open JustinPrivitera opened 1 month ago

JustinPrivitera commented 1 month ago
std::string RC_CINEMA_WEB =  R"xyzxyz(

{
  "schema": 
  {
    "index.html": {"dtype":"char8_str","number_of_elements": 1,"offset": 0,"stride": 1,"element_bytes": 1,"endianness": "little"},
    "cvlib": {"dtype":"char8_str","number_of_elements": 1,"offset": 1,"stride": 1,"element_bytes": 1,"endianness": "little"}
  },
  "data": 
  {
    "base64": "ab"
  }
}
)xyzxyz";

conduit::Node res;
res.parse(RC_CINEMA_WEB,"conduit_base64_json");
std::cout << res.to_yaml() << std::endl;

gives you

index.html: "i�"
cvlib: "�"

Both should be a single character long. index.html is picking up the character from cvlib.

cyrush commented 1 month ago

thanks for finding this reproducer.

cyrush commented 1 month ago

How did you create this example? The string usually should include the null term char.

JustinPrivitera commented 1 month ago

I stole from here and modified: https://github.com/Alpine-DAV/ascent/blob/develop/src/libs/ascent/utils/ascent_resources_cinema_web.hpp

JustinPrivitera commented 1 month ago

Here's a better example:

std::string RC_CINEMA_WEB =  R"xyzxyz(

{
  "schema": 
  {
    "index.html": {"dtype":"char8_str","number_of_elements": 1,"offset": 0,"stride": 1,"element_bytes": 1,"endianness": "little"},
    "cvlib": {"dtype":"char8_str","number_of_elements": 1,"offset": 1,"stride": 1,"element_bytes": 1,"endianness": "little"},
    "cvlib2": {"dtype":"char8_str","number_of_elements": 1,"offset": 2,"stride": 1,"element_bytes": 1,"endianness": "little"}
  },
  "data": 
  {
    "base64": "ab123"
  }
}
)xyzxyz";

conduit::Node res;
res.parse(RC_CINEMA_WEB,"conduit_base64_json");
std::cout << res.to_yaml() << std::endl;
index.html: "i�v"
cvlib: "�v"
cvlib2: "v"
cyrush commented 1 month ago

Strings usually should include the null term char. Can you try creating one with conduit?

cyrush commented 1 month ago

When i see number of elements ==1, there maybe an expectation that is a null termed string.

Could be why our other cases are wrong, or it could mean this case is ill posed.

JustinPrivitera commented 1 month ago

How do I put the null term char into base64 representation to test?

JustinPrivitera commented 1 month ago

I would argue that conduit should only read number_of_elements elements instead of reading until it hits a null terminator