marmeladema
commented
3 years ago It seems that the code snippet I mentioned before does not actually work.
Here is an example of trying to re-use a document object after having encountered a parsing error:
#include <iostream>
#include "simdjson.h"
using namespace std;
struct simdjson_context {
simdjson::ondemand::parser parser;
simdjson::ondemand::document document;
};
int main() {
simdjson::error_code error;
simdjson_context *context = new simdjson_context();
auto json = R"({"key": "value")"_padded;
cout << "Step 1: parsing truncated document should fail" << endl;
{
auto error = context->parser.iterate(json.data(), json.size(), json.size() + simdjson::SIMDJSON_PADDING).get(context->document);
if (error == simdjson::SUCCESS) {
simdjson::ondemand::object object;
error = context->document.get_object().get(object);
if (error == simdjson::SUCCESS) {
// Never reached because an error is reported
abort();
} else {
cout << "get_object() error: " << simdjson::error_message(error) << endl;
}
} else {
cout << "iterate() error: " << simdjson::error_message(error) << endl;
}
}
json = R"({"key": "value"})"_padded;
cout << "Step 2: parsing a valid document should succeed" << endl;
{
context->document.~document();
context->document = simdjson::ondemand::document();
auto error = context->parser.iterate(json.data(), json.size(), json.size() + simdjson::SIMDJSON_PADDING).get(context->document);
if (error == simdjson::SUCCESS) {
simdjson::ondemand::object object;
error = context->document.get_object().get(object);
assert(error == simdjson::SUCCESS);
simdjson::ondemand::field field;
for (auto it = object.begin(); it!=object.end(); ++it) {
error = (*it).get(field);
if (error == simdjson::SUCCESS) {
cout << "key = " << field.key() << endl;
} else {
cout << "get() error: " << simdjson::error_message(error) << endl;
}
}
} else {
cout << "iterate() error: " << simdjson::error_message(error) << endl;
}
}
return 0;
}And it prints:
Step 1: parsing truncated document should fail
get_object() error: The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc.
Step 2: parsing a valid document should succeed
get() error: The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc.Step 1 behaves properly, an error is reported but when trying to re-use the same document instance on new data (valid this time), and error is reported again whereas it should not. I understand that it's probably because re-using a document is not meant to work but having such a capability would be really helpful.
lemire
jkeiser
Hello!
I am working on a rust project that is using simdjson internally and in order to easily interface with it, I have created a small C shim where all simdjson classes are hidden and everything is exposed through abstract pointers and new/free like functions.
Given the current explicit warning in the documentation:
It means that I have to free and allocate a new document each time I need to call
.iterate()on the parser instance.I would like not to have to pay that allocation cost and re-use a document instance each time I need to parse something. Would doing the following be considered good practice?
Additionally, I am definitely not C++ fluent but I've read here and there that calling destructors explicitly was a bad design. Is there another API available in order to achieve this?
Thank you in advance for your help!