Closed WebFreak001 closed 2 years ago
similarly trying to call serializeJson
on the IonDescribedValue
gives this compilation error:
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(1557,19): Error: no property `putSymbolId` for type `JsonSerializer!("", Appender!(char[]))`, did you mean `mir.ser.json.JsonSerializer!("", Appender!(char[])).JsonSerializer.putSymbol`?
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(516,57): Error: template instance `mir.ion.value.IonSymbolID.serialize!(JsonSerializer!("", Appender!(char[])))` error instantiating
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(551,22): instantiated from here: `serializeImpl!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/package.d(735,32): instantiated from here: `serialize!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(566,23): instantiated from here: `serializeValue!(JsonSerializer!("", Appender!(char[])), IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(578,28): ... (1 instantiations, -v to show) ...
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(378,34): instantiated from here: `serializeJsonPretty!(IonDescribedValue)`
source/served/lsp/protocol.d(96,26): instantiated from here: `serializeJson!(IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(1788,37): Error: template instance `mir.ion.value.IonDescribedValue.serializeDummy!(JsonSerializer!("", Appender!(char[])))` error instantiating
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(528,53): instantiated from here: `serialize!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(551,22): instantiated from here: `serializeImpl!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/package.d(735,32): instantiated from here: `serialize!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(566,23): ... (2 instantiations, -v to show) ...
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(378,34): instantiated from here: `serializeJsonPretty!(IonDescribedValue)`
source/served/lsp/protocol.d(96,26): instantiated from here: `serializeJson!(IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(531,53): Error: template instance `mir.ion.value.IonSexp.serialize!(JsonSerializer!("", Appender!(char[])))` error instantiating
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(551,22): instantiated from here: `serializeImpl!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/package.d(735,32): instantiated from here: `serialize!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(566,23): instantiated from here: `serializeValue!(JsonSerializer!("", Appender!(char[])), IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(578,28): ... (1 instantiations, -v to show) ...
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(378,34): instantiated from here: `serializeJsonPretty!(IonDescribedValue)`
source/served/lsp/protocol.d(96,26): instantiated from here: `serializeJson!(IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(2234,23): Error: no property `putKeyId` for type `JsonSerializer!("", Appender!(char[]))`, did you mean `mir.ser.json.JsonSerializer!("", Appender!(char[])).JsonSerializer.putKey`?
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(534,55): Error: template instance `mir.ion.value.IonStruct.serialize!(JsonSerializer!("", Appender!(char[])))` error instantiating
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(551,22): instantiated from here: `serializeImpl!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/package.d(735,32): instantiated from here: `serialize!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(566,23): instantiated from here: `serializeValue!(JsonSerializer!("", Appender!(char[])), IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(578,28): ... (1 instantiations, -v to show) ...
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(378,34): instantiated from here: `serializeJsonPretty!(IonDescribedValue)`
source/served/lsp/protocol.d(96,26): instantiated from here: `serializeJson!(IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(2814,23): Error: no property `putAnnotationId` for type `JsonSerializer!("", Appender!(char[]))`, did you mean `mir.ser.json.JsonSerializer!("", Appender!(char[])).JsonSerializer.putAnnotation`?
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(2594,34): Error: template instance `mir.ion.value.IonAnnotations.serialize!(JsonSerializer!("", Appender!(char[])))` error instantiating
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(537,66): instantiated from here: `serialize!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(551,22): instantiated from here: `serializeImpl!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/package.d(735,32): instantiated from here: `serialize!(JsonSerializer!("", Appender!(char[])))`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(566,23): ... (2 instantiations, -v to show) ...
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ser/json.d(378,34): instantiated from here: `serializeJsonPretty!(IonDescribedValue)`
source/served/lsp/protocol.d(96,26): instantiated from here: `serializeJson!(IonDescribedValue)`
../../../.dub/packages/mir-ion-1.0.31/mir-ion/source/mir/ion/value.d(487,9): Error: cannot return non-void from `void` function
IonDescribedValue
isn's supposed to be serializable directly. But you can do serialize it with symbol table:
static if (__traits(hasMember, serializer, "putKeyId"))
{
alias unwrappedSerializer = serializer;
}
else
{
import mir.ser.unwrap_ids;
auto unwrappedSerializer = unwrapSymbolIds(serializer, symbolTable);
}
unwrappedSerializer.serilalizeValue(value);
BTW, it feels you are doing something wrong.
Could you use JsonAlgebraic instead of JsonToken
?
For example, deserializeJson!JsonAlgebraic
or deserializeText!JsonAlgebraic
(for better JSON error messages).
ah JsonAlgebraic might be a better option. Can I then later convert the JsonAlgebraic to custom types through mir-ion somehow?
My idea with JsonToken was that it would optimally just be a slice of the source input for minimal overhead, but that didn't really seem possible with how deserialization works.
ah JsonAlgebraic might be a better option. Can I then later convert the JsonAlgebraic to custom types through mir-ion somehow?
You would need to write your own recursive constructor in the custom type.
My idea with JsonToken was that it would optimally just be a slice of the source input for minimal overhead, but that didn't really seem possible with how deserialization works.
Yes, it isn't possible. Json is converted to ion and then deserialized. The ion data is located on the stack or allocated using C library functions and scoped buffers.
If you need to read JSON from a file you can use deserializeJsonFile
.
In the same time, unlikely you may need to care about performance. It would be very fast anyway.
The fastest way is deserialization of a custom D types directly form json.
For example:
struct S
{
double[] array;
string name;
}
auto s = `{"array":[1.0,4.8],"name":"example"}`.deserializeJson!S;
I'm trying to use mir-ion with JSON-RPC where I basically have a wrapper JSON that contains additional information like method name and request id and then has the params, which are later passed to the proper handler through a custom type.
I feel like wrappers like this should be common enough that it should be somehow possible to have some kind of any
type that can be later converted to the actual resolved type. With painlessjson before I simply stored a JSONValue object everywhere where I either didn't want to implement any (de)serialization but still wanted to give the user access to custom fields or to delay (de)serialization for later/earlier.
struct Request
{
SumType!(string, int) id;
string method;
JSONValue params;
}
Mir doesn't support JSONValue
and SumType
, you can use Mir's JsonAlgebraic
and Variant
instead.
struct Request
{
import mir.algebraic: Variant;
import mir.algebraic_alias: JsonAlgebraic; // it is a recursive Variant.
Variant!(string, int) id;
string method;
import mir.serde;
@serdeOptional @serdeIgnoreDefault
JsonAlgebraic params;
}
Ah, for deserialization please use deserializeDynamicJson
- it can handle internal JsonAlgebraic
data.
I've got most things I needed rewritten now, but JsonAlgebraic doesn't seem to work with all strings.
I tried this code:
import std.stdio;
import mir.serde;
import mir.algebraic_alias.json : JsonAlgebraic;
import mir.deser.json : deserializeDynamicJson;
void main()
{
auto v = deserializeDynamicJson!JsonAlgebraic(`{"a":[1,"world",false,null]}`);
writeln(v);
}
which crashes at runtime with
mir.ion.exception.IonException@/home/webfreak/dev/serve-d/mir-ion/source/mir/deser/package.d(1230): Unexpected IonTypeCode for Algebraic!(TaggedType!(typeof(null), "null_"), TaggedType!(bool, "boolean"), TaggedType!(StringMap!(This, uint), "object"), TaggedType!(double, "float_"), TaggedType!(long, "integer"), TaggedType!(This[], "array"), TaggedType!(string, "string"))
----------------
/home/webfreak/dev/serve-d/mir-ion/source/mir/deser/ion.d:58 void mir.deser.ion.deserializeIon!(mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic, false).deserializeIon!().deserializeIon(ref mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic, scope const(char[][]), mir.ion.value.IonDescribedValue) [0x55c4f1866e79]
/home/webfreak/dev/serve-d/mir-ion/source/mir/deser/ion.d:124 int mir.deser.ion.deserializeIon!(mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic, false).deserializeIon!().deserializeIon(ref mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic, scope const(ubyte)[]).__foreachbody3(const(char[])[], mir.ion.value.IonDescribedValue) [0x55c4f1866d24]
/home/webfreak/dev/serve-d/mir-ion/source/mir/ion/stream.d:42 pure @nogc @safe int mir.ion.stream.IonValueStream.opApply(scope int delegate(const(char[])[], mir.ion.value.IonDescribedValue) pure @nogc @safe).__lambda2(mir.ion.exception.IonErrorCode, const(char[])[], mir.ion.value.IonDescribedValue) [0x55c4f18a005c]
/home/webfreak/dev/serve-d/mir-ion/source/mir/ion/stream.d:233 const pure nothrow @nogc scope @trusted int mir.ion.stream.IonValueStream.opApply(scope int delegate(mir.ion.exception.IonErrorCode, const(char[])[], mir.ion.value.IonDescribedValue) pure nothrow @nogc @safe) [0x55c4f18a04a2]
/home/webfreak/dev/serve-d/mir-ion/source/mir/ion/stream.d:247 const pure @nogc scope @trusted int mir.ion.stream.IonValueStream.opApply(scope int delegate(mir.ion.exception.IonErrorCode, const(char[])[], mir.ion.value.IonDescribedValue) pure @nogc @safe) [0x55c4f18a0823]
/home/webfreak/dev/serve-d/mir-ion/source/mir/ion/stream.d:39 const pure @nogc scope @safe int mir.ion.stream.IonValueStream.opApply(scope int delegate(const(char[])[], mir.ion.value.IonDescribedValue) pure @nogc @safe) [0x55c4f189fff6]
/home/webfreak/dev/serve-d/mir-ion/source/mir/ion/stream.d:79 const scope int mir.ion.stream.IonValueStream.opApply(scope int delegate(const(char[])[], mir.ion.value.IonDescribedValue)) [0x55c4f18a01af]
/home/webfreak/dev/serve-d/mir-ion/source/mir/deser/ion.d:122 void mir.deser.ion.deserializeIon!(mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic, false).deserializeIon!().deserializeIon(ref mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic, scope const(ubyte)[]) [0x55c4f1866cc9]
/home/webfreak/dev/serve-d/mir-ion/source/mir/deser/json.d:417 void mir.deser.json.deserializeDynamicJson!(mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic).deserializeDynamicJson(ref mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic, scope const(char)[]) [0x55c4f1866c86]
/home/webfreak/dev/serve-d/mir-ion/source/mir/deser/json.d:408 mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic mir.deser.json.deserializeDynamicJson!(mir.algebraic.Algebraic!(mir.algebraic.TaggedType!(, "null_").TaggedType, mir.algebraic.TaggedType!(bool, "boolean").TaggedType, mir.algebraic.TaggedType!(mir.string_map.StringMap!(mir.algebraic.This, uint).StringMap, "object").TaggedType, mir.algebraic.TaggedType!(double, "float_").TaggedType, mir.algebraic.TaggedType!(long, "integer").TaggedType, mir.algebraic.TaggedType!(mir.algebraic.This[], "array").TaggedType, mir.algebraic.TaggedType!(immutable(char)[], "string").TaggedType).Algebraic).deserializeDynamicJson(scope const(char)[]) [0x55c4f1866c44]
source/app.d:9 _Dmain [0x55c4f1866958]
good catch!
Fixed in v1.0.41
btw, deserializeJson
should understand now that JsonAlgebraic
is a dynamic type.
I'm trying to get a JSON representation of IonDescribedValue like so:
however this ends in this exception:
when trying to do