eclipse-cyclonedds / cyclonedds-python

Other
54 stars 44 forks source link

Check well-formedness of CDR before extracting key #261

Open eboasson opened 4 days ago

eboasson commented 4 days ago

The python serializer doesn't detect that in:

enum E { A, B, C }
union T switch(E) { ... }
union U switch(boolean) { case true: T value }
struct S { U m }
write (S(m=T(discriminant=B, value=...)))

there's an entire union missing in the value to be written, and because it can't find "B" in the list of labels in U (i.e., "B" is not in "[True]") it serializes "B" and skips "value". It so happens that "B"'s serialization matches that of "True" (on a little endian machine, but there are variants where endianness doesn't matter), and so "dds_stream_extract_key" will take the "true" case and try to deserialize T. It doesn't handle malformed input, so bad things happen.

I don't know how to prevent this type confusion in the Python serializer, but I do know how to check for malformed input.