Fiddler inspector

[jdom]

It would be nice to create a fiddler inspector, where you could point to a path with bond declaration files, and deserialize a request or response to a particular bond type that you specify, similar to how JSON is displayed.

Stretch goal: given the same configuration, allow fiddler to build a new request by serializing some JSON to a bond type.

[sapek]

It could be implemented as transcoding to JSON using runtime schema. In order to use .bond files as input we need another feature: #75.

[sapek]

75 was fixed with aeccf7d

[jzabroski]

Is there a way I can "know" a bond schema as a consumer? Microsoft Teams search is awful, and while their Twitter account says they are working to improve it, it literally takes me over an hour to scroll back to conversation on 12/17/2017 with a coworker I talk to daily (my boss). I see that it is implemented in Microsoft Bond, and so I was hoping to capture a request in Fiddler, intercept it, and hopefully force the pagination to be useful.

It's especially worse because if my boss IMs me while I am scrolling through the history, it resets my scroll window and I have to re-scroll again, costing another hour.

[chwarr]

@jzabroski, depending on which protocol is used, there may or may not be metadata included in the payload. This metadata may or may not include field names.

• Simple Binary: No metadata. Basically impossible to decode without a schema. Schemas must match exactly to decode correctly.
• Compact Binary/Fast Binary: Has field types and field numbers, but no names. Can decode without a schema using a protocol reader/transform directly. Backward/forward compatible changes handled without user intervention.
• JSON/XML: Has names but not exact field types. Backward/forward compatible changes handled without user intervention.

[dend]

@chwarr is there a C# sample that demonstrates compact binary decoding without a schema?

[khli002007]

في خميس، 27 يناير، 2022 في 10:50 م، كتب Den Delimarsky < @.***>:

@chwarr https://github.com/chwarr is there a C# sample that demonstrates compact binary decoding without a schema?

— Reply to this email directly, view it on GitHub https://github.com/microsoft/bond/issues/74#issuecomment-1023582948, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASCZ32OUZU7RSX4N4CDW4P3UYGOZ3ANCNFSM4BLRKYWQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

What do you want to be pious?

[chwarr]

No, there isn't a C# example of reading a struct via a reader directly. The basic algorithm using a CompactBinaryReader instance is:

1. ReadStructBegin
2. ReadFieldBegin: this will tell you the field ordinal and its type
   • if type is BT_STOP, be done with this struct
   • if type is BT_STOP_BASE, continue reading fields but know that you're one level lower in the hierarchy
   • if type is a scalar, string, or blob, call the appropriate ReadFoo method
   • if type is a container, call the appropriate ReadContainerBegin method: this will tell you the count of elements and the element type(s).
     • read count container elements, using the container type to know how to read each item
     • call ReadContainerEnd
   • if type is a struct, recur
3. ReadFieldEnd
4. Loop back to step 2 until you see BT_STOP or BT_STOP_BASE.
5. ReadStructEnd

Without a schema you won't be able to know some things, like the names of fields, whether a field is an enum or and int, whether a field is bonded, or whether a field is a list or a vector or a nullable.

Looking at the DeserializeCB.expressions file may be helpful. It's a syntax tree of the generated deserialization code for a struct that uses most of the features of Bond.

[dend]

Thank you for the follow-up, @chwarr - I think I can do something like:

var ib = new Bond.IO.Unsafe.InputBuffer(File.ReadAllBytes("req.bin"));
var cbr = new CompactBinaryReader<Bond.IO.Unsafe.InputBuffer>(ib);

cbr.ReadStructBegin();

BondDataType dt = BondDataType.BT_BOOL;
ushort id = 0;

while (dt != BondDataType.BT_STOP)
{
    cbr.ReadFieldBegin(out dt, out id);
    Console.WriteLine(dt + " " + id);
    if (dt == BondDataType.BT_STRING)
    {
        var stringValue= cbr.ReadString();
        Console.WriteLine(stringValue);
    }
    cbr.ReadFieldEnd();
}

It's interesting though that when I use ReadString in the example above (this is purely experimental), I am getting an exception:

At least from the outline you described, it seems that I am doing the right thing, but the error pops up regardless. Am I missing a step?

[dend]

On a different Bond-encoded file, the code above worked just fine, so it must be some kind of malformed file, otherwise I'd expect that to succeed. Unless there is an easy way to diagnose the issue?

[chwarr]

If you don't read the field, you need to skip it.

ReadFieldBegin is "interpret the next bytes as if they were the preamble of the next field", not "seek to next field and read it's preamble".

You need to make sure you initialize the reader with the right protocol and version. There's no header or checksum in serialized data, so the reader can't tell if you pointed a Fast Binary reader at Compact Binary stream. (Marshalled data has a header that indicates the protocol and version in the first four bytes.)

[dend]

Thanks @chwarr - I think I am getting somewhere with this. In a very unoptimized way, this can look something like:

var ib = new Bond.IO.Unsafe.InputBuffer(File.ReadAllBytes("response_data.bin"));
var cbr = new CompactBinaryReader<Bond.IO.Unsafe.InputBuffer>(ib, 2);

cbr.ReadStructBegin();

BondDataType dt = BondDataType.BT_BOOL;
ushort id = 0;

while (dt != BondDataType.BT_STOP)
{
    cbr.ReadFieldBegin(out dt, out id);
    Console.WriteLine(dt + " " + id);
    if (dt == BondDataType.BT_STRING)
    {
        var stringValue = cbr.ReadString();
        Console.WriteLine(stringValue);
    }
    else if (dt == BondDataType.BT_LIST)
    {
        BondDataType listContent = BondDataType.BT_BOOL;
        int counter = 0;
        cbr.ReadContainerBegin(out counter, out listContent);
        Console.WriteLine("Inside container: " + listContent);
        if (listContent == BondDataType.BT_STRUCT)
        {
            BondDataType structDt = BondDataType.BT_BOOL;

            cbr.ReadStructBegin();
            while(structDt != BondDataType.BT_STOP)
            {
                cbr.ReadFieldBegin(out structDt, out id);
                Console.WriteLine(structDt + " " + id);
                if (structDt == BondDataType.BT_STRING)
                {
                    var stringValue = cbr.ReadString();
                    Console.WriteLine(stringValue);
                }
                else
                {
                    if (structDt != BondDataType.BT_STOP)
                    {
                        cbr.Skip(structDt);
                    }
                }
            }
            cbr.ReadStructEnd();
        }
        cbr.ReadContainerEnd();
    }
    else
    {
        if (dt != BondDataType.BT_STOP)
        {
            cbr.Skip(dt);
        }
    }
    cbr.ReadFieldEnd();
}

Non-production quality code, obviously, but it allows reading in the structure here. This is exactly what I was looking for in terms of raw mode. From the field IDs, types, and values, I can then infer the schema.

Is there a way to automatically produce a schema from compact binary content (even though there are no names for fields) through code or the compiler that is built-in?

[chwarr]

Glad you were able to get it working.

Is there a way to automatically produce a schema from compact binary content (even though there are no names for fields) through code or the compiler that is built-in?

No, there is nothing in the library nor in the compiler.

[r12f]

I hit the same problem, so I made a small tool to inspect the binary data. It follows the same idea of @chwarr , but supports all the types: https://github.com/r12f/BondInspector. Hope this helps!

[dend]

In a similar vein, I built this to analyze Bond easier: https://github.com/dend/bond-reader