XDR spec: Support comments, directives, includes and imports, namespaces, nested data structures

[cblp]

I'm trying to apply RPC generator to Stellar spec: https://github.com/stellar/stellar-core/tree/master/src/xdr

There are lines like

// comment

%#include "file.h"

namespace {
...
}

Is it possible to support this syntax?

I found

token = PT.makeTokenParser PT.LanguageDef
  { PT.commentStart    = "/*"
  , PT.commentEnd      = "*/"
  , PT.commentLine     = "%"

in the parser, but in this case this is insufficient.

Ideally, it should support both % and // as commentLine, and also recognize directives like

%#include "xdr/Stellar-ledger-entries.h"

with emitting something like

import Stellar.Ledger.Entries

I also can rewrite

%#include "xdr/Stellar-ledger-entries.h"

into

%#import Stellar.Ledger.Entries

by hand (only 6 places), but I cannot add my code to the emitted module. Or is there a means to do it, that I missed?

[cblp]

Well, it can be corrected by hand and sed, but here's something really problematic:

union LiquidityPoolParameters switch (LiquidityPoolType type)
{
case LIQUIDITY_POOL_CONSTANT_PRODUCT:
    LiquidityPoolConstantProductParameters constantProduct;
};

setup: "lib/Stellar/XDR/Transaction.x" (line 8, column 1):
unexpected "{"
undefined type: "LiquidityPoolType"

Is it possible to support this construct?

[cblp]

Maybe that was only because of the parser doesn't know about LiquidityPoolType.

Now something even more interesting:

struct AccountEntryExtensionV1
{
    Liabilities liabilities;

    union switch (int v)
    {
    case 0:
        void;
    case 2:
        AccountEntryExtensionV2 v2;
    }
    ext;
};

nested data structures are not supported: "accountEntryExtensionV2'ext"
CallStack (from HasCallStack):
  error, called at ./Network/ONCRPC/XDR/Generate.hs:96:42 in ONC-RPC-0.1-ADMVgajLyqiLiFx4EW9Hax:Network.ONCRPC.XDR.Generate

[cblp]

If I understand correctly,

struct Foo {
  union {
  } bar;
};

is equivalent to

union Foo_bar {
};
struct Foo {
  Foo_bar bar;
};

[cblp]

This is how we can emit references to nested types: 71cc25992572d49a3a7ee5eb3013e9d478bc78ab. Dirty, but works, partly. Now I need to generate the bodies of declarations.

[dylex]

Not supporting // and %# was somewhat intentional, as they are not part of the XDR spec. Rather they are passed through and interpreted by the C pre-processor. However, supporting // is probably okay. The includes are a bit tricky because there's no direct conversion. It may make sense to pass-through % lines into haskell to allow code insertion, but this would mean rewriting things in different ways. (In general, some amount of pre-processing is expected to be necessary.) Do you have XDR files that won't build without the includes?

The nested structures is more interesting. The main problem is how to name them. As you say, you could name it Foo_bar, but theoretically there could already be a different top-level type named that (and in fact is in some of the specs I tested on). You also want them to have predictable names so you can use them. I haven't looked at this for a while, but there may be some option like Foo'bar that could work.

[cblp]

I'm working with this spec only https://github.com/stellar/stellar-core/tree/master/src/xdr