Tool package directive can't generate multi-level namespaces for C++

[bhamiltoncx]

In C++, multi-level namespaces look like:

namespace foo {
namespace bar {
// code for namespace foo::bar goes here
}
}

Currently, Files.stg for C++ only supports a single level namespace:

https://github.com/antlr/antlr4/blob/master/tool/resources/org/antlr/v4/tool/templates/codegen/Cpp/Files.stg#L50

Probably the right thing to do is to explicitly make file.genPackage an array and have the string templates iterate over each element to open / close the namespaces as appropriate for the language.

One question then is, what is the delimiter? Should we always require foo.bar.baz à la Java, or support foo::bar::baz as well?

[bhamiltoncx]

/cc @mike-lischke

[mike-lischke]

Hmm, no need for an array I think. The template can split and iterate also over a simple string (at least that's what I remember). For the delimiter we can allow both, trying double colon first and then dot. (I wish C++ would allow for namespace foo::bar { ... }).

[bhamiltoncx]

Yeah, if ST can do arbitrary splits and iterates then we could do that in each language's Files.stg.

[bhamiltoncx]

(It seems like it wouldn't be hard to make it an array, though, and that would avoid duplicating lots of split and iterate code.)

[mike-lischke]

Sure, it's just a matter where the splitting happens. But since only C++ needs that we should probably prefer the template, no?

[bhamiltoncx]

Part of what brought this up was me exploring what it would take to port the runtime and codegen to PHP/Hack, which use a pretty unusual syntax for namespaces Foo\Bar\Baz.

Since we need some sort of generalized way to transform input namespace delimiters to the target language, I figured we could do it at a high level.

[mike-lischke]

Good point, but on the other hand when you split at a high level you force all targets to construct their namespaces/import modules/etc. from that array, which is currently not necessary. What's the lesser evil?

[mike-lischke]

Ok, looks like we need a higher level solution. I cannot split a string by a delimiter in a template (I can join multiple to one though). But this is then not purely C++ related and all templates need an update. An array of package names sounds like the way to go here, especially as I have to determine how many namespace blocks were openend, so I can close them in a loop.

[jmairboeck]

C++17 has introduced support for nested namespace declarations like namespace foo::bar { ... } but I'm not sure whether Antlr can or should require that yet.

Even if trying to use this feature, specifying a colon-separated namespace on the command line leads to compile errors.

[derekklo]

Any likelihood of this issue getting resolved? I imagine it's a pretty common case within any significant C++ project. Though I would love to use C++17, it's still a couple years from adoption at my workspace.

[mike-lischke]

I'm afraid I won't have time to work on that. Any volunteer?

[francis0407]

I met this issue in my project, too. My team are using a script for this problem temporarily, but it would be better if there is an official solution.

[skoehler]

When running antlr 4.7.2, I have been using -package A::B and this has been working just fine with gcc. However, Visual Studio correctly complains that this a C++17 feature called nested namespace declaration. So instead of namespace A::B {...}, antlr should generate namespace A { namespace B {...}}. I can see how that doesn't fit into the current design. Has this been fixed in antlr 4.8 by any chance? IMHO, ANTLR should definitely not require C++17 for a feature like that.

[mike-lischke]

As mentioned above I will not have time to implement that. And I'm also not sure if this is possible to implement at all, because ANTLR4 uses a template system with generic text replacement. There's no way to convert text beyond some pretty basic functionality.

It's perhaps better to develop the grammar with a simple namespace, until everything is in a good shape and then switch to a nested namespace, once you don't need to re-generate the parser too often. Another alternative is to keep the generated files private with a simple namespace and use nested namespaces in wrapper classes which provide the underlying parsing functionality on a higher level.

[skoehler]

Does the template engine support string substitution? For example, you could replace every :: with { namespace and then namespace A::B { becomes namespace A { namespace B {. A similar trick might work for the closing braces, but I'm afraid you would need regular expressions to transform A::B into curly braces.

An alternative is to simple prepare the namespace declaration in Java and provide 2 strings in addition to the unprocessed template string. I can't imagine that the code generation is only done within the templates. I would assume that the Java code already included some language specific parts. But I might be wrong.

Yet another alternative is that you allow us to override the default generated namespace declaration (including the opening and closing curly brace) by some custom strings specified via -D on the command line.