Abstract syntax tree (AST) for rust-file

[xilec]

Many smart operations (and not only smart, e. g. #11) within true IDE (not text editor with macroses) requare knowledge about AST.

Is there any plans for realizing rust AST?

Good open source example of AST is Roslyn (https://roslyn.codeplex.com/)

[vosen]

Also NRefactory. Rust's own parser is mostly usable (http://doc.rust-lang.org/syntax/parse/parser/struct.Parser.html#method.parse_crate_mod) but, as I've mentioned it in #5, it is unsuitable for our use, because it can't really recover from errors. One way of implementing it would be to encode rust's grammar in ANTLR. There's ANTLR lexer in rust sources already (https://github.com/rust-lang/rust/blob/master/src/grammar/RustLexer.g4). Though such project should probably be done in C/rust. Other projects, like racer, would be interested in this too. Alternatively one could improve rust's libsyntax. I'm sure rust devs would welcome such changes, as this issue is quite an annoyance when using the language. As for us, I plan to do what I've been doing so far. That is, ignore the issue and work on he other stuff. We're really quite behind on the other fronts (debugging, project system, etc).

[xilec]

Nitra looks very promising, especially for developing plugin for Visual Studio.

[vosen]

In the meantime, I tried my hand at writing Rust lexer from scratch here. I'll probably replace out ANTLR-based lexer after some testing and I'll be experimenting with writing a parser that is more suitable for our use. There's a quality Rust grammar here, so it might be quite doable.

[Boddlnagg]

Wouldn't it also make sense to have a parser (and semantic analysis engine similar to NRefactory/Roslyn) for Rust implemented in C#? I could imagine that interfacing with a native parser may be quite difficult and require us to keep in-memory caches duplicated (once in the native implementation, which would be a DLL that exports a C interface, and once in VisualRust).

In the (veeery) long term this could be extended to be a complete alternate Rust compiler implementation, and I think it would make sense to have an implementation in a different language than Rust itself. (It would also be cross-platform now that MS is opening the .NET platform). Of course, a .NET-based implementation might not be as fast as a native one.

I don't want to push in either direction here, just want to discuss the tradeoffs involved.

[vosen]

Yes, writing a C interface and C# wrapper would be tedious, but there are important reasons to favor Rust. I disagree on the memory costs. C and C# wrapper could expose free-form access to the AST (with memory-costly things like substrings computed on-demand) and high level operations (or high level wrapper), where memory would be duplicated at the end, when returning results.

Simply, there's a much larger pool of people interested in parsing Rust that write Rust than pool of people interested in parsing Rust that write C#. This is kind of thing that can benefit all Rust IDEs.. Also, we would hopefully be able to reuse libsyntax's later stages of the compilation: symbol resolution, binding, etc.

Feel free to push in any direction. It's all academic anyway until someone actually starts writing code.