Language Server Support

[BracketMaster]

I'm wondering how much it would take to bring a language server to Bluespec.

[rsnikhil]

One problem is that we don't have a standalone grammar for either BSV or BH, separate from the actual Haskell code that implements the parsers, and those parsers are a bit messy. I had been thinking of writing a treesitter grammar which I believe is commonly used nowadays for language servers.

http://tree-sitter.github.io/tree-sitter/
https://github.com/tree-sitter/

Please let me know if you'd like to collaborate on this (I can put together the grammar, but I don't know much how to incorporate into a language server.)

[krame505]

The language server protocol has support for semantic tokens, which can be used to provide highlighting - I could imagine building a language server in Haskell directly uses the existing parser, instead of rewriting it with treesitter.

https://github.com/haskell/lsp should be helpful here for building an LSP implementation in Haskell.

[ThePerfectComputer]

Well - the issue isn't only highlighting - what about jump to definition

[krame505]

That sort of information could be extracted from the symbol table - the LSP implementation could utilize the relevant bits of the compiler implementation, too. I was just responding to Nikhil's comment about using Tree-sitter for parsing.

[ThePerfectComputer]

Ah...

This makes me think of another concern however - which is I think that bsc should have first class build files kind of how Rust has Cargo.toml and Swift has Package.swift.

If bsc has support for a Package.bsc that contained all the necessary info to build the project, then the Haskell server could know where to find bsc files and would be able to jump to definition accordingly...

[quark17]

BSC's build files can be queried using bluetcl (to get information such as the position of definitions within source files). The BDW graphical development environment is an example of using bluetcl: the source code for BDW just does the windowing, and contains no special knowledge about BSC's files; all the info that BDW displays (and it's ability to jump to locations in the source or signal names in the generated Verilog) comes from bluetcl queries.

If you've got a situation where a text file of information (like Package.bsc) would be preferable, then you can create that by running a bluetcl script: after BSC executes, run a script that reads in the design and outputs the info you need from it. We could make this a feature of BSC itself, but that would require discussions to settle on a format etc; in the meantime, anyone can write a bluetcl script and start developing a format that works for them.

Bluetcl just means a set of commands that have been embedded inside Tcl. And so it's easy to write Tcl scripts that directly make calls to these commands. If there's interest in having these commands be embedded into another language (for example, Python), that would be easy for someone to do, and would not impact existing bluetcl or bsc support/maintenance.

The commands currently available in bluetcl are documented in the BSC user guide.

[ThePerfectComputer]

OK - wow! This is very powerful. I'm constantly being surprised at how much BSC can do each day. I wish more people knew about it...

I'll take a look at the manual.

[ThePerfectComputer]

If there's interest in having these commands be embedded into another language (for example, Python), that would be easy for someone to do, and would not impact existing bluetcl or bsc support/maintenance.

How would one go about this?

[ThePerfectComputer]

I'm assuming that tcl would link to haskell via C-ffi somewhere?

[quark17]

The BDW graphical development environment is an example of using bluetcl

Also, there are simpler examples in the BSC repo, in bsc/util/bluetcl-scripts/. BDW is big code base, and this directory has some one-page examples: one walks a design to find all the Verilog files (generated and library), to provide to a synthesis tool, say; one looks at the ports of a generated module and, if the types are structs, generates a wrapper Verilog module that separates the fields into their own Verilog ports.

If there's interest in having these commands be embedded into another language (for example, Python), that would be easy for someone to do, and would not impact existing bluetcl or bsc support/maintenance.

How would one go about this?

Here's what we do for Tcl: We use Haskell's foreign function interface (FFI) to export a Haskell function as a callable C function. We then link that with handwritten C code, which pulls it into the Tcl interpreter as a new library.

If there's another language that you want to embed BSC commands into, and if that language supports importing C functions into the language, then you can do something similar: In the BSC codebase, declare the function export; and then follow the import steps for that other language.

You can't use the existing export, because it does have some features that are specific to Tcl. But, the exported function is -- mostly -- a BSC-to-Tcl wrapper around a core that's regular BSC code. So you'd need to write a wrapper around the core functions, but you wouldn't need to reinvent any of the core. That's what I meant by "would be easy for someone to do".

The file that defines the Tcl wrapper is bluetcl.hs and you can see the export declaration at the top:

foreign export ccall "blueshell_Init_Foreign" blueshell_Init :: TclInterp -> IO Int

It takes in a TclInterp object, which is an abstract type that can't be operated on by Haskell, it can only be passed an argument into C functions from the Tcl codebase (such as functions for raising errors and functions for constructing new value objects).

The exported function is defining a Tcl library, so there's a namespace of multiple commands, that are registered by this function. If you look down in the file, you'll see tclCommands defines a list. And further down, you can see that a command like type (for querying info about a BSC type) is defined by tclType, which is a function that switches based on the subcommand given to it. That's because of how Tcl commands work: you type cmd subcmd args. The return value in Tcl is a string, made up of a list of tokens, and when a Haskell function tclType is ready to return an argument, it provides a list of strings, and the wrapper converts that into a Tcl return value (by calling some of the C functions to create a new object and populate it). You'll see that tclType calls typeAnalysisToHTclObj to construct its return value, and that's defined in the file TypeAnalysisTclUtil.hs, which is a wrapper around the pure BSC package TypeAnalysis.hs.

So, at the core, there's a function that's being called, and a wrapper that packages it up and converts types back and forth. (The wrapper also catches any exceptions in the Haskell code and raises them in the other language by appropriate call to an imported foreign function, or by returning a null value.) In the case of Tcl, all the functions are packaged up into one function, which instantiates them as a library. But for other languages, you might just export each individual function -- which would be a lot simpler. The wrapper, in that case, would just need to massage between types.

Anyway, just to complete the story for Tcl: The exported function is used inside a handwritten C file that's also in bsc/src/comp/ as bluetcl_Main.hsc, and you can see that it expects the exported C function:

#include "BlueTcl_stub.h"

I believe this is a constructed name, containing the exports declared in the BlueTcl package (which is the package name inside bluetcl.hs -- I'm not sure why the file and package name don't have the same case).

For Tcl, our C file defines main, so we use GHC to link it all together (and pass it the -no-hs-main). That's because we're generating a new Tcl executable that has BSC commands built in. By using GHC to do the linking, it provides the necessary flags for including the RTS etc that the Haskell code needs. If you want to embed BSC commands as a library that's dynamically loaded into the language (and I guess we could have gone that route with Bluetcl), you'll need to create that differently. (You might have to call GHC's link step with -v to see what flags it gives, if you're going to create the library yourself and not with GHC's linking.)

[ThePerfectComputer]

OK - it sounds like I have some options and I have roughly an idea of how I would stitch this all together - although - I'm quite happy to write haskell directly actually - I could straight up do a language server in haskell I would think?

Also - BlueSpec classic seems like a fairly powerful language - this could be a crazy idea - but how suited is it to write general purpose programs such as a language server?

[quark17]

I could straight up do a language server in haskell I would think?

Yes. If you're writing a language server for BSC, it's probably best to do it directly in Haskell, as @krame505 suggested.

Also - BlueSpec classic seems like a fairly powerful language - this could be a crazy idea - but how suited is it to write general purpose programs such as a language server?

BH/Classic isn't any more powerful than BSV (just possibly more concise). You probably could write a language server in BH or BSV, but I think you'd be shoehorning it into a computation model that isn't appropriate, so I wouldn't recommend it.