search

pomsky-lang / pomsky

A new, portable, regular expression language
https://pomsky-lang.org
Apache License 2.0
1.28k stars 19 forks source link

Distribute Pomsky as a native library #62

Open lppedd opened 1 year ago

lppedd commented 1 year ago

Distributing Pomsky not only in the CLI format, but also as a native library (.so, .dll), allows tools to compile and receive an output in a standardized format which can be interpreted through interfaces like JNI or JNA. Example usage: compilation and error reporting inside of an IDE.

I think a project like this https://github.com/Dushistov/flapigen-rs might come handy.

Aloso commented 1 year ago

My idea was to add a CLI flag to print compilation results as JSON, which could be used by IDEs. However, that is not implemented yet.

But to make your IDE extension work, I think all you need is jni-rs. You can follow this tutorial (maybe there are better ones, this is just the first one I found) and make a crate that exports a Java_Pomsky_compile function for the JNI. Then you just need to map Rust types to Java types. For example, a CompileError can be returned as two integers for the Span and a string.

By the way, I was planning to look at your lexer and BNF grammar but didn't have time until now. I'll try to give you some feedback tomorrow.

lppedd commented 1 year ago

@Aloso JSON is a good idea. However, are we sure it's ok performance wise? I would like to call it every time there is an edit, like in the playground, and it has to report errors as fast as possible. Playground uses a WASM module, so it's basically a native interaction.

If I make a crate, do I have to recompile each time a new version comes out? I believe the answer is yes, which means it becomes a bit cumbersome.

lppedd commented 1 year ago

This is how an IDE would handle this situation.

  1. Allow the user to select a CLI executable
  2. Extract the Pomsky version from the CLI
  3. If the user enables the enhanced error reporting, the native library is downloaded through Github
  4. A native call is performed and errors are overlayed
Aloso commented 1 year ago

Creating a process takes less than a millisecond on recent hardware, I think, so it's not that expensive to call pomsky's CLI. But to get the best performance, using something like JNI would be better. Yes, that requires recompiling the dynamic library every time a new version comes out, but maybe that can be automated somewhat.

lppedd commented 1 year ago

@Aloso just keep in mind extensions should just inherit from the tool. If extensions have to carry their own compiled versions of Pomsky, it becomes a binary hell.