manifold-systems / manifold

Manifold is a Java compiler plugin, its features include Metaprogramming, Properties, Extension Methods, Operator Overloading, Templates, a Preprocessor, and more.
http://manifold.systems/
Apache License 2.0
2.36k stars 124 forks source link

[Proposal] Inline-assembler (Continuation of pre-processor, but for assembly) #596

Open terminalsin opened 3 months ago

terminalsin commented 3 months ago

Is your feature request related to a problem? Please describe. When developing sensitive java applications rises the need to have maximized performance, or perhaps more control over the lower level... or you're writing an obfuscator! Perhaps even writing sensitive unit tests? Or teaching students how Java bytecode...

All these reasons lead to one main idea: inlined assembly. This is notably found in other languages, such as:

- Rust 👇
```rust
fn main() {
    let result: i32;
    unsafe {
        asm!("movl $$5, {0}", out(reg) result);
    }
    println!("Result: {}", result);
}

import "fmt"

func main() { var result int asm volatile ( "movl $5, %%eax;" "movl %%eax, %0;" : "=r" (result) : : "%eax" ) fmt.Println("Result:", result) }


You get the jist. What about Java? Well, there have been attempts, albeit some more successful than others:
- https://github.com/xxDark/asm-inline

Furthermore, many syntaxes and assemblers exist:
- https://github.com/davidar/jasmin (legacy)
- https://github.com/jumanji144/Jasm (more modern and stable)

**Describe the solution you'd like**
Ideally, it'd be amazing to be able to implement either:

- A preprocessor-like syntax for bytecode
- A standardised label for bytecode

Example:
```java
public class InlineBytecodeExample {

    // Hypothetical inline bytecode method
    public int executeInlineBytecode() {
        int result;

        // Hypothetical inline bytecode syntax
        bytecode {
            // Push 5 onto the stack
            iconst_5;
            // Store the top of the stack into the local variable 'result'
            istore result;
        }

        return result;
    }

    public static void main(String[] args) {
        InlineBytecodeExample example = new InlineBytecodeExample();
        int result = example.executeInlineBytecode();
        System.out.println("Result: " + result);
    }
}

Describe alternatives you've considered n/a, all alternatives are washy washy

Additional context There are several roadblocks to this one needs to consider very heavily:

However! If this does shape up and become a reality, I pledge to writing and create a full class for my university which makes use of Manifold to teach students about Java bytecode assembly

rsmckinney commented 3 months ago

Interesting idea.

Initial thoughts...

The inline project appears to be a post compilation step to swap out the ASM block call with the bytecode in the ASM block. Interesting. Whereas the jasmine project is basically a mnemonic/symbolic bytecode parser to enable decompilation to the symbolic format to edited then reparsed and rewritten to the original class file.

If I understand correctly, your proposal is essentially to combine these concepts to enable inline symbolic bytecode, but without an extra build step, meaning you want it directly integrated into the compile via manifold.

Being an inline feature, the bytecode would be limited to locals and control instructions, no declarations such as classes, methods, fields.

First off, a grammar and parser would be needed for the symbolic bytecode language. I assume the jasmine project has one that could be borrowed. I would lean more toward writing an antlr grammar targeting the subset for inline bytecode, or just writing a custom recursive descent parser. Probably the latter.

The parser would probably transform to ASM calls.

One obstacle would be stack map frames. The frame in the enclosing bytecode would have to be rewritten if any locals are introduced in the inline.

Another tricky part involves local variable references. Since bytecode accesses them as offsets in the stack, the parser would have to calculate these offsets independently. This step could be avoided if not for the parser having to verify local references during the symbol attribution phase as opposed to the generation phase.

The difficulty with lambdas/invokedynamic is generating the synthetic method corresponding with the lambda. But since manifold executes as part of the compiler, it could leverage javac's built-in functionality for that -- inlining would happen in the compiler's generation phase.

I'm sure there are other pitfalls, these are just off the top of my head. But nothing strikes me as a show stopper, not yet anyway.

Honestly though, I'm not sure I would have time to give this project the treatment it deserves. We'll see.

Cheers!

terminalsin commented 3 months ago

Interesting idea.

Initial thoughts...

The inline project appears to be a post compilation step to swap out the ASM block call with the bytecode in the ASM block. Interesting. Whereas the jasmine project is basically a mnemonic/symbolic bytecode parser to enable decompilation to the symbolic format to edited then reparsed and rewritten to the original class file.

If I understand correctly, your proposal is essentially to combine these concepts to enable inline symbolic bytecode, but without an extra build step, meaning you want it directly integrated into the compile via manifold.

Being an inline feature, the bytecode would be limited to locals and control instructions, no declarations such as classes, methods, fields.

First off, a grammar and parser would be needed for the symbolic bytecode language. I assume the jasmine project has one that could be borrowed. I would lean more toward writing an antlr grammar targeting the subset for inline bytecode, or just writing a custom recursive descent parser. Probably the latter.

The parser would probably transform to ASM calls.

One obstacle would be stack map frames. The frame in the enclosing bytecode would have to be rewritten if any locals are introduced in the inline.

Another tricky part involves local variable references. Since bytecode accesses them as offsets in the stack, the parser would have to calculate these offsets independently. This step could be avoided if not for the parser having to verify local references during the symbol attribution phase as opposed to the generation phase.

The difficulty with lambdas/invokedynamic is generating the synthetic method corresponding with the lambda. But since manifold executes as part of the compiler, it could leverage javac's built-in functionality for that -- inlining would happen in the compiler's generation phase.

I'm sure there are other pitfalls, these are just off the top of my head. But nothing strikes me as a show stopper, not yet anyway.

Honestly though, I'm not sure I would have time to give this project the treatment it deserves. We'll see.

Cheers!

Love this reply, all very good observations. After looking at JASM some more, with some tweaks to its internals the parsing at the code level can be done directly. All that would need to be accounted for are variable declarations in previous scope, a task I'd find trivial. I'll link this to the developer whom I know personally to see if he can introduce some api changes to account for this.

If the above is feasible, all that would need to be created is a translation layer into Jasm AST, then out with the bytecode. In regards to frame computation, these are, as far as I know, implicitly handled by ASM given a previous frame. Should be fine? We could generate standard bytecode initially then wrap it in ASM and have it recompute frames after the final result is generated... that would save us the burden of any complicated algorithm going in and out of assembly form.

As for lambdas/invokedynamics, the synthetic method's burden would fall upon the user to create, as the instruction itself is merely just a call to a method returning a callsite.

Fun stuff to discuss and look into :)