Extract XP meta information into an optimization

[thekid]

Currently XP meta information is always emitted for compiled types. This however manifests a runtime dependency on XP Framework Core for all compiled code, something we might not want.

This issue suggests moving this behavior behind a command line flag -O xp, enabled by default in the compiling class loader

//cc @Danon

[thekid]

If XP meta information is omitted, then we need to emit comments and annotations to the generated class instead! With that, there are some issues when trying to keep line numbers intact:

APIdoc

Some types are not expressible via PHP syntax and would need to be embedded in APIdoc comments. However, these would change the line numbering in most cases.

Consider the following input:

1 | function main(array<string> $args): void { ... }

Emitting the types as this APIdoc for PHP 7.0, results in the following:

1 | /**
2 |  * @param  array<string>
3 |  * @return void
4 |  */
5 | function main(array $args) { ... }

The function is now on line 5, which leads to line numbers being off in warnings and backtraces, making it really hard to debug the program!

Attributes

Attributes cannot co-exist on the same line as the elements they're attached to in PHP 7.0. This is OK for classes and methods but will have an impact on parameters.

Consider the following input:

1 | function create(#[Inject] Source $source): Target { ... }

Emitting this for PHP 7.0 would require inserting line breaks:

1 | function main(
2 |   #[Inject]
3 |   Source $source
4 | ): Target { ... }

While the function is now still on the correct line, its body is not, with the same undesirable effect as above!

[thekid]

Instead of declaring static function __init() { ... } and invoking it, we could merge the code into __static(), which will be called by the XP Framework's class loading facilities, but not in regular use.

[thekid]

APIdocs could be emitted in a compact form on the same line as the declaration:

1 | /** Comment | @param array<string> | @return int */ function main(array $args) { ... }

Diff for Compiler

diff --git a/src/main/php/lang/ast/emit/PHP.class.php b/src/main/php/lang/ast/emit/PHP.class.php
index 9eb06b8..6f798aa 100755
--- a/src/main/php/lang/ast/emit/PHP.class.php
+++ b/src/main/php/lang/ast/emit/PHP.class.php
@@ -664,7 +664,20 @@ abstract class PHP extends Emitter {
       DETAIL_ARGUMENTS   => []
     ];

-    $method->comment && $result->out->write("{$method->comment}\n");
+    // Emit API doc comment
+    $tags= '';
+    foreach ($method->signature->parameters as $param) {
+      $tags.= ' | @param '.($param->type ? $param->type->name() : 'var');
+    }
+    $tags.= ' | @return '.($method->signature->returns ? $method->signature->returns->name() : 'var');
+
+    if ($method->comment) {
+      $method->comment= "/** ".str_replace("\n", '¶', trim($method->comment, ' */')).' |'.substr($tags, 2).' */ ';
+    } else {
+      $method->comment= "/**".substr($tags, 2).' */ ';
+    }
+
+    $method->comment && $result->out->write("{$method->comment}");
     $this->emitAnnotations($result, $method->annotations);
     $result->out->write(implode(' ', $method->modifiers).' function '.$method->name);
     $this->emitSignature($result, $method->signature)

Diff for Core

diff --git a/src/main/php/lang/reflect/ClassParser.class.php b/src/main/php/lang/reflect/ClassParser.class.php
index b999800f8..d4782f3fc 100755
--- a/src/main/php/lang/reflect/ClassParser.class.php
+++ b/src/main/php/lang/reflect/ClassParser.class.php
@@ -611,17 +611,17 @@ class ClassParser {
             DETAIL_ARGUMENTS    => [],
             DETAIL_RETURNS      => null,
             DETAIL_THROWS       => [],
-            DETAIL_COMMENT      => trim(preg_replace('/\n\s+\* ?/', "\n", "\n".substr(
+            DETAIL_COMMENT      => trim(preg_replace(['/\n\s+\* ?/', '/¶/'], "\n", "\n".substr(
               $comment, 
               4,                              // "/**\n"
-              strpos($comment, '* @')- 2      // position of first details token
+              max(strpos($comment, ' | @') - 4, strpos($comment, '* @') - 2)
             ))),
             DETAIL_ANNOTATIONS  => $annotations[0],
             DETAIL_TARGET_ANNO  => $annotations[1]
           ];
           $annotations= [0 => [], 1 => []];
           $matches= null;
-          preg_match_all('/@([a-z]+)\s*([^\r\n]+)?/', $comment, $matches, PREG_SET_ORDER);
+          preg_match_all('/@([a-z]+)\s*([^\r\n\|]+)?/', $comment, $matches, PREG_SET_ORDER);
           $comment= '';
           $arg= 0;
           foreach ($matches as $match) {
diff --git a/src/test/php/net/xp_framework/unittest/reflection/ClassDetailsTest.class.php b/src/test/php/net/xp_framework/unittest/reflection/ClassDetailsTest.class.php
index c9bd0252d..77b8b08ed 100755
--- a/src/test/php/net/xp_framework/unittest/reflection/ClassDetailsTest.class.php
+++ b/src/test/php/net/xp_framework/unittest/reflection/ClassDetailsTest.class.php
@@ -172,6 +172,24 @@ class ClassDetailsTest extends \unittest\TestCase {
     $this->assertEquals('[:(function(): int)]', $details[DETAIL_ARGUMENTS][0]);
   }

+  #[Test]
+  public function compact_notation_with_comment() {
+    $details= $this->parseComment('/** Returns something | @return int */');
+    $this->assertEquals(['Returns something', 'int'], [$details[DETAIL_COMMENT], $details[DETAIL_RETURNS]]);
+  }
+
+  #[Test]
+  public function compact_notation_with_multiline_comment() {
+    $details= $this->parseComment('/** Exits¶Sets code 2  | @return int */');
+    $this->assertEquals(["Exits\nSets code 2", 'int'], [$details[DETAIL_COMMENT], $details[DETAIL_RETURNS]]);
+  }
+
+  #[Test]
+  public function compact_notation_with_two_parameters() {
+    $details= $this->parseComment('/** @param string | @param int */');
+    $this->assertEquals(['string', 'int'], $details[DETAIL_ARGUMENTS]);
+  }
+
   #[Test]
   public function throwsList() {
     $details= $this->parseComment('

However, this is only readable for the XP Framework - others are unlikely to adapt this.

[thekid]

Code generation for XP

• [x] Emit static initializer in __static()
• [x] Exclude comments and annotations from output - however, they wouldn't hurt being there either!
• [x] Generate XP meta information (including comments and annotations) in __init()

Code generation for plain PHP 8

• [x] Emit static initializer in __init()
• [x] Do not emit more specific types as API docs (prevents line number problem described above)
• [x] Emit comments, type, property and method annotations.
• [x] Emit parameter annotations on the same line for PHP 8

Code generation for plain PHP 7

• [x] Emit static initializer in __init()
• [x] Do not emit more specific types as API docs (prevents line number problem described above)
• [x] Emit comments, type, property and method annotations.
• [x] Decide on one of the following options:
  • [ ] Do not emit parameter annotations (prevents line number problem described above)
  • [ ] Emit #[@$param: Annotation] on the method which is parseable by XP Framework but will produce a syntax error in PHP 8
  • [ ] Emit /** @Annotation */ besides the parameter using the Doctrine format. However, Doctrine does not support parameter annotations

[thekid]

Emitting comments and annotations is a bit tricky as they're attached to the relevant node. Example:

 8 | class Demo {
 9 |
10 |   /** Entry point */
11 |   #[Annotated]
12 |   public static function main(array<string> $args): void {
13 |     ...
14 |   }
15 | }

This is parsed into a lang.ast.nodes.Method, its line member being set to 12 - so when emitMethod() is reached, we'll already be at line 12.

One solution could be to set the method start line to the comment's line when attaching the comment to have enough room. We'd however need to insert this room if we don't emit the comments - but this could be done by writing equivalent whitespace instead:

Method("main", line: 10) {
  Comment("/** Entry point */", line: 10, end: 10)
  Annnotations([Annotated => null], line: 11, end: 11)
  Signature([$args], void, line: 12, end: 12)
  Body {
    // First node here is on line 13
  }
}