RSA, Polyglot and Descriptor Generation

[PengZheng]

For our RSA to be more successful, we should generate descriptor from the more readable header file automatically and make RSA available to other languaes like javascript and Java. Of course these should belong to another extended discussion.

_Originally posted by @PengZheng in https://github.com/apache/celix/pull/544#discussion_r1187123749_

Indeed the future or RSA, polyglot and descriptor gen is more a future discussion. Note that I did some - small - experimentation with libclang to parse remote service header and generate descriptors. So I think this is feasible, but requires effort. And then maybe use something like python pycparser.

_Originally posted by @pnoltes in https://github.com/apache/celix/pull/544#discussion_r1187710575_

[PengZheng]

The aim of descriptor generation for remote services is to make service header file the single source of truth. To this end, we need to be able to annotate service function argument (am=pre/am=out).

I tried the following prototype and believe it could work:

//calculator_service.h
#ifndef CALCULATOR_SERVICE_H_
#define CALCULATOR_SERVICE_H_

#define CALCULATOR_SERVICE              "org.apache.celix.calc.api.Calculator"
#define CALCULATOR_SERVICE_VERSION      "1.3.0"
#define CALCULATOR_CONFIGURATION_TYPE   "org.amdatu.remote.admin.http, celix.remote.admin.shm"

/*
 * The calculator service definition corresponds to the following Java interface:
 *
 * interface Calculator {
 *      double add(double a, double b);
 *      double sub(double a, double b);
 *      double sqrt(double a);
 * }
 */
typedef struct [[clang::annotate("celix::services", CALCULATOR_SERVICE, CALCULATOR_SERVICE_VERSION)]] calculator_service {
    void *handle;
    int (*add)(void *handle, double a, double b, double *[[clang::annotate_type("celix::pre")]] result);
    int (*sub)(void *handle, double a, double b, double *[[clang::annotate_type("celix::pre")]] result);
    int (*sqrt)(void *handle, double a, double * [[clang::annotate_type("celix::pre")]] result);
} calculator_service_t;
#endif /* CALCULATOR_SERVICE_H_ */

// calculator_service.c
#include "calculator_service.h"

Note that calculator_service.c is only used for descriptor generation and thus is deliberately left empty.

$ clang-15 -Xclang -ast-dump -fsyntax-only -std=c2x calculator_service.c
TranslationUnitDecl 0x559464320ec8 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x5594643216f0 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x559464321490 '__int128'
|-TypedefDecl 0x559464321760 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x5594643214b0 'unsigned __int128'
|-TypedefDecl 0x559464321a68 <<invalid sloc>> <invalid sloc> implicit __NSConstantString 'struct __NSConstantString_tag'
| `-RecordType 0x559464321840 'struct __NSConstantString_tag'
|   `-Record 0x5594643217b8 '__NSConstantString_tag'
|-TypedefDecl 0x559464321b00 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x559464321ac0 'char *'
|   `-BuiltinType 0x559464320f70 'char'
|-TypedefDecl 0x559464321df8 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list 'struct __va_list_tag[1]'
| `-ConstantArrayType 0x559464321da0 'struct __va_list_tag[1]' 1
|   `-RecordType 0x559464321be0 'struct __va_list_tag'
|     `-Record 0x559464321b58 '__va_list_tag'
|-RecordDecl 0x55946437c3a8 <./calculator_service.h:36:9, line:41:1> line:36:103 struct calculator_service definition
| |-AnnotateAttr 0x55946437c450 <col:18, col:99> "celix::services"
| | |-ConstantExpr 0x55946437c500 <line:23:41> 'char (*)[37]'
| | | |-value: LValue <todo>
| | | `-ImplicitCastExpr 0x55946437c4e8 <col:41> 'char (*)[37]' <ArrayToPointerDecay>
| | |   `-StringLiteral 0x55946437c308 <col:41> 'char[37]' lvalue "org.apache.celix.calc.api.Calculator"
| | `-ConstantExpr 0x55946437c5a0 <line:24:41> 'char (*)[6]'
| |   |-value: LValue <todo>
| |   `-ImplicitCastExpr 0x55946437c588 <col:41> 'char (*)[6]' <ArrayToPointerDecay>
| |     `-StringLiteral 0x55946437c388 <col:41> 'char[6]' lvalue "1.3.0"
| |-FieldDecl 0x55946437c648 <line:37:5, col:11> col:11 handle 'void *'
| |-FieldDecl 0x55946437cb50 <line:38:5, col:103> col:11 add 'int (*)(void *, double, double, double * [[clang::annotate_type(...)]])'
| |-FieldDecl 0x55946437ce90 <line:39:5, col:103> col:11 sub 'int (*)(void *, double, double, double * [[clang::annotate_type(...)]])'
| `-FieldDecl 0x55946437d258 <line:40:5, col:95> col:11 sqrt 'int (*)(void *, double, double * [[clang::annotate_type(...)]])'
`-TypedefDecl 0x559464383368 <line:36:1, line:41:3> col:3 calculator_service_t 'struct calculator_service':'struct calculator_service'
  `-ElaboratedType 0x559464383310 'struct calculator_service' sugar
    `-RecordType 0x55946437c430 'struct calculator_service'
      `-Record 0x55946437c3a8 'calculator_service'

[[clang::annotate("celix::services")]] serves two purposes:

1. Help the generator locate the root AST we are interested.
2. Pass service name and version to descriptor generator.

We need clang::annotate_type to support per-argument annotation, which is not possible using clang::annotate.

clang::annotate_type is a relatively new attribute supported by clang, which is ignored by older version of clang (like clang-14). To use it, we need either -std=c++11 or -std=c2x. Here, I prefer to treat the service header as a C header rather than C++.

Another downside of using clang::annotate_type is that it is not exposed by libclang, we need libTooling to finish our job. Rather than pip install libclang and let CMake call a python script, we need to:

1. Develop the generator in C++ based on libTooling.
2. Build the generator first before using it to generate descriptors.
3. Support cross compilation. In conan's terminology, the generator should be built in build context rather than host context.

I'm still in prototyping phase. All comments are welcome. @pnoltes @xuzhenbao