search

orc-lang / orc

Orc programming language implementation
https://orc.csres.utexas.edu/
BSD 3-Clause "New" or "Revised" License
41 stars 3 forks source link

Improve the Orc foreign interfacing support #184

Open arthurp opened 7 years ago

arthurp commented 7 years ago

The current system has two major wart that makes using it to integrate with modern libraries combersome.

  1. It does not support callbacks from non-Orc to Orc code without custom Scala/Java code to support it.
  2. It is impossible to implement a Java interface from within Orc without custom Scala/Java code acting as a shim.

In addition, in the future we would like to be able to integrate with non-Java environments, for instance, C++, gRPC, or JavaScript. So the external interfacing system should be system agnostic.

The goals of this work is to realize the Orc integration vision and extend it:

  1. Orc as a central language connecting other components.
  2. Those components can be written in any number of other languages and running remotely or locally on a number of different platforms and runtimes.
  3. Support Orc code running on different platforms such as JavaScript in the browser without having to totally change how the code imports external code (the locally available standard library would still change on a browser, but the way those libraries are imported shouldn't change too much).

In effect the overall foreign interface support should be independent of the external language and of Orc's runtime environment. Specific, language runtime combinations may have special features or requirements, but the basic concepts should not change.

arthurp commented 7 years ago

P/Invoke and the unmanaged-layer APIs used to communicate with the CLR may provide inspiration on how to support in process foreign calls for Orc. Chris says they are smooth and relatively easy to use.

arthurp commented 7 years ago

Both these features could be improved on the JVM using the Truffle Polyglot engine. This allows calls between all Polyglot languages and with Java. The polyglot languages include: Ruby, LLVM bitcode, and even native code.

See http://graalvm.github.io/graal/truffle/javadoc/index.html?com/oracle/truffle/api/interop/package-summary.html for more information on truffle interop.

arthurp commented 7 years ago

I have had thoughts on this. Java 8 (and earlier to my surprise) supports java.lang.reflect.Proxy which generates (at runtime) an implementation of an interface which calls an InvocationHandler whenever a method on the interface is called. This could be used to implement a generic external service method which builds a Java wrapper around an Orc object which calls the Orc methods when the Java methods are called. This will eliminate the need for Java/Scala shim code.

However, to implement the wrapping interface external method, we need a way for Java to call back into Orc. So we will need to extend the OrcRuntime interface (or maybe the soon to be renamed call Handle interface) with methods to allow Java invoke Orc code. I actually like the idea of putting it in Handle. This has the property that any Orc code called by the Java method will execute in the same groups as the call to the Java method. This means that killing the Java method would also prevent it from calling into Orc again which seems very natural.

It's not totally clear to me what the Java code should be able to call in Orc. But I think having three methods would be a good start:

  1. h.callValue(v: AnyRef, arguments: Array[AnyRef]): Simulate executing the Orc code v(arguments...)
  2. h.getField(v, field: Field): Simulate executing the Orc code v.field. This would be useless unless the result is used somehow.
  3. h.callObjectMethod(v: AnyRef, methodName: Field, arguments: Array[AnyRef]): Simulate executing the Orc code v.methodName(arguments...). This is technically a combination of h.getField and h.callValue, but it will be a really common case.

All of the methods would return a future of some kind that allows the Java code to schedule handling of the result or simply ignore it. The future may need to be able to actually block as well. This would be discouraged, but will be required for things like calling Orc code while holding onto the Swing EDT to prevent synchronization problem in Swing. Note that, in theory, when a thread blocks on one of the returned futures it could actually add itself to the thread pool and help perform the work needed to complete the future.

The Java wrapper instance external method could be implemented independently of the Backend (Token or PorcE), but each backend would still need to implement support for calls into Orc separately. But I don't think it would be that hard (given the version I have already done for the Token interpreter).