Open cxbrooks opened 8 years ago
This is on hold because we need to use OSGi bundles for the PtII code because we are depending on bundle activation. The Tycho build uses a non-OSGi bundles for the PtII code because Tycho can only access bundles in Software Sites.
http://www.vogella.com/tutorials/EclipseTycho/article.html#tychotargetplatform says:
Tycho only supports Software Sites as locations. Directory, Installation and Feature are not supported and are ignored by the build process.
Now that Erwin has fixed up the Triqiuetrum build using Tycho, we can build RCP images, so supporting CapeCode can proceed.
To get a list of which jar files are used by CapeCode
cd $PTII
make echo_jars JARS=CAPECODE_JNLP_JARS | grep .jar | awk -F \" '{print $2}' | sort | uniq | sed 's@../dists/ptII11.0.devel/jar_dist/@@'
See capecodeJars.txt
The list of packages that are used by the StockTick demo can be found by doing:
export JAVAFLAGS=-verbose
$PTII/bin/vergil $PTII/org/terraswarm/accessor/demo/StockTick/StockTick.xml >& /tmp/classes.txt
grep 'Loaded ' /tmp/classes.txt | awk '{print $2}' | sort | uniq | awk -F . '{ for(i=1;i<NF;i++) printf("%s.", $i); printf("\n")}' | sort | uniq > ~/tmp/packages.txt
Note that Cape Code is not a blocker for the Triquetrum First Release: 21st October 2016 #71
At first, because it would take awhile to get the CQs done, the Cape Code sources will not be checked in to the Eclipse Repo.
One issue here is that the Cape Code configuration consists of a multiple Accessors, where each Accessor is an instance of org.terraswarm.accessor.jjs.JSAccessor, but has different values for the script parameter. It is not clear to me how we would create an Actor bundle that will support this.
http://wiki.eclipse.org/Triquetrum/Extending_Triquetrum#Creating_an_actor_bundle describes how to create an actor bundle where each actor is implemented by a Java class.
However, in Cape Code, we have multiple actors that each implement org.terraswarm.accessor.jjs.JSAccessor.
BTW - When $PTII/bin/vergil -capecode
is run, there is an Accessors node in the actor tree view. When that node is opened, org/terraswarm/accessor/AccessorLibrary.java gets https://www.terraswarm.org/accessors/index.json which contains:
[
"audio",
"cameras",
"devices",
"gdp",
"image",
"net",
"robotics",
"geodesy",
"services",
"signals",
"test",
"utilities"
]
Each of the elements represents a directory, so https://www.terraswarm.org/accessors/audio/index.json is opened, which contains:
[
"AudioCapture.js",
"AudioPlayer.js",
"ClipPlayer.js"
]
https://www.terraswarm.org/accessors/audio/AudioCapture.js is opened and passed to the accessorToMoML() method in org/terraswarm/accessor/JSAccessor.java, which returns MoML that looks like
<entity name="AudioCapture" class="org.terraswarm.accessor.jjs.JSAccessor">
<property name="accessorSource" value="https://www.terraswarm.org/accessors/audio/AudioCapture.js"/>
<input source="file:/Users/cxh/ptII/org/terraswarm/accessor/accessors/web/audio/AudioCapturePtDoc.xml"/>
<property name="script" value="..."
<property name="_tableauFactory" class="ptolemy.vergil.toolbox.TextEditorTableauFactory">
<property name="attributeName" value="script"/>
<property name="syntaxStyle" value="text/javascript"/>
</property>
<property name="script">
<property name="style" class="ptolemy.actor.gui.style.NoteStyle">
<property name="note" value="NOTE: To see the script, invoke Open Actor"/>
</property>
</property>
</entity>
So, what we need to do is modify the bundle activator to do the above access of the index.json files and instantiation.
Christopher, could you send me a concrete example of the capecode configuration of some accessors?
I think what you need is to be able to add multiple entries in the palette definition, based on a same underlying actor Java class, each with some different parameters/properties. And this is already possible in theory in Triquetrum. I.e. we can set properties on actors in the palette. But this has not yet been used/proven and may not be sufficient.
I looked around in ptII/ptolemy/configs/capecode/ but it seems the real accessor configuration info should come from :
<!-- Standard TerraSwarm accessors. -->
<entity name="Accessors" class="org.terraswarm.accessor.AccessorLibrary">
<configure source="https://www.terraswarm.org/accessors"/>
</entity>
and I could not see the XML there that could show how an accessor actor is defined in a capecode configuration.
Or is this done based on what you describe above, i.e. there's some js/json files that get transformed by the JSAccessor actor code itself?
If the last part of your comment means that the palette is not known/defined upfront in a Vergil configuration, but is generated on the fly each time, reading the json/js file on the terraswarm URL, I can see several issues :
Here's a zip file of the configuration. I created it by editing ptolemy/configs/test/configTest.tcl:
bash-3.2$ svn diff ptolemy/configs/test/configTest.tcl
Index: ptolemy/configs/test/configTest.tcl
===================================================================
--- ptolemy/configs/test/configTest.tcl (revision 75383)
+++ ptolemy/configs/test/configTest.tcl (working copy)
@@ -47,7 +47,7 @@
set loader [[$parser getClass] getClassLoader]
-set URL [$loader getResource ptolemy/configs/ptiny/configuration.xml]
+set URL [$loader getResource ptolemy/configs/capecode/configuration.xml]
puts "Checking [$URL toString]"
set object [$parser {parse java.net.URL java.net.URL} $URL $URL]
set configuration [java::cast ptolemy.kernel.CompositeEntity $object]
@@ -57,5 +57,7 @@
set results [[java::cast ptolemy.actor.gui.Configuration $configuration] check]
puts $results
+puts [$configuration exportMoML]
+
# The list of filters is static, so we reset it
java::call ptolemy.moml.MoMLParser setMoMLFilters [java::null]
bash-3.2$
and then running
$PTII/bin/ptjacl configTest.tcl >& ~/tmp/capeCodeConfiguration.xml
and then editing that file and removing some text from the beginning.
Here's the file:
Below are some answers to your questions:
In Vergil, the palette is created when the Accessors tab in the actor palette is opened. Vergil has lazy evaluation of the elements in the actor palette. This means that startup is fast and if there are problems with a set of actors, they don't cause problems until you try to use them. If you search the actors for an actor by name, then each set of actors is opened as the search proceeds.
If you are working offline, then Cape Code looks in $PTII/org/terraswarm/accessor/accessors/web for the accessors. In fact, it looks there first and then looks at the website.
How the AccessorLibrary class works is that it looks for a index.json file that names directories that contain index.json files. These second-level index.json files name .js files that define the accessors. The accessor .js files are read in and then converted to MoML, which is then parsed and added to the configuration. I think we will want the bundle activator to do this parsing for us.
The input element causes the MoMLParser to read input from a file as if the contents of that file were present in the MoML.
Not getting this done by Tuesday is not a problem. I'm on holiday today.
In other news, I added org.terraswarm.accessor to osgi-2-0/. When I run mvn clean verify
, the build fails for the following reasons.
My idea is to get the above to compile and then work on the activator. I'm not going to get much done today.
I have a first hacked-together result, combining the triquetrum branch #189 with a capecode palette provider in https://github.com/erwindl0/triquetrum-capecode, a set of 3rdparty deps like jackson and netty and local hacks in ptII code...
The palette provider adds the BaseA.js test accessor to the Triquetrum palette.
Main purpose is to try to understand what it takes, the code or workspace setup is not OK at all for real usage.
Besides updating some ptII classes (as mentioned previously by Christopher as well), I see following issues that must be addressed, both in ptII Java code and in JavaScript files (at least these are things I am aware of at this moment, we may discover more...) :
Things that are not compatible with OSGi-based modularity :
I copied the files on some fixed local paths and changed the lookup code in some places to point there i.o. using $CLASSPATH-relative paths etc.
Triquetrum-specific issues for CapeCode :
exports.setup = function () { this.input('in1'); console.log('base set up'); }
Should the BaseA accessor get an 'in1' input port? For the moment it only has a 'script' input port and an 'error' output port. Is there some actor lifecycle method to invoke before rendering it in the editor? Cool, thanks for taking a look! This looks promising.
About $CLASSPATH, I spent some time last week getting Cape Code to work when run from just jar files (I think). The code unjars files and directories to a temporary location and then reads files from that location. The temporary files are deleted upon exit. This is a half step, a better solution would be to read the jar files and provide the content of the files without creating a temporary file. We do this for the Duktape-based accessor hosts. (Duktape is a JavaScript interpreter written in C. org/terraswarm/accessor/accessors/adm/hosts/duktape/eduk is a Duktape host that is meant for an embedded platform that has no file system, instead the contents of the .js files are encoded in arrays.)
I can see why $CLASSPATH would be a problem with OSGi. It would be really helpful to be able to support $CLASSPATH. If we can't support it, we need something similar. As a workaround, for CapeCode we could just provide either a directory tree that has .js files or a jar file that has the .js files that gets unjared.
About your BaseA accessor, the input port "in1" should be created.
The way this works is that ptolemy/actor/lib/jjs/JavaScript.java has _createEngineAndEvaluateSetup(), which contains:
// Create proxies for the port.
// Note that additional proxies may need to be created in initialize(),
// if they are created by a setup() function call.
_proxies = new HashMap<NamedObj, PortOrParameterProxy>();
_proxiesByName = new HashMap<String, PortOrParameterProxy>();
for (TypedIOPort port : portList()) {
// Do not convert the script or error ports to a JavaScript variable.
if (port == script.getPort() || port == error) {
continue;
}
PortOrParameterProxy proxy = new PortOrParameterProxy(port);
_proxies.put(port, proxy);
_proxiesByName.put(port.getName(), proxy);
}
I think the comment about initialize might not be accurate.
I created a model with a BaseA JavaScript actor that has your JavaScript code in it. The model is here:
Looking at the xml, the port is present:
<entity name="JavaScript" class="ptolemy.actor.lib.jjs.JavaScript">
<property name="script" class="ptolemy.actor.parameters.PortParameter" value="exports.setup = function () { this.input('in1'); console.log('base se\
t up'); }">
<property name="style" class="ptolemy.actor.gui.style.NoteStyle">
<property name="note" class="ptolemy.kernel.util.StringAttribute" value="NOTE: To see the script, invoke Open Actor">
</property>
</property>
</property>
<property name="_tableauFactory" class="ptolemy.vergil.toolbox.TextEditorTableauFactory">
<property name="attributeName" class="ptolemy.kernel.util.StringAttribute" value="script">
</property>
<property name="syntaxStyle" class="ptolemy.kernel.util.StringAttribute" value="text/javascript">
</property>
</property>
<property name="_location" class="ptolemy.kernel.util.Location" value="{295, 165}">
</property>
<port name="in1" class="ptolemy.actor.TypedIOPort">
<property name="input"/>
<property name="defaultValue" class="ptolemy.data.expr.Parameter">
</property>
</port>
</entity>
When I open the model, stdout shows that setup() is invoked:
bash-3.2$ $PTII/bin/vergil -capecode ~/tmp/BaseA.xml
JavaScript: base set up (Thread[AWT-EventQueue-0,6,main])
JavaScript: base set up (Thread[AWT-EventQueue-0,6,main])
I'm not sure why it is invoked twice, that seems buggy.
I'll take a look at your code and see what changes you had to make. I'm working a short day today and not working much over the weekend, so it might be Monday before I get to this.
Good suggestions!
Something I had forgotten to mention before : I also had to remove the moml parts about NoteStyle and TextEditorTableauFactory, as they're linked to Vergil-specific classes that are not in the Ptolemy II OSGi bundles. A more ambitious approach would be to try to implement these concepts on Eclipse RCP technology, but that's too complex for me...
Removing NoteStyle and TextEditorTableauFactory seems reasonable to me. In Vergil, NoteStyle just modifies the parameter configuration window with a note that says "To see the script, invoke Open Actor". In Vergil, TextEditorTableauFactory brings up an editor for the .js.
I took a look at your code and it looks like you are loading in capeCodeHost.xml and localFunctions.xml with an absolute path that is Windows-dependent which is fine in the near term.
One possible change that is missing is that ./ptolemy.gui/original-src/ptolemy/actor/gui/JNLPUtilities.java should probably be updated to get my change that will copy a directory from a jar file to a temporary location. I'm not sure if this will change things, but it might.
BTW - I'm still working on actor/lib/jjs/NashornAccessoHostApplication.java
That file loads a composite accessor and, if it worked, might be a good quick test.
However, it is non-deterministic, so I suggest avoiding it.
bash-3.2$ (cd $PTII/org/terraswarm/accessor/accessors/web/hosts; java -classpath $PTII ptolemy.actor.lib.jjs.NashornAccessorHostApplication -accessor -timeout 10000 hosts/nashorn/test/testNashornHost.js)
TestAdder: inputLeft: 10 inputRight: 0
TestGain: input 10 gain: 4
TestAdder: inputLeft: 10 inputRight: 40
TestAdder: inputLeft: 10 inputRight: 40
50
bash-3.2$ (cd $PTII/org/terraswarm/accessor/accessors/web/hosts; java -classpath $PTII ptolemy.actor.lib.jjs.NashornAccessorHostApplication -accessor -timeout 10000 hosts/nashorn/test/testNashornHost.js)
TestGain: input 10 gain: 4
TestGain: input 10 gain: 4
TestAdder: inputLeft: 10 inputRight: 40
undefined
bash-3.2$
Indeed, the disclaimer I wrote before is VERY important : these are only some hacks to get a better understanding of the situation ;-)
Op 12/2/2016 om 5:24 PM schreef Christopher Brooks:
Removing NoteStyle and TextEditorTableauFactory seems reasonable to me. In Vergil, NoteStyle just modifies the parameter configuration window with a note that says "To see the script, invoke Open Actor". In Vergil, TextEditorTableauFactory brings up an editor for the .js.
I took a look at your code and it looks like you are loading in capeCodeHost.xml and localFunctions.xml with an absolute path that is Windows-dependent which is fine in the near term.
One possible change that is missing is that ./ptolemy.gui/original-src/ptolemy/actor/gui/JNLPUtilities.java should probably be updated to get my change that will copy a directory from a jar file to a temporary location. I'm not sure if this will change things, but it might.
BTW - I'm still working on actor/lib/jjs/NashornAccessoHostApplication.java
That file loads a composite accessor and, if it worked, might be a good quick test.
However, it is non-deterministic, so I suggest avoiding it.
|bash-3.2$ (cd $PTII/org/terraswarm/accessor/accessors/web/hosts; java -classpath $PTII ptolemy.actor.lib.jjs.NashornAccessorHostApplication -accessor -timeout 10000 hosts/nashorn/test/testNashornHost.js) TestAdder: inputLeft: 10 inputRight: 0 TestGain: input 10 gain: 4 TestAdder: inputLeft: 10 inputRight: 40 TestAdder: inputLeft: 10 inputRight: 40 50 bash-3.2$ (cd $PTII/org/terraswarm/accessor/accessors/web/hosts; java -classpath $PTII ptolemy.actor.lib.jjs.NashornAccessorHostApplication -accessor -timeout 10000 hosts/nashorn/test/testNashornHost.js) TestGain: input 10 gain: 4 TestGain: input 10 gain: 4 TestAdder: inputLeft: 10 inputRight: 40 undefined bash-3.2$ |
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/eclipse/triquetrum/issues/18#issuecomment-264495384, or mute the thread https://github.com/notifications/unsubscribe-auth/AGKZ4cVXB2iYlcyckhn-DSRpcq9rs7-Mks5rEEZNgaJpZM4HJBtd.
I'm somewhat stuck here and could use some help
Triquetrum uses Vert.x, so I would like osgi-2.0/org.terraswarm.accessor to some how require vert.x
I was thinking that osgi-2-0/pom.xml would contain:
<module>org.terraswarm.accessor</module>
and osgi-2-0/org.terraswarm.accessor/META-INF/MANIFEST.MF would contain:
Require-Bundle: ptolemy.moml;bundle-version="11.0.0",
vertx-core;bundle-version="3.3.3"
because https://mvnrepository.com/artifact/io.vertx/vertx-core/3.3.3 includes the following code for maven:
<!-- https://mvnrepository.com/artifact/io.vertx/vertx-core -->
<dependency>
<groupId>io.vertx</groupId>
<artifactId>vertx-core</artifactId>
<version>3.3.3</version>
</dependency>
However, that does not work, probably because Tycho is not looking for the Maven repository or because the Maven file is not correct.
I do have ~/.m2/repository/io/vertx/vertx-core/3.3.3/vertx-core-3.3.3.jar
and the META-INF/MANIFEST.MF of vertx-core-3.3.3.jar contains:
Bundle-SymbolicName: io.vertx.core
...
Bundle-Name: Vert.x Core
Bundle-Version: 3.3.3
Maven-Version: 3.3.3
Created-By: Apache Maven Bundle Plugin
Build-Jdk: 1.8.0_92
Maven-Group-Id: io.vertx
I tried modifying osgi-2-0/org.terraswarm.accessor/META-INF/MANIFEST.MF to
Require-Bundle: ptolemy.moml;bundle-version="11.0.0",
Vert.x Core;bundle-version="3.3.3"
and
Require-Bundle: ptolemy.moml;bundle-version="11.0.0",
io.vertx.core;bundle-version="3.3.3"
but neither one worked, for the latter case, mvn clean verify
resulted in:
[ERROR] Cannot resolve project dependencies:
[ERROR] Software being installed: org.terraswarm.accessor 11.0.0.qualifier
[ERROR] Missing requirement: org.terraswarm.accessor 11.0.0.qualifier requires 'bundle io.vertx.core 3.3.3' but it could not be found
[ERROR]
[ERROR] See http://wiki.eclipse.org/Tycho/Dependency_Resolution_Troubleshooting for help.
[ERROR] Cannot resolve dependencies of MavenProject: org.ptolemy.tycho:org.terraswarm.accessor:11.0.0-\
SNAPSHOT @ /Users/cxh/src/osgi-2-0/org.terraswarm.accessor/.polyglot.build.properties: See log for details -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MavenExecutionException
Really what I need is a p2 repository that has the appropriate file.
https://github.com/glyn/vert.x.osgi has a bundling/ directory that will create vertx jar files that look OK to me.
https://github.com/eclipse/vert.x/issues/708 is interesting.
There is https://projects.eclipse.org/projects/rt.vertx/releases/3.4.0
https://groups.google.com/forum/#!forum/vertx is of interest
https://www.eclipse.org/community/eclipse_newsletter/2016/october/article4.php uses maven, instead of an Eclipse p2 site.
However, shouldn't the Vert.x OSGi files be available somewhere in a p2 site? Presumably they have passed CQ and there are signed jars somewhere?
If not, then I guess I need to add https://github.com/glyn/vert.x.osgi to the osgi-2-0 repo so that we can create the bundles, sign them and put them on the ptolemy p2 site.
Tycho by default only considers p2 sites, but you can set an extra flag "pomDependencies" to allow "standard" maven dependencies in addition to that, as follows :
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho-version}</version>
<configuration>
<pomDependencies>consider</pomDependencies>
<target>
...
Did you try that? Then I would expect that the vert.x core dependency (which is a valid bundle) is correctly found during the build.
By the way, vert.x is not used by Triquetrum normally. I had added it in my trial to support a CapeCode-like setup, as I think it's rather a CapeCode dependency.
But I never considered getting a build working ;-), just a trivial demo from my workspace...
Thanks for the tip about pomDependencies. I used that to get mvn verify
working.
I also created two plugin projects by using File -> New -> Project -> Plug-in Development -> Plug-in from existing JAR archive. I created one for org.json, the other for io.vertx.core.
One downside is that these two projects are missing something, so I can't yet deploy them on the osgi-2-0 p2 server.
I was able to run a trivial example. A word of explanation is in order. The accessor system has what we call accessor hosts. Cape Code is an accessor host that allows combinations of Ptolemy actors and accessors. There is also a Nashorn host that is implemented in Java, but does not use Ptolemy actors, it only uses accessors.
ptolemy/actor/lib/jjs/NashornAccessorHostApplication.java defines a main() method that invokes the Nashorn Accessor host.
A simple test is:
bash-3.2$ (cd $PTII/org/terraswarm/accessor/accessors/web; $PTII/bin/ptinvoke ptolemy.actor.lib.jjs.Na\
shornAccessorHostApplication -timeout 10000 -js hosts/nashorn/test/testNashornHost.js)
commonHost.js: processCommandLineArguments(): Setting timeout to stop after 10000 ms.
Instantiated accessor TestComposite with class test/TestComposite
TestAdder: fire(): inputLeft: 10 inputRight: 40
50
No standalone accessors were instantiated
commonHost.js: processCommandLineArguments(): Maximum time reached. Calling stopAllAccessors().
ptolemy/actor/lib/jjs/modules/process/process.js: called exit()
bash-3.2$
To replicate this in a project that has the osgi-2-0 bundles:
svn co https://repo.eecs.berkeley.edu/svn/projects/eal/ptII/branches/osgi-2-0
Then start up Eclipse, create a new empty workspace and import the projects from the directory created above.
Create a Java Application:
org.terraswarm.accessor
ptolemy.actor.lib.jjs.NashornAccessorHostApplication
-timeout 10000 nashorn/test/testNashornHost.js
/Users/cxh/ptII/org/terraswarm/accessor/accessors/web/hosts
When the application is run, you should see something like:
Instantiated accessor TestComposite with class test/TestComposite
java.lang.ArrayIndexOutOfBoundsException
TestAdder: fire(): inputLeft: 10 inputRight: 40
50
Clearly, we need to fix the ArrayIndexOutOfBoundsException.
We also need to resolve things like how we find files and $CLASSPATH.
BTW - there was some interesting discussion about org.json at http://dev.eclipse.org/mhonarc/lists/incubation/msg00341.html
CapeCode uses org.json. We have a copy of it in the ptII repo. I wanted to provide org.json to Triquetrum, so I built an org.json bundle.
The incubation list at http://dev.eclipse.org/mhonarc/lists/incubation/msg00341.html has details about org.json. Apparently there is a bundle in Orbit somewhere. However, they have not been able to get the org.json author to agree to licensing terms on a more recent version. There was a suggestion of using JSON-P.
In the near term, we can do nothing.
In the short term, we should see about using the org.json bundle in Orbit.
In the longer term, ptII should probably be migrated off of org.json.
The Ptolemy group at Berkeley are developing Cape Code, which uses Accessors as proxies to interact with the IoT.
CapeCode uses Ptolemy II, which has the notion of configurations that presents to the user different sets of actors and other features.
It would be helpful if we could have a CapeCode configuration in Triquetrum. This would probably be a separate product that included a separate palette.
In Ptolemy II, the CapeCode configuration may be found in
ptII/ptolemy/configs/capecode/
.CapeCode uses Nashorn, so Java 1.8 would be required.
Currently, to make this work, we would need to create a set of jar files that contain the Ptolemy classes.
The UI would need to be extended so that we have things like: