Web console demo unable to run a .jar file

[riking]

A utility to unzip a jar file that you just "uploaded" so that it could be run would be nice. I don't want to select every file of several subfolders using the file browser - oh wait, they're all saved to root and some class names conflict.

Can you get something for this? :-)

[perimosocordiae]

This is pretty similar to a really old issue: #71. We've made some progress on the jar-in-browser front, but the pieces aren't fully put together yet. If you want to send a pull request, contributions are welcome! :smile:

For anyone looking at how they might do this, check out the jar support in the console frontend (console/runner.coffee). The overall flow will probably look something like that.

[perimosocordiae]

An alternative, easier fix would be supporting whole-directory, path-preserving, file uploads.

[riking]

Yeah, I didn't do any research into the code or history yet.

I'll try that

[jvilk]

Jar file support in the browser is something that I really want at some point.

I'm unsure if the HTML5 FileReader API supports whole-directory uploads. I'll have to look into it (if you find anything, let me know).

I just realized we don't even have a mkdir command exposed in the in-browser terminal... so, even though the "File Upload" feature uploads to the "current directory", you can't create directories! (Unless you code a Java program to make one for you...)

[jvilk]

After a quick search, this seems to exist now: http://gildas-lormeau.github.io/zip.js/

If we combine this with a tmpfs FileProvider in the web browser (to prevent backing everything in LocalStorage), we can get JAR file support.

(I'm willing to do the tmpfs part and mount the temporary filesystem at /tmp if someone does the other part)

[perimosocordiae]

Actually, the library we use for unzipping on the node frontend is based on a different browser-based zip library (which I mentioned in my commit message for 683faeeae79da39f93f3052477d64dd1fea31437).

[jvilk]

Ah! I missed that commit message.

brb making a tmpfs for the browser

[int3]

brb making a tmpfs for the browser

I was actually thinking that the proper way to jar file handling -- regardless of whether we're in the browser or in node -- is to extract class files lazily. We shouldn't have to pay a ~2 second startup cost each time just to unzip the jar.

[jvilk]

Well, yeah. That would actually be ideal. The whole 'extract the entire jar file approach' was a way to get it working quickly using the functionality that is working.

When I get around to fixing the ClassLoader concurrency bug, I'll look into adding support for jar files in the classpath, which the ClassLoader would lazily extract class files from.

[riking]

I'm still trying to build this :p

Current problem is that making tools/preload fails on the tar command saying that....Well then. java.lang.ClassNotFoundException\: ..classes.util.Javac: Those two dots might be a problem.

EDIT: That's wonderful, the error message was pasted into tools/preload.

Sure, I'll pbin the make command.

[perimosocordiae]

Building is a weak area for Doppio at the moment. Care to pastebin the full output of the failing command(s)?

[jvilk]

Indeed, they are! Making tools/preload relies on the Node frontend functioning properly. It looks like it threw a ClassNotFoundException, though.

Try doing a make -j4 test first. Note that you must be using Java 6 (check with java -version). Two of the tests (iirc) are going to fail, but that's normal (PackageTest and... some Clinit test).

[riking]

Yup, I'm defaulting to java 7. Will update-alternatives and retry (http://paste.ubuntu.com/5694460/)

[riking]

Hm, same error...

Deleted all of build/, still failing 4 tests not 2.

[perimosocordiae]

As a side note: we should look into what it would take to support building/testing with Java 7.

[riking]

Well, this seems silly.

Failed output test classes/test/MesaTest
--SNIP-- Basically all thread numbers were 1 lower

[riking]

Nope, build still failing :(

Are you sure you want to regenerate tools/preload? (y/n)
y
Generating list of files to preload in browser... (will take a few seconds)
COPYFILE_DISABLE=true && tar -c -T tools/preload -f build/dev/browser/mini-rt.tar
tar: Exception in thread "main" java.lang.NoClassDefFoundError\: java.lang.ClassNotFoundException\: ..classes.util.Javac: Cannot stat: No such file or directory
tar: Caused by\: java.lang.ClassNotFoundException\: ..classes.util.Javac: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
make: *** [build/dev/browser/mini-rt.tar] Error 2

Well, we know it's because of the two dots.

EDIT: Well hey, tried tools.sh again and it actually failed.

[jvilk]

MesaTest has failed on me previously in a Linux environment, so maybe it's flaky.

Supporting Java 7 = changing javap's output entirely, iirc. :frowning:

riking: What test does tools/preload contain? And does classes/util/Javac.class exist?

[perimosocordiae]

Try running make java, which should ensure the existence of classes/util/Javac.class.

[riking]

Ayup, that's it.

kane@KANEUBUNTU:~/projects/doppio$ make java
javac classes/util/Javap.java
classes/util/Javap.java:30: package sun.tools.javap does not exist
    sun.tools.javap.Main javap = new sun.tools.javap.Main(out);
                   ^
classes/util/Javap.java:30: package sun.tools.javap does not exist
    sun.tools.javap.Main javap = new sun.tools.javap.Main(out);
                                                    ^
2 errors
make: *** [classes/util/Javap.class] Error 1

Hmm.. Did update-alternatives with both sun-java-6 and sun-java-7, neither worked.

[riking]

@jvilk Contents of tools/preload: http://pastebin.com/07r3Ecme Note the exception on the first line.

[perimosocordiae]

Yep, the preload file is generated by listing the classes loaded by running classes/util/Javac.

So it looks like your native Java's class library doesn't match ours. Does your java -version say it's Java 6?

[jvilk]

Weird. Sounds like your native Java's class library is different than we expected! We expect there to be a sun.tools.javap package that contains an interface for running the javap command from within Java.

You sure you're using Java 6? Check javac -version.

[riking]

javac 1.6.0_27

kane@KANEUBUNTU:~/projects/doppio$ sudo update-alternatives --config java
There are 6 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                           Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-7-openjdk-i386/jre/bin/java   1071      auto mode
  1            /usr/bin/gij-4.6                                1046      manual mode
  2            /usr/bin/gij-4.7                                1047      manual mode
* 3            /usr/lib/jvm/java-6-openjdk-i386/jre/bin/java   1061      manual mode
  4            /usr/lib/jvm/java-6-sun/jre/bin/java            63        manual mode
  5            /usr/lib/jvm/java-7-openjdk-i386/jre/bin/java   1071      manual mode
  6            /usr/lib/jvm/java-7-sun/bin/java                0         manual mode

Press enter to keep the current choice[*], or type selection number: 3

Btw, update-alternatives java includes javac & friends.

[perimosocordiae]

You might be able to hack-fix it by compiling with: javac -cp vendor/classes classes/util/Javac.java

[riking]

That somehow made the error message cleaner:

Generating list of files to preload in browser... (will take a few seconds)
COPYFILE_DISABLE=true && tar -c -T tools/preload -f build/dev/browser/mini-rt.tar
tar: Exception in thread "main" java.lang.NoClassDefFoundError\: ./classes/util/Javac (wrong name\: classes/util/Javac): Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
make: *** [build/dev/browser/mini-rt.tar] Error 2

[riking]

!!! Running tools/setup.sh again is working.

[perimosocordiae]

Aside: it might be worth adding the classpath hack (javac -cp vendor/classes) to the Makefile to avoid these issues in the future.

[riking]

Needed to apt-get install ruby-ffi ruby1.9.1-dev coffeescript As well as some other stuff that was easy to figure out earlier.

I was about to suggest gksudo, but not everybody would have that.

Setup finished.

[riking]

....Aaaand same error.

:(

[riking]

By the way, my original motivation for trying this was, frankly, pretty absurd: to run https://github.com/Bukkit/CraftBukkit in the browser. (It's basically Minecraft, except the server.)

[perimosocordiae]

Which, the make java failure, or the downstream failure to generate the preload file? Try running:

node build/release/console/runner.js classes/util/Javac ./classes/test/FileOps.java --list-class-cache > tools/preload

and let us know what it says.

Sadly, Doppio doesn't (and likely won't) support sockets due to JS limitations, so Bukkit may be a non-starter. On the other hand, this is useful information for us! :+1:

[riking]

Yeah, I do believe sockets in JS aren't possible yet. I think I looked into that before.

[riking]

@perimosocordiae That command results in no errors, and cat tools/preload runs off the buffer.

[riking]

Okay, did make dev and said "no".

[jvilk]

Only WebSockets are possible. WebSockets only allow outgoing connections, so running a Minecraft server wouldn't work, unfortunately. Also, WebSockets are not interoperable with regular sockets, so you wouldn't be able to connect a Minecraft client running in Doppio to a server without the server having a special adapter for handling WebSocket connections.

(Not to mention that Doppio doesn't have support for the graphics stuff yet, and isn't fast enough to run Minecraft right now)

[riking]

Hey, nobody said anything about actually letting people connect. That's crazy talk.

[riking]

Mmmm, cpu 0 at 100%.

[riking]

After 3 minutes of CPU time, it has printed out this:

229 recipes
27 achievements
Exception in thread "process reaper" java.lang.Error: Doppio doesn't support forking processes. Command was: `sh -cstty -a < /dev/tty`
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.access$500(UNIXProcess.java:36)
    at java.lang.UNIXProcess$1$1.run(UNIXProcess.java:125)

Well, darn :P

[jvilk]

!!!

CJ, that's the same thing that Jython prints out when it fails (#177). That HAS to be a JCL function testing for some functionality.

Riking, it's possible we'll be able to get past that point eventually if we can figure out what the JCL is testing for...

[riking]

Well, I can tell you for sure CB doesn't intentionally create a thread called "process reaper".

Note: confirmed, "reaper" never appears in repo

[int3]

Fiddling with the classpath now.

[riking]

8:30 and counting, only 3.1% memory

[jvilk]

riking: Oh, cool, you're a CraftBukkit contributor! Do you use processes for anything? And do you know what CraftBukkit code might be triggering the UNIXProcess code?

It's possible it's some static code that runs as part of clinit.

[perimosocordiae]

Try running the release version, if you aren't already.

[riking]

Found it. jline.internal.TerminalLineSettings.java:188

    private String exec(final String... cmd) throws IOException, InterruptedException {
        assert cmd != null;

        ByteArrayOutputStream bout = new ByteArrayOutputStream();

        Log.trace("Running: ", cmd);

        Process p = Runtime.getRuntime().exec(cmd);

        InputStream in = null;

[riking]

Flow:

• public TerminalLineSettings() throws IOException, InterruptedException calls get("-a")
• public String get(final String args) throws IOException, InterruptedException calls stty(args)
• private String stty(final String args) throws IOException, InterruptedException calls exec(String.format("%s %s < /dev/tty", sttyCommand, args))
• private String exec(final String... cmd) throws IOException, InterruptedException calls Runtime.getRuntime().exec(cmd)

By the way, it's still running and hasn't crashed out.

[perimosocordiae]

So it's a JLine thing. Should've known.

[riking]

It's run for 31 CPU minutes, but sadly I'm going to have to kill it in order to sleep :(

[perimosocordiae]

I've disabled JLine as of commit ab79a98f3c1e924e541f5c0aaa17d3b16a672350, so try again and see what happens.