Hypervisor

[danxn]

Hello, sorry for a bit stupid question, I just started to learn runtime.js.

My question is - is it difficult to write hypervisor with runtime.js? All what I need is convinient "wrapper" for distributed virtualization system and tool which will allow to create in comparatively easy way RDP or VNC-like protocol.

I know that cloud systems is very popular now on server-side, but my idea is to connect mobile devices like smartphones, tablets and laptops in one personal cloud, or personal computing network, which will allow to run different OSes and applications. The main goal - is to combine computing performance of each device and to make one computer from all this stuff with help of network GUI access to each of them from one GUI terminal.

It is also can be usefull for server-side javascript programming, of course, but first I interested in personal application - when each new (actually - old) device like smartphone or tablet can be connected to this personal cloud and increase its computing performance.

I've experimented a bit with Android LinuxDeploy, VNC and few smartphones and tablets - it looks nice. Each VNC session looks just like additional dashboard and all data can be shared via network disks. But it looks a bit like frankenstain, and may be runtime.js can provide suitable tool for creating this system with all it's components on low level with relatively small efforts?

P.S. I know about current ARM limitations - it's not a critical issue because Intel start to promote it's atom platform and there is a particular number of tablets and smartphones with i386 architecture already available.

[iefserge]

@danxn Hello, thank you for question. Sorry for the missing or incomplete runtime.js documentation, I hope I'll be able to update it soon, as well as upload new website.

Current runtime.js version needs KVM host to run on top of (runtime.js as a guest machine). This limitation is mainly because there is no support (aka drivers) for physical hardware and the system uses virtio as the only way to communicate with the outside world. The only supported architecture is x86_64 (though it should be relatively easy to create i386 port. are there 64 bit smartphones?).

I don't really know how Android LinuxDeploy works. Does it run a VM on Android? If that's the case it might be possible to also run KVM and runtime.js there.

[danxn]

@iefserge Hi, thanks for explanation it makes whole picture more clear.

Current runtime.js version needs KVM host to run on top of (runtime.js as a guest machine). This limitation is mainly because there is no support (aka drivers) for physical hardware and the system uses virtio as the only way to communicate with the outside world.

So, the runtime.js for now is far from turning into hypervisor and works inside KVM hypervisor? Ok, not a problem, anyway possibility to write and run compiled JS for servers is a big advantage. Accounting on it's relative simplicity - it's really cool platform for web server in cloud.

The only supported architecture is x86_64 (though it should be relatively easy to create i386 port. are there 64 bit smartphones?).

Yes, i've made a mistake - Intel Atom SoCs supports x8664 architecture, but not in all series: [Atom(system_onchip)](http://en.wikipedia.org/wiki/Atom%28system_on_chip%29), as here said: "All Atom processors implement the x86 (IA-32) instruction set; however, support for the Intel 64 instruction set was not added until the desktop Diamondville and desktop and mobile Pineview cores. The Atom N2xx and Z5xx series Atom models cannot run x86-64 code.[21] The Centerton server processors will support the Intel 64 instruction set.[7]".

Mostly spreaded Atom SoCs for smartphones should support 64-bit instructions set (it's mainly Z2xx and Z3xx series): quick price search. The choice of tablets is wider: price search results

I don't really know how Android LinuxDeploy works. Does it run a VM on Android? If that's the case it might be possible to also run KVM and runtime.js there.

It's open source tool, you can find it here - LinuxDeploy repo. Here the article which describes in details with screenshoots how it is used (chapter "Going the chroot Root: Linux Deploy"). Linux Deploy uses chroot to run on the same machine where Android runs, they are working together at the same time on the same hardware. Works fine, I've tested it on ARM smartphone and ARM/x86-64(Atom Z3xx) tablets - nodejs and express works just fine, but for ARM node should be compiled from sources. So the Linux KVM with runtime.js should work as well.

I'm not sure, but may be mobile SoCs is a good direction to move runtime.js from virtual host to hardware envieronment? I just don't know what drivers set needed for SoCs, but theoretically it should be more stable than for standalone CPU.

[iefserge]

@danxn

So, the runtime.js for now is far from turning into hypervisor

Yep, the initial idea is to be able to create small, immutable and specialized runtime.js VM images that include application code and only necessary system components. Those images then could be deployed into cloud.

I'm not sure, but may be mobile SoCs is a good direction to move runtime.js from virtual host to hardware envieronment?

It would be very interesting to run it on mobile SoC, if anyone interested wants to hack on this. I'm curious, what's the use of such a runtime.js powered mobile device?

[piranna]

It would be very interesting to run it on mobile SoC, if anyone interested wants to hack on this. I'm curious, what's the use of such a runtime.js powered mobile device?

Raspberry Pi, do i need to say more? :-D

[danxn]

@iefserge

It would be very interesting to run it on mobile SoC, if anyone interested wants to hack on this. I'm curious, what's the use of such a runtime.js powered mobile device?

Well, the performance of contemporary mobile SoCs is pretty close to performance of 5-years-old top server platforms. And more than enough if you need something like web-shop or blog. Especially if these SoCs presented in more than one instance and runs cloud. Accounting on their size and low requirements to cooling and power system this is very intresting alternative for collocation, for example. My goal I've described in first post - I need "modular" PC which will be assembled from "cubicles" like smartphones and tablets (and other stuff), each element will increase whole system performance almost in linear manner and resources distribution will be done on applications level. Pretty cool computer, I think :) I just need one tool (actually - some kind of "glue" or "wrapper") to make it. For now this is set of different modules, asking different tools for it's integration and improvement.

@piranna

Raspberry Pi, do i need to say more? :-D

Yes, sure, and not only Raspberry PI. There are lot of minicomputers based on mobile SoCs - like Intel Edison, pcDuino or Cubiboard. There are also some projects which runs JS on very low-level SoCs, like Arduino (Atmel microcontrollers) for example Espruino and Onion Omega. (Ops, sorry, these two systems uses ARMs, but anyway they are close to Arduino in size :)

So, in short: 1) Robots 2) CNCs/3D Printers 3) Smart house applications 4) Embeded electronics

I.e. Internet of the things. 10MB OS with all V8 functional? Pretty cool for data processing, I think.

[iefserge]

@danxn @piranna Ah, that's very cool. Looks like runtime.js should be able to boot on Intel 64 bit chips (it works on my Intel Desktop PC), but without any access to network. Driver would be necessary to make this work. Wouldn't say it's hard to write one, but would require some pretty low-level JavaScript code :)

[piranna]

Basic keyboard and VGA control is somewhat easy and just enough for a hello world... :-) Don't know how much difficult would be to program a network card, but at least the generic ones availables on QEmu or VmWare should be throroughlty documented.

(And a minimal Node.js compatibility, and we have there NodeOS smashing cloud server everywhere :-P)

[danxn]

@iefserge

Looks like runtime.js should be able to boot on Intel 64 bit chips (it works on my Intel Desktop PC), but without any access to network.

Ok, great.

Take a look at this -

This is Intel Edison (new version of Intel Galileo).

• Its dimensions are 35.5 x 25 x 3.9 mm, with components on both sides.
• The board's main SoC is a 22 nm Intel Atom "Tangier" (Z34XX) that includes two Atom Silvermont cores running at 500 MHz and one Intel Quark core at 100 MHz (for executing RTOS ViperOS).
• The SoC has 1 GB RAM integrated on package.
• There is also:
  • 4 GB eMMC flash on board,
  • Wi-Fi,
  • Bluetooth 4
  • and USB controllers.
• The board has 70-pin dense connector (Hirose DF40) with USB, SD, UARTs, GPIOs.
• The price of the device is around 50 USD.[13]
• It runs Yocto Linux with development support for:
  • Arduino IDE,
  • Eclipse (C, C++, Python),
  • Intel XDK (NodeJS, HTML5),
  • and Wolfram.[14][15]

http://en.wikipedia.org/wiki/Intel_Edison

Here /proc/cpuinfo can be found.

Looks like this is good platform to teach runtime.js "to swim" in hardware :)

How do you think? Looks like it needs only network driver to use WiFi, and it will be possible to run "half-metal node server" and to test it's performance on 500MHz core and compare it with similar software instance (Edison supports node.js on top of the Yokto Linux).

Driver would be necessary to make this work. Wouldn't say it's hard to write one, but would require some pretty low-level JavaScript code :)

Well, I never wrote drivers for network interface cards, but everything once should be done for the first time :) May be you can give some links for quick intro to this?

[danxn]

@piranna

and we have there NodeOS smashing cloud server everywhere

Do you mean this is all what is needed to setup cloud platform for nodejs/runtime.js apps? Wow, it's like a bomb. From my point of view, of course.

[piranna]

@piranna

and we have there NodeOS smashing cloud server everywhere

Do you mean this all what is needed to setup cloud platform for nodejs apps? Wow, it's like a bomb. From my point of view, of course.

Network card (I/O) and maybe a hard disk for data storage, since you already have CPU and RAM. Do you really think is it needed anymore for an OS running miles away on a data center? And by using a networked data storage like Amazon S3 or Dropbox you only need the network card support :-) How do they do to connect PICs and Arduinos to internet? What's the Internet of Things more than a lot of dumb devices without any local interface working as micro web servers? :-D

[piranna]

Ok, maybe Node.js support would need more work (mostly API compatibility things), but it's already Javascript and v8 and the only special thing is libuv, so... :-) El 10/06/2015 00:05, "piranna@gmail.com" piranna@gmail.com escribió:

@piranna

and we have there NodeOS smashing cloud server everywhere

Do you mean this all what is needed to setup cloud platform for nodejs apps? Wow, it's like a bomb. From my point of view, of course.

Network card (I/O) and maybe a hard disk for data storage, since you already have CPU and RAM. Do you really think is it needed anymore for an OS running miles away on a data center? And by using a networked data storage like Amazon S3 or Dropbox you only need the network card support :-) How do they do to connect PICs and Arduinos to internet? What's the Internet of Things more than a lot of dumb devices without any local interface working as micro web servers? :-D

[danxn]

@piranna

What's the Internet of Things more than a lot of dumb devices without any local interface working as micro web servers? :-D

Yes, definitely.

[iefserge]

@danxn Here is an example of (unfinished) RTL8139 driver https://github.com/runtimejs/runtime/blob/master/initrd/driver/rtl8139.js. Device is controlled by reading/writing memory mapped registers exposed through ArrayBuffer in JavaScript.

lspci on Linux could be used to find out which network controller (vendorId and deviceId codes) the board has. Then it should be pretty easy to find a device spec online.

btw, runtime.js is able to scan PCI devices as well, https://github.com/runtimejs/runtime/blob/libos/js/core/pci/scan.js#L815-L818. Could console.log that output as well :)

[piranna]

Awesome :-)

[danxn]

@iefserge

Then it should be pretty easy to find a device spec online.

Well, this is the problem for Edison, because of BCM43340 (wifi/bluetooth module). It's proprietary part of Edison platform and should be used with Broadcom firmware:

https://github.com/instantinfrastructure/edison-src/tree/master/broadcom_cws/wlan License: https://github.com/instantinfrastructure/edison-src/blob/master/broadcom_cws/wlan/firmware/LICENCE.broadcom_bcm43xx

Is it possible to use this kind of drivers with runtime.js?

And one more question - as I can see the size (in lines, files and folders) of existing BCM43xx driver is much larger than rtl8139.js. Is it possible to make runtime.js driver for bcm43340 a bit smaler? :) Or it should be probably larger than runtime.js repo?

[piranna]

Speaking in Node.js terms, the firmware could be added some Javascript bindings and require() it as a compiled module... Don't know how much difficult it could be on runtime.js, though...

[piranna]

Anyway, before wireless, I would write an ethernet network driver, they are easier since you don't need to take in account rf.

[danxn]

@piranna

Speaking in Node.js terms, the firmware could be added some Javascript bindings and require() it as a compiled module... Don't know how much difficult it could be on runtime.js, though...

Well, yes, I think it's not a big problem from programming side, but may be some license limitations or something else exists for runtime.js

[piranna]

Quick read I don't find any problem there, re-minds me to MIT license but not sure. Anyway, we don't need to copy the code, just download and compile it on install time... :-) I find it more problematic about how to use it: is it a library, or a kernel module?

[danxn]

@piranna

Anyway, before wireless, I would write an ethernet network driver, they are easier since you don't need to take in account rf.

This is very easy if you use "more opened" modules, like ESP8266, for example: https://gist.github.com/teos0009/2f61b6b031a6eb35c48b

This code even shorter than rtl8139.js But in case with BCM43340 - this looks like building OS from scratch :)

@piranna

Quick read I don't find any problem there, re-minds me to MIT license but not sure. Anyway, we don't need to copy the code, just download and compile it on install time... :-) I find it more problematic about how to use it: is it a library, or a kernel module?

Good question. May be this will explain something: https://github.com/instantinfrastructure/edison-src/tree/master/device-software/meta-edison/recipes-kernel/bcm43340

[danxn]

May be usage of ESP8266 is better solution for now. But it's speed less than needed for cloud, but as sample of using in low-level automation - very usefull.

[danxn]

One more usefull repo: https://github.com/01org/edison-bcm43340

[danxn]

@iefserge

@danxn

Is it possible to make runtime.js driver for bcm43340 a bit smaler? :)

Ok, I see it will depend on common device controlling schema complicity (i've compared rtl8139.js with this RTL8139 driver - http://lxr.free-electrons.com/source/drivers/net/ethernet/realtek/8139too.c)

[danxn]

Holy...

PCI Local Bus Specification

Ok, I'll disappear from thread for a while :)

[piranna]

PCI Local Bus Specification

Cool! :-)

[iefserge]

@danxn runtime.js handles all the PCI stuff and wraps memory/IRQ resources into JS objects. Driver code just needs to ask for a specific device using vendor/device codes https://github.com/runtimejs/runtime/blob/libos/js/driver/virtio/index.js#L36

[danxn]

@iefserge

runtime.js handles all the PCI stuff and wraps memory/IRQ resources into JS objects.

phew.. ok XD

Driver code just needs to ask for a specific device using vendor/device codes

I've found this list in BCM43340 driver repo - https://github.com/01org/edison-bcm43340/blob/master/common/include/devctrl_if/wlioctl_defs.h

Looks like this is what is needed. So, I have to read the rest of this driver and understand how it is controlled from C code and then make the same on runtime.js. This will take some time.

[danxn]

One more problem with Edison - it's unclear what exactly device is BCM43340:

root@EdisonArduino:~# lspci 00:00.0 Host bridge: Intel Corporation Device 1170 (rev 01) 00:01.0 SD Host controller: Intel Corporation Device 1190 (rev 01) 00:01.2 SD Host controller: Intel Corporation Device 1190 (rev 01) 00:01.3 SD Host controller: Intel Corporation Device 1190 (rev 01) 00:02.0 Display controller: Intel Corporation Device 1182 (rev 01) 00:04.0 Serial controller: Intel Corporation Device 1191 (rev 01) 00:04.1 Serial controller: Intel Corporation Device 1191 (rev 01) 00:04.2 Serial controller: Intel Corporation Device 1191 (rev 01) 00:04.3 Serial controller: Intel Corporation Device 1191 (rev 01) 00:05.0 Serial controller: Intel Corporation Device 1192 (rev 01) 00:06.0 System peripheral: Intel Corporation Device 1193 (rev 01) 00:06.1 System peripheral: Intel Corporation Device 1193 (rev 01) 00:07.0 System peripheral: Intel Corporation Device 1194 (rev 01) 00:07.1 System peripheral: Intel Corporation Device 1194 (rev 01) 00:07.2 System peripheral: Intel Corporation Device 1194 (rev 01) 00:08.0 Communication controller: Intel Corporation Device 1195 (rev 01) 00:08.1 Communication controller: Intel Corporation Device 1195 (rev 01) 00:08.2 Communication controller: Intel Corporation Device 1195 (rev 01) 00:08.3 Communication controller: Intel Corporation Device 1195 (rev 01) 00:09.0 Communication controller: Intel Corporation Device 1196 (rev 01) 00:09.1 Communication controller: Intel Corporation Device 1196 (rev 01) 00:09.2 Communication controller: Intel Corporation Device 1196 (rev 01) 00:0a.0 Communication controller: Intel Corporation Device 1197 (rev 01) 00:0b.0 Encryption controller: Intel Corporation Device 1198 (rev 01) 00:0c.0 System peripheral: Intel Corporation Device 1199 (rev 01) 00:0d.0 Multimedia audio controller: Intel Corporation Device 119a (rev 01) 00:0e.0 System peripheral: Intel Corporation Device 119b (rev 01) 00:11.0 USB controller: Intel Corporation Device 119e (rev 01) 00:12.0 Signal processing controller: Intel Corporation Device 119f (rev 01) 00:13.0 Co-processor: Intel Corporation Device 11a0 (rev 01) 00:14.0 Co-processor: Intel Corporation Device 11a1 (rev 01) 00:15.0 System peripheral: Intel Corporation Device 11a2 (rev 01) 00:16.0 Co-processor: Intel Corporation Device 11a3 (rev 01) 00:16.1 Co-processor: Intel Corporation Device 11a4 (rev 01) 00:17.0 System peripheral: Intel Corporation Device 11a5 (rev 01) 00:18.0 Display controller: Intel Corporation Device 11a6 (rev 01)

But this is not critical.

[danxn]

BTW This is known issue for Intel Edison:

https://communities.intel.com/message/269591

BCM43340 Programmer's Guide

Kudzu Dec 21, 2014 9:16 AM

Unfortunately BC has a long history of not publicly providing SDKs or programmers docs much to the frustration of the Linux, WRT, etc community. It appears BC has opened somewhat with some docs on their site for some chipsets but the BC43340 is nowhere to be found. This seems an odd choice for an open source platform.

Short of reverse engineering the driver in the Yocto build, how can on obtain the software interface docs for the 43340? We are planning to port Cosmos (C# Open Source Managed Operating System) (http://www.goCosmos.org) to the Edison and will need to talk directly to the wifi chipset.

[iefserge]

@danxn Ah, couldn't find any specs for BCM43340 either, and its driver code looks very complex. My guess is that it would extremely hard to support it.

[danxn]

@iefserge

My guess is that it would extremely hard to support it.

Yeah, I've tired to browse available driver sources for this chipset (BCM43340). Ok, nevermind. May be when runtime.js will be ported to ARM it will be easy to find suitable network adapter.

Anyway you got the idea: with help of SoCs it is possible to create hardware platform for appropriate software. Like Raspberry PI, but specialized for your needs. This is not very expensive now, and can be done quick enough. For example you can use Raspberry PI project experience. You see - how many analogs appears today? Manufacturers of this devices can place any SoC you need to the board (ARM or Intel Atom for example, or something else). And you can support your own hardware platform for "micro-cloud modules". This is very intresting thing :)

In other words - you can create hardware platform for your OS, support only this set of hardware and provide to user hardware and software as one product - "JS Cloud Card" :) Especially if this boards (actually - cards, accounting on their actual size) can be quickly connected to the cloud. It can be very interesting thing.

[danxn]

@iefserge Guess it's close to you - IKT, you can ask them for prices. From my experience prototype will cost about 1000-3000$, plus small series something around 5000-10000$

Or you may try it on kickstarter, but anyway you need working prototype for crowdsourcing company.

[danxn]

Or you can try something like barebone-nettop like Intel NUC.