qtjs with specific version of Node.js

cfsghost commented 10 years ago

I am now working on an open source JavaScript OS with Node.js for embedded system and implementing a lot of low level APIs to access hardware and system, it make JavaScript developer be able to do embedded system works.

qtjs is doing well to integrate Qt/QML into Node.js, it can be good solution for UI rendering on JavaScript OS.

In fact, qtjs works well, but I found a problem that it doesn't work with stable version Node.js something's like v0.10.24.

My suggestion is, if it's possible to compile qtjs with specific version of Node.js, it will be useful.

svalaskevicius commented 10 years ago

Hi @cfsghost,

Thanks for your interest!

As for the node v0.10 - I've upgraded it to 0.11.x and I believe by the time qtjs becomes stable node 0.11 will be stable as well :)

IIRC, the only incompatibility is the new version of V8 engine upgraded in Node 0.11 - please see these commits, which were for upgrading the supported V8 version: https://github.com/svalaskevicius/qtjs-generator/compare/321f4d2988771a231ec05547d554f20a03a8ad67...d5f76f76ab394831c2067d22684a153240ce8631

I believe reverting those should revert the support for the older V8 again, but I haven't tested it after upgrading.

Is it a hard requirement to use Node v0.10 and not 0.11? As far as I've tried, most of the npm installable dependencies are working OOTB (except with contextify, but it also has a fix available).

Regards

cfsghost commented 10 years ago

:-)

Actually, v0.11 is unstable version, APIs in v0.11 will be modified at anytime if Node.js development team wants to.

Next stable version is v0.12. Before it comes, most developers are using v0.10 for production.

Cheers

svalaskevicius commented 10 years ago

true :) but I hope they wouldn't downgrade the V8 version in v0.12..

Also, I am not sure how true is this, but it looks like v0.12 should be ready quite soon? And basing new functionality on an older V8 will require fixing it later to update.

As for reverting back to v0.10 - its definitely possible and relatively easily doable - I reckon it should take ~0.5-1d to revert those (possibly a few more) commits and recompile..

I guess it all depends on when do you want to put your OS to production vs a testing environment :)

cfsghost commented 10 years ago

Not sure when is v0.12 released, I hope it comes soon. :-S

I will keep working on stable version with less trouble. so If v0.12 was released, I will try to put all my work to the new base. That's why I hope qtjs can provide a easy way to specify Node.js version, it's helpful.

BTW, hope I can done OS and release it this month. But one thing I have to do is to port qtjs to ARM platform and test it. If you can help me to make it be able to support v0.10, I won't need to do it by myself. :-D

svalaskevicius commented 10 years ago

Ok, I'll add v0.10 support.

Btw, qtjs is still building to quite huge binaries - is this ok for your embedded OS? (the size is mainly due to cpgf's variant templates and is a non-trivial task to reduce.)

cfsghost commented 10 years ago

I think footprint is not big problem, but still need to try. :-)

svalaskevicius commented 10 years ago

Hi @cfsghost,

I've created a branch to support node v0.10 - please try it out - backport/node_v0.10 (you will also need to update submodules and rebuild v8, cpgf, or do a clean checkout and repeat the build procedure as in README.md).

The diff can be seen at https://github.com/svalaskevicius/qtjs-generator/compare/backport;node_v0.10

As for testing - I've only tried the provided examples (in the examples dir) and they're working ok.

Please keep me updated about the progress on your OS ;)

cfsghost commented 10 years ago

Cool. I build it successfully on my x86 PC. After testing with some native Node.js module, there is a problem I found. Needs your help.

The following is a native node.js module to access DBus IPC protocol via libdbus, but it doesn't work with qtjs: https://github.com/cfsghost/node-dbus

node-dbus is written by me. I integrate uv event loop into it to handle DBus messages. But the problem is, node-dbus cannot found uv symbol:

fred@Fred-Laptop:~/opensource-work/node-dbus/examples$ /home/fred/sources/qtjs-generator/tools/qtjs hello
/home/fred/sources/qtjs-generator/tools/../build/src/runner/qtjs: symbol lookup error: /home/fred/opensource-work/node-dbus/build/Release/dbus.node: undefined symbol: uv_default_loop

On my OS, a lot of module are using node-dbus to access system APIs. I have to make sure it works with qtjs.

svalaskevicius commented 10 years ago

Hi,

From a quick glance it looks like it might be related to not exporting symbols..

My first test would be adding these in runner.pri:

QMAKE_CXXFLAGS_RELEASE -= -fvisibility=hidden -fvisibility-inlines-hidden -s
QMAKE_LFLAGS_RELEASE -= -fvisibility=hidden -fvisibility-inlines-hidden -s

However, I'll check this a bit more tomorrow evening (its now 1am for me) :)

Regards,

svalaskevicius commented 10 years ago

Hi @cfsghost ,

I've tried to npm install cfsghost/node-dbus - it failed w/o telling what's the problem (which I assume is a problem in install scripts).

However if I clone your repository manually and run node-gyp configure build it works fine and I could run the provided examples/hello.js.

I still think the problem you've mentioned relates to the default exports visibility - and have updated the code 4f215cd...9013366. (btw, which compiler are you using? as currently I am testing on clang 3.4)

Please can you update the same branch and try again?

Thanks.

cfsghost commented 10 years ago

Still got the same problem with latest update.

I can see -fvisibility=hidden -fvisibility-inlines-hidden -s in compilation messages. It's no use modifying runner.pri.

BTW, I am using GCC 4.8 on Debian.

svalaskevicius commented 10 years ago

can you execute qmake command again? as it will prepare a different Makefile based on runner.pri.

cfsghost commented 10 years ago

I did, but the same problem there. it's weird. :-/

svalaskevicius commented 10 years ago

I can confirm the change in runner.pri is removing the compile flags for me.

can you try deleting the whole build/src/runner/ directory, running qmake and building it again?

If this doesn't help, can you edit src/qtjs.pri and remove these lines:

QMAKE_CXXFLAGS_RELEASE += -fvisibility=hidden -fvisibility-inlines-hidden -s
QMAKE_LFLAGS_RELEASE += -fvisibility=hidden -fvisibility-inlines-hidden -s

After this qmake, clearing build/src/runner/*.o files and rebuilding it again should definitely skip these flags.

svalaskevicius commented 10 years ago

To explain - these flags are only required to reduce the built bindings size by not exporting their symbols, however, qtjs itself can have symbols exported.

If the above works I'll split qtjs.pri and create specific one for building the libs

cfsghost commented 10 years ago

It's no use deleting build/src/runner/ directory, but it works by editing src/qtjs.pri

It's compiling now, i will see the result later.

svalaskevicius commented 10 years ago

:) I'll have to reduce the compile time one day.. currently the focus was to make it working, so I added lots of #include's everywhere.. hopefully the clang based generator will help to identify only the required includes in each file.

cfsghost commented 10 years ago

Compile completely. But still cannot found uv_default_loop when running it. :-S

fred@Fred-Laptop:~/opensource-work/jsdx-connman/node_modules/dbus/examples$ /home/fred/sources/qtjs-generator/tools/node hello.js 
/home/fred/sources/qtjs-generator/tools/../build/src/runner/qtjs: symbol lookup error: /home/fred/opensource-work/jsdx-connman/node_modules/dbus/build/Release/dbus.node: undefined symbol: uv_default_loop

svalaskevicius commented 10 years ago

Please can you check if the symbols are exported in the binary?

e.g.

nm -g /home/fred/sources/qtjs-generator/build/src/runner/qtjs | grep uv_default_loop

should result in:

00000000005038c0 T uv_default_loop

svalaskevicius commented 10 years ago

scratch that. I can reproduce it now as well - although it was working ok yesterday. very interesting - will let you know once I get it working

cfsghost commented 10 years ago

Here is the result:

nm -g /home/fred/sources/qtjs-generator/build/src/runner/qtjs | grep uv_default_loop
00000000004c8210 T uv_default_loop

svalaskevicius commented 10 years ago

interestingly it works with clang if I have this in my flags:

QMAKE_CXXFLAGS += -fsanitize=address
QMAKE_LFLAGS += -fsanitize=address

it doesn't otherwise.. still checking :)

svalaskevicius commented 10 years ago

it looks like adding QMAKE_LFLAGS += -rdynamic helps for clang :)

Please can you pull changes from github, remove build/src/runner/, run qmake, build it again, and see if it works? (rebuilding node-dbus from git clean state :) )

cfsghost commented 10 years ago

Good job! It's working now! :-)

I will start to cross-compile for ARM before going to bed. (I cannot imagine how long it will take ...)

Well, I notice qtjs takes a lot of time(around 7 seconds) to load itself on my computer (i7 CPU). any idea?

As you said, reducing footprint size is a way out. Maybe we can make several individual modules with dynamic loading?

svalaskevicius commented 10 years ago

glad it worked :)

yes.. 7 seconds is what I get too.. I have a few ideas in mind and will need to spend some time investigating them..

current ideas:

split into more dynamic libs, e.g. widgets are the heaviest component and could be really easily disabled (when loading).
allow custom built libs with white-listed classes to bind - a rather easy way to only bind whats need on a per app specific basis. complexity increases depending on how clever the selection should be - should it preselect dependent classes? also I guess its unsuitable for an OS which should have all bindings available by default..
implement functionality based on require or "import" functionality, where js would tell what it needs and c++ would bind it only if its needed. would solve loading speed, but not the size of the libraries
rethink cpgf variant templates - they produce the most bytes in the compiled libs - solves size, but not the loading speed..

re "Maybe we can make several individual modules with dynamic loading?" - its already generating individual modules (as *.so libraries) - core, widgets, qml and gui, just need to implement the dynamic loading :)

svalaskevicius commented 10 years ago

re the compiled binary size, I saw @wqking was interested in it too - http://www.cpgf.org/task/index.php?do=details&task_id=2 :)

cfsghost commented 10 years ago

I think this one will solve most problems:

implement functionality based on require or "import" functionality, where js would tell what it needs and c++ would bind it only if its needed. would solve loading speed, but not the size of the libraries

It can fully support all components, but no need to load all every time. Application developer can just choice what they need to use with "require" or "import".

Actually I wrote a module before, trying to use QML binding in Node.js. With brig module, I can only load a few QML components and libraries for "JUST UI rendering". (BTW, Compare with Qtjs, it's just a toy ... lol) https://github.com/cfsghost/brig

svalaskevicius commented 10 years ago

Thanks! I like this idea the most as well - although I'd like to reduce the redistributable sizes as well, this one solves performance issues while still maintaining flexibility.

I see you've also implemented libuv event dispatcher for qt - I couldn't find one when I needed it :)

cfsghost commented 10 years ago

If it's possible to separate cpgf API from Node.js and V8 initialization? Just keep the basic Node/V8 functionality and qt event loop, to avoid loading speed problem. Then I can now use Qtjs on my OS instead of original Node.js.

Make cpgf and Qt API become a individual module, then we can do the following:

var cpgf = require('cpgf');
var qt = require('qt');

Of course it will be still slow but at least there is no problem without loading them.

svalaskevicius commented 10 years ago

Yes, it's a step towards loading just the required components and could easily be implemented by deferring the loading in CpgfBinder class.

I'll see if I can improve this during the weekend (currently busy hunting down a bug in an unrelated side: qml+qsyntaxhighlighter don't play nicely)

svalaskevicius commented 10 years ago

I've refactored the qmake build scripts separating some options for plugins only.

with Qt HEAD version from git and node v0.11 (qtjs master br) I get a relatively quick loading (<1s) :)

 ~/p/d/q/examples master [12:56]> time node core.js 
"[object QPoint]" "a string" true 123 true 2 3556498 
"NAME CHANGED! [object QString]" 
0.58user 0.05system 0:00.87elapsed 72%CPU (0avgtext+0avgdata 112328maxresident)k
0inputs+0outputs (0major+31274minor)pagefaults 0swaps

(Although I haven't tried in node v0.10 branch)

cfsghost commented 10 years ago

Cool! That's good news!

svalaskevicius commented 10 years ago

Hi @cfsghost!

Just to check how is it going with the OS :)

Also, regarding the slower performance on this branch - apparently cpgf/cpgf#18 has not been merged to this branch - its is available on the master br though.

Regards,

cfsghost commented 10 years ago

Hi,

Cool!!

I want to update status for you but so busy now. :-(

Anyway, for next month an open source event in my country, I will try to show everybody the OS with your work!!!

Another target is, I founded a open platform project - "HanGee" to make any kinds of devices in Taiwan this month(The reason is complicated...). Currently, thousands of people supports me to do it. So I am trying to get contact with manufacturers and branding to support and join us. If it's possible to realize our plan, I will try to put JavaScript OS with your work on it. and make everybody to develop mobile application with pure JavaScript. :-D

https://hangee.hackpad.com/%25E5%25BC%2584%25E4%25B8%2580%25E5%2580%258BHangee#HanGee-Project

Currently, we only have a website in Chinese: http://han-gee.com/

Cheers, Fred

2014-03-23 4:01 GMT+08:00 Sarunas Valaskevicius notifications@github.com:

Hi @cfsghost https://github.com/cfsghost!

Just to check how is it going with the OS :)

Also, regarding the slower performance on this branch - apparently cpgf/cpgf/18 has not been merged to this branch - its is available on the master br though.

Regards,

Reply to this email directly or view it on GitHubhttps://github.com/svalaskevicius/qtjs-generator/issues/9#issuecomment-38362437 .

svalaskevicius commented 10 years ago

wow, awesome news! let me know how it goes in the event (and with HanGee) :)

wqking commented 10 years ago

I checked the project home site. Though I live in China mainland, I really like your wish: 我們仍然期望 HanGee 能促成科技版的『看見台灣』，讓眾人透過 HanGee 計劃看見台灣人才的熱情和原創實力。

Good job.

wqking commented 10 years ago

@cfsghost I hope you can update your project status for the landmarks. I'm really keen to know how QtJS and the underlying cpgf work with a real world great product.

svalaskevicius commented 9 years ago

Hi @cfsghost !

Just wanted to check how is it going? How's the OS.. and are you still using qtjs? :)

I'm currently going through the issues in github and wondering if this one can be closed or can I still help you anyhow?

Regards, Sarunas

svalaskevicius commented 9 years ago

closing the issue as its outdated and there is a new nodejs version released. (and even iojs :) )

svalaskevicius / qtjs-generator

qtjs with specific version of Node.js #9