General discussion

[dukzcry]

Hi, @mordak.

Thanks for going back to work on your project and taking my changes! There's one more critical fix now: https://github.com/repos-holder/bb10-native-tools/commit/c33561238cbbb24bdf85033338cf8582259ff93f Though it should be investigated whether all these flags are needed. My assumption is that just --with-arch=armv7-a is needed to get all required builtins compiled in. Without this you may encounter "undefined reference to sync*" linking errors when compiling some big C++ projects on BB10 device. And while usage of some atomic primitives can be disabled in gcc's bits/c++config.h of native tools package, it wouldn't help in case of e.g. sync_synchronize() call which protobuf package does on QNX platform. Here's where rebuilding of GCC with above flags (taken from Momentics BB10 gcc info) helped.

Regarding OpenSSH. I think it's not strictly necessary to provide it. BGShellPlus already provides ssh client, and there's ssh server in a BB10! While last can't be started by user right out of the box there's a way to do this either by writing of a small LD_PRELOAD wrapper for getpwuid() or by using of https://github.com/repos-holder/bb10-fakeroot which is anyway required for pkgsrc usage.

[mordak]

Hey!

You have no idea how happy I was to see your fork and all the patches you did - especially getting g++ to work. I pulled your whole repo and then merged everything in. Fantastic!

I'll pull in the diff you linked to there - makes sense to me. Thanks for giving me the heads up!

For OpenSSH, I knew about the ssh binary in BGShellPlus, but wanted to build it myself. Pulling down a different app just to extract the ssh binary seemed unnecessary, and this way I can keep up with the latest version if I want. I was really just after ssh and scp. I agree it isn't strictly necessary, but appears to be of interest, based on feedback I've gotten on Term48. I'd be happy to modify the build to make sshd work out of the box (I already modified getpwuid to get ssh to work), but am equally happy to do the work to exclude it from the build and fall back to the bundled version. I think there's value in having a build for it, even if it's not on by default.

I saw your stuff on pkgsrc - neat! I had tried to get pkgsrc up and running on the playbook ages ago and gave up in frustration. It would be nice to have a proper package manager in the terminal. I keep meaning to dig into this more, but have limited time..

Will have some cycles to put into this over the weekend. Cheers!

[dukzcry]

Thanks for the great work :+1: Without your https://github.com/mordak/playbook-dev-tools I doubt I'd ever try to go any further. I'll update the link on a forum to point to your repo instead of mine, since it's ahead of it (plus you did more things in a right way, like /usr/lib packing) and will do direct pull requests from now in case if there will be any urgent fixes needed :) You've seen that I broke some things in favour of faster testing. You've eliminated most of these nasty things, but there are some left. Here's https://github.com/mordak/playbook-dev-tools/blob/master/lib.sh in function package_build() 'make clean' is commented. I've also disabled building of Ruby stuff :(

So you wrote your own terminal emulator for Blackberry? Cool! I'm still using BGSP as I don't have a Passport and still on 10.2, so never really looked for other options... Understood about ssh, sorry for the noise then. If you would like to provide sshd, getpwnam() should be faked too. Here's LD_PRELOAD sample: https://github.com/repos-holder/bb10-fakeroot/commit/b1b5ba48717146b1ce07375812ebe7b5cf4c69e7

Yes, pkgsrc is fun. Unfortunately I don't have a time to create and maintain a BB10 pkgsrc tree and send patches upstream. I hope @djbclark and possibly other people will take over this.

[mordak]

Merged in your change today and pushed it up. Thanks again!

I am very happy to merge in pull requests like this, so by all means send them in. I tend to work on this in bursts, and am very happy when others can pick up the ball and run with it while my attention is on something else. :-)

I did review all your changes (your diffs were easy to follow, BTW), and intentionally kept the commenting out of clean, and the disabling of ruby, and removing the uninstall script, etc.. I think I understand why you made these changes, and they seem reasonable to me, so I am happy to keep them.

What device do you have? If you can get 10.3 (it's being pushed now), I can send you the .bar for Term48 if you want. I think it should run fine on most BB10 devices, but I haven't tested them (and the simulator is actually different in important ways), so didn't want to publish for a device if I am not sure it will work. If it works for you, then I can expand the list of supported devices.

I was actually looking at FreeBSD pkg for package management. I was initially hoping to port homebrew (hence ruby) or macports, but they both make a lot of assumptions about the system that aren't true on BB10. nix and OpenBSD ports are also possible options. pkgsrc seems like an easy win since it already runs on QNX, but I found it really hard to make happy in a non-root / non-standard configuration (fakeroot would have been handy). Like you though, I just don't have a lot of time to devote to getting a package manager running.

Anyway, thanks again for your work on all of this. I'm looking forward to more if it in the future. Cheers!

[dukzcry]

Yay, 10.3 came today! So would be glad to test your Term48 once :-)

[mordak]

Hey!

Glad that 10.3 landed for your device. Shoot me an email at mordak48@gmail.com and I'll hook you up with the Term48 bar.

Cheers

[halfnibble]

I had to copy pbhome.zip over to my device and modify the install .sh to not attempt to download the zip file. For some reason, it always failed to download over my WiFi network (it would show a 0 byte file). It could just be my network though.

Any chance of packaging this whole process into a bar file that handles everything? Or are there too many restrictions to allow this?

[mordak]

Yes and no.

It is entirely doable to package up the compiled binaries and host them somewhere, with the exception of the header files and shared libraries from the BB dev tools install (last two lines of bootstrap/gcc/build.sh). The headers and shared libs are not redistributable, and without them gcc is kind of useless.

The consequence of this is that in order to host the binaries and get people to a useful gcc install, they would need to download the dev tools and repackage the headers and shared libs anyway. That's almost the same thing as just building it all yourself, so I'm not sure that hosting binaries is worth the effort.

[halfnibble]

Sorry to be obtuse, but could you explain why those header files and shared libs are not redistributable?

[mordak]

No problem.

If you check the header files, you'll see the copyright notice prohibits redistribution. For example, if you look at /bbndk/target_10_3_0_698/qnx6/usr/include/stdlib.h, you'll see this:

• $QNXLicenseC:
• Copyright 2007, 2009, QNX Software Systems. All Rights Reserved. *
• You must obtain a written license from and pay applicable license fees to QNX
• Software Systems before you may reproduce, modify or distribute this software,
• or any work that includes all or part of this software.

The other headers are generally licensed the same way.

If you look at the SDK License (presented when you install Momentics), it says:

"You do not directly or indirectly market, rent, distribute, transfer, license, sublicense, sell, or furnish to any third party all or any part of, the SDK or copies of any part thereof"

So the headers and libs are not redistributable.

[halfnibble]

Ok, that's reassuring that it's only a license issue. I thought maybe I misunderstood the installation process.

How much do you think they'd charge to license it for this developer targeted app? I'd be willing to put a few hundred USD on a license.

Also, even though gcc may be out of the question, what about the other applications? The SSH app is a million times better than the Android ports. Can't we package the SSH binary in with the term48 app? Or have some mechanism to allow downloading the binary?

[mordak]

I think any talk of purchasing a license is just crazy. I can contact them and see if they would be willing to allow me to redistribute gratis.

Redistributing the other stuff is entirely doable, and it's on my radar to look into. There would be some checks to do to ensure that any shared libraries can be found reliably (or just build static?), but there aren't a lot of barriers to bundle binaries with something like Term48.

The other reason I haven't really done this is because I'd much (much much) prefer to solve the problem in general with some mechanism to allow downloading binaries on demand. This is the definition of a package system, and there are lots of options (and lots of work to keep up to date). The obvious choice is pkgsrc, which already runs on QNX and some others have already been working on. So it would be better if I could just do that, but I don't really have the time / resources to keep up with such a thing, and I haven't seen a third party wrapping this up nicely yet. In a perfect world, the official pkgsrc repository starts getting updated with arm binaries (ftp://ftp.netbsd.org/pub/pkgsrc/packages/QNX/), and then it would be as simple as shipping the pkg binaries, provided the compile paths (ie, not /usr/local) work on BB10.

[halfnibble]

Ok, I think I understand. It seems that "pkgsrc" would be an ideal solution, but not easy to implement.

What if (tell me if this is a crazy idea), I created a simple web application that allowed uploading of binaries by trusted developers, and a simple JSON REST API to be used by client devices. Then, we develop a simple shell application that could run in term48 to download the binaries. The binaries would be stored on my server, and could have a matching JSON file with any known requirements (a certain OS version, or another binary). All this information would be exposed by the REST API and handled by the shell application.

So, someone using term48 could run

term48-install 'ssh'

And the shell application would:

GET https://myserver.com/api/install/?app=ssh

Then get a response like:

{
    "download_url": "https://myserver.com/api/file/ssh_v1.0",
    "requirements": [
        { "OS": "10.3.1" }
    ]
}

etc...

Would this work and be easy to implement? (Creating the web application would be easy for me to do).

[mordak]

So if you said you wanted to build a package system for BB10 I would offer you my unconditional encouragement, but I do think it's a bit crazy. :-)

Implementing just a simple download + unpack tool is a step in the right direction, and would be a definite improvement (pwget + pbpkgadd already sort of do this). The next step is package upgrades, which implies package removal, and then installing the new version (minding local changes). There are many package systems that do this already (pkgsrc, pkgng, homebrew, apt, yum, nix, macports, portage, etc.), and they also solve all the other things that package managers should do (integrity checks, sample configs, dependencies, etc.). So in the end, I think it would be better to get / port an existing system for package management.

All of that said, if you want to whip up a handy tool to automatically download + install hosted binaries somewhere, I will happily pull it in. Some progress is better than no progress.

[halfnibble]

BTW, I'm still working on the package system concept. I DO have these tools compiled and hosted on my GCloud with a modified install script. It makes installing the tools on a new device really simple. As in, download the 135 MB file and run the installer script. (It was compiled for BB 10.3.1, but I've tested it on an older BlackBerry with 10.2 and BGSHELL and it works).

However, that doesn't solve the licensing issue, so the files are only for my personal use. Who could we contact about that licensing issue?

[halfnibble]

Also, I have a question about compiled apps. So, if I write and compile a C++ program on one BlackBerry, and then transfer the binary to another BlackBerry, is it impossible to run it? I might be doing something terribly wrong, but I can't chmod or chown the binary for some reason. Possibly a security feature?

[mordak]

For distributing QNX files, you have to contact QNX. The details are usually in the license who to contact. For the QNX stuff, the email address is in the header files.

For the other question, yes, there should not be any problem running a binary compiled on one BB on another BB, so long as they have compatible libraries (so, they are the same or compatible OS versions, etc.). chmod and chown may be broken on BB10 because the shell is running as an unprivileged / nonexistent user, and they both restrict what you can do based on who you are.

[khapota]

Compiled successfully in ubuntu 15.04 and Mac OSX, but had to use old version of texinfo (4.13).

[ghost]

Hi guys, I am trying to compile some basic apps on the phone itself and am getting similar looking segmentation violation compiler errors with both lynx and cmake. Is there anything generic I need to tweak (stack size?) on the system level?

[mordak]

Sorry, I don’t have any suggestions off the top of my head. You might have some luck if you check the QNX pkgsrc repo to see if they do anything to those tools to compile them. I am afraid that I won’t be a lot of help since I am not on BB10 anymore, and am travelling for the next couple of weeks with very spotty internet coverage. Good luck!

[yamsergey]

Hey-hey. Don't want to bother anyone. But maybe I can save some time here. I've got git-scm binaries there http://54.37.67.113/bb10/packages/ (thanks to @mordak I guess :) ), but when I do clone it works not stable. Sometimes it works but most of the time the cloning is failed due to fatal: bad config line {number} in file {cloned_repo_folder}/.git/config. Interesting that when I init git repo twice in the same dir I see the same error and the config file contains weird symbols: Any idea?

[mordak]

Sorry, nope. This sounds like a problem with the git binary? FWIW, I do not host compiled binaries anywhere, so I do not know who is hosting those packages you are using. The error about the bad config line seems like an obvious place to start. What is the config line? Is there anything odd about it (special / invisible characters)? Does the line have meaning in the version of git you are using? If you are getting random errors (as you indicate), it could be some kind of memory corruption that only manifests some of the time. You could try compiling the git binary yourself and seeing if you can get a newer version to compile. Anyway, sorry I can't be more help.

[yamsergey]

@mordak thanks a lot :)