Hi jrmarino, about Illumos

[hoper-a11y]

Pardon me if it's the wrong place for that.

Illumos is broad, not just another variant of Solaris 10. Currently you only allow using binary built on your Solaris 10u8 server to keep it portable across the Illumos system. I think you should lift this limitation. First, currently there are too few packages. Second, Illumos has diverted too much from the original Solaris 10. I have one example for you: they replaced libdemangle and develop their own libdemangle-sys. This caused the openjdk package from Joyent's pkgsrc to break on latest OpenIndiana. I think the same thing would happen with raven, as raven was built on even older system than the SmartOS version Joyent used to build their pkgsrc packages.

Please split Illumos and Solaris because they are no longer the same. For Illumos:

DilOS already uses APT, they use a tool to convert OpenIndiana's IPS package to DEB.

Tribblix already uses SVR4 package, they also use a tool to convert OpenIndiana's IPS package to DEB.

SmartOS already have pkgsrc.

All of them don't need raven. So there only left OpenIndiana and OmniOS, both used IPS. My idea is untested but I think packages built on OpenIndiana will also work on OmniOS.

So if you want binary package for Illumos, please build it on OpenIndiana. Don't use Solaris 10u8, please. OpenIndiana has modern toolchain and will let a broader range of packages possible to build compared to the old Solaris 10u8.

You still have Solaris 10 as a separate target so you will lost no customer but indeed have more.

And yes, please allow building from source (ports) on Illumos, too. Binary only is too limited.

Please let me know what you think.

[kraileth]

Sol 10 was chosen for the simple reason that it works as the lowest common denominator for the Solaris platform as a whole. We're of several problems:

1) Up until recently Ravenports couldn't even build the latest Clang on Solaris (it actually was an optimization issue, but it shows how much the ancient Solaris linker that we use struggles. 2) There's ports that won't build on 10u8 but would build fine on newer Solaris/illumos. We definitely don't want those to be cut off. 3) The whole platform being tied to an ancient release of Solaris that's not too easy to come by today is far less than ideal. There were in fact plans for a modern illumos platform separate from Sol10. We were in favor of OmniOS but ultimately unsure if that would mean the packages could work on OI and so on. OmniOS delivering a pretty small OS core, my guess is that chances are better that OmniOS-built packages work on OI, too, than vice-versa.

However ultimately somebody simply has to give it a try. There are a few things that we need to figure out, but as far as I can say, RP is pretty open to any suggestions from the illumos community. Pkgsrc is great and doing a very valuable service to less widely used platforms, but I think that a multi-platform packaging framework that makes use of modern tooling would also be beneficial (RP was designed to make long-term package maintaining as low-effort as possible to save much-needed manpower especially on more exotic platforms).

I'm writing a little tool for platform-bringup. So far I've avoided illumos because I don't feel confident enough to do things right. If you'd be ready to lend a hand (I'm mostly a Solaris newcomer but I found that I like what I saw so far), I can give getting Raven working on OI a try.

[hoper-a11y]

I think we should have more luck with OI rather than OmniOS. OmniOS also used to pkgsrc too much that they even provide a pkgsrc branded zones type. OmniOS also more updated than OI. There are things that you have on OmniOS but don't have on OI. OI is more vanilla, so there is more chances package built on OI to work on OmniOS than the other way around.

And don't worry about Clang. OI already has Clang-8 and Clang-9 in their IPS repo. Clang is available on pkgsrc for a much longer time :)

I don't know how to code but I'm pleased to help you.

[kraileth]

Now that's some valuable information, thank you for it! And going with OI over OmniOS sounds like a reasonable choice then. Alright, so far I've only had experience with that old Solaris 10, OmniOS, Tribblix and v9os. Had OI on my "try this" list for a while now, probably it's time to actually do it and have a look around.

I'm also not a coder and while I originally doubted that a sysadmin who can barely read some simple C code could be any help for a project like RP - well, more than 150 ports later, I think that my efforts actually can make a difference. Truth be told, I needed quite a bit of help in the beginning. But fortunately BSD has this culture of mentoring newcomers and after a while I felt comfortable in more and more areas. Doing porting work is probably the closest you get to program sources without being a programmer. :wink:

The first thing that I'd need to know: What is the canonical way to determine the platform you're on with any illumos system? Basically that is: What combination of uname -$something do I have to use to detect which distro the machine is running? Or are there other commands that help with this? Any input on that would be much appreciated.

[hoper-a11y]

Try this: https://gist.github.com/natefoo/7af6f3d47bb008669467

[hoper-a11y]

Please let me know about your progress and the troubles you encountered. Might be I could help :)

[jrmarino]

So I need to know this:

1. If ravenports is bootstrapped on OmniOS, will they also work on OpenIndiana?
2. If ravenports is bootstrapped on OpenIndiana, will they also work on Omnios?

pkgsrc is an operational disaster. I know; I ran it for years on DragonFly before become so fed up that I created dports on DragonFly. The amount of work it takes to keep dports in sync is worth it compared to "just using pkgsrc" for people that know both systems, so that should tell you something. So just because a platform "has pkgsrc" doesn't mean they aren't a candidate to use ravenports significantly.

But Kraileth is correct -- we have been considering splitting away from solaris into descendants because it's getting harder to work around missing libc functions and because of linker issues. That need lessened a bit when we got llvm compiling again but it has been a longer term goal.

We haven't been pushing ravenports due to the instability of pkg/ravensw. I wanted to fix that forever before leaving a bad taste in peoples mouths.

[hoper-a11y]

I think it would work. But I have never tried to use OmniOS repos on OI. So to be honest, I don't know. All of the discussion between me and Kraileth are just pure speculations at best.

[hoper-a11y]

It seemed that they have separate repos for each illumos distros: http://sfe.opencsw.org/quickrepolinks The guys described his build environment here: https://sfe.opencsw.org/manual-setup-build-enviroment

[jrmarino]

that's what I'm afraid of. I liked having a common repository that works for all descendants. It's possible that SmartOS, OpenIndiana, and OmniOS are effectively 3 distinct platforms.

[hoper-a11y]

that's what I'm afraid of. I liked having a common repository that works for all descendants. It's possible that SmartOS, OpenIndiana, and OmniOS are effectively 3 distinct platforms.

I think you should omit SmartOS from the list because Joyent is pretty much a pkgsrc shop now. SmartOS also is a hypervisor OS and it run entirely on RAM so I think there is no business for ravenports there.

So we only left with OI and OmniOS. I don't know about the ravenports' build infrastructure but I think provide two separate repos just for these two OS is not a big deal at all. Of course I can't know how much it will cost for you to doing so.

BTW, I think I should back to my original intent when I started this issue. I asked you to allow us to build ports from ravenports on Illumos based system just like on other systems, currently you only allowed us to install binary packages from your repo.

[jrmarino]

I can't stop you from building ports from source, but that's definitely against the original goal. And of all platforms, solaris-based platforms are the ones it makes the least sense to be source-based. right now the requirement to build packages for illumos is that you need an machine with solaris 10u8. Even when/if I bootstrap it for omnios, it will still require an exact release. Unlike BSD and linux and macos, solaris will not run older releases in the build environment. The build environment has to match the host environment without exception. That's what I've seen on solaris and I have no reason to think that restriction will be gone on OmniOS or OpenIndiana.

[jrmarino]

Regarding Smart OS, if people need pkgsrc packages there, I don't see why they could use packages from another source. I know Joyent won't support me but that shouldn't stop their customers.
Maybe I'm missing something ...

[kraileth]

Dunno if I ever mentioned it, but even the Sol 10u8 that I got is not entirely compatible with the one that you run. It was immediately crashing on me when attempting to build any package. I solved this by overwriting libc in the sysroot with the library from the os installation if I remember correctly.

For such reasons providing a bootstrap package for Solaris that allows source-based operations across the whole ecosystem is probably impossible (unless somebody comes up with some clever trick or anything). The illumos distros being open source helps a lot in that case, though. We could target - say the latest LTS for e.g. OmniOS (I'm not familiar with the OI release model, yet) - and everybody else could get that and start contributing packages that were tested for illumos. Probably not perfect, but much better than now.

And in the long run I'd like to make platform bring-up as easy as possible, so that mostly anybody can get Raven up and running on platforms that have no (or at least no compatible) bootstrap package available.

Edit: Joyent might be a special case. They pay people to work on Pkgsrc and it seems to work well for them. That doesn't mean than Raven would make no sense on SmartOS. Pkgsrc has a lot more packages available but RP usually provides much more current software. Raven can easily be used in addition to Pkgsrc if you want some software that's not in Pkgsrc or if you need a newer version.

[hoper-a11y]

Regarding Smart OS, if people need pkgsrc packages there, I don't see why they could use packages from another source. I know Joyent won't support me but that shouldn't stop their customers. Maybe I'm missing something ...

People not need pkgsrc there, jrmarino. pkgsrc is Joyent's official ports system for SmartOS and the official way of installing prebuilt binary packages (by Joyent) on SmartOS. So they are forced to use pkgsrc, not because they need it. Just like Debian forced it users to use it apt and dpkg package system. Since I have never used SmartOS, I don't know if they still keep using IPS for the system packages or completely switched to pkgsrc. But I think it's not a big deal if they are still using IPS for system and pkgsrc for other stuffs. On FreeBSD, we already have the separation between base and ports. So it's nothing strange to see something similar on SmartOS.

[hoper-a11y]

I can't stop you from building ports from source, but that's definitely against the original goal. And of all platforms, solaris-based platforms are the ones it makes the least sense to be source-based. right now the requirement to build packages for illumos is that you need an machine with solaris 10u8. Even when/if I bootstrap it for omnios, it will still require an exact release. Unlike BSD and linux and macos, solaris will not run older releases in the build environment. The build environment has to match the host environment without exception. That's what I've seen on solaris and I have no reason to think that restriction will be gone on OmniOS or OpenIndiana.

Because I feared that it would double the cost and stressed your server so I suggest allowing us to build from ports, not only install binary packages from yours.

But if you don't mind about cost, then the best scheme for Illumos is:

OmniOS has a LTS release, you only build packages for that particular release, not the bloody branch. So, firstly we must have an OmniOS LTS repo.

OpenIndiana is a bit different, they are a semi rolling release system just like Debian Testing that called OpenIndiana Hipster, the users constantly receive updates and upgrade to newer versions on the fly, but their release scheme is 6 months a release. It's all up to you to provide a repo for OpenIndiana or not, but the build server of yours also have to run the latest up to date Hipster (we assumed all users have to update to the latest version).

SmartOS: you could try to compete with pkgsrc but I don't think you will have much luck there, it's all up to you.

Update: there are also other Illumos distro, but I think supporting them is only increasing the cost, but, it's all up to you:

DilOS: they use APT, they build packages from Debian's package sources and convert OpenIndiana's IPS package to their DEB format.

Tribblix: they use ZAP, they convert OpenIndiana's IPS package to their SVR4 format.

Conclusion: first split up Illumos from Solaris and provide a repo for OmniOS LTS first, if it's widely adopted then we will think about supporting other distros, but now only try with OmniOS. FreeBSD's pkg already supporting installing packages to jail. So I think it would be easy for you to adopt the code to make it support installing packages to OmniOS zones. If you are good at marketing, you could have OmniOS to create a Ravenports branded zones just like they are currently have a pkgsrc branded zones. Good luck!

Update2: You could use just one server running OmniOS and build packages for other distros! Just utilize zones. They are just like FreeBSD's jails. OmniOS zones could run other Illumos distros just fine. You could even make a Solaris 10 zones on OmniOS, don't need a separate Solaris 10 server to build packages for Solaris 10 anymore! I used to hear about your Synth tool but have never used it. Perhaps you could extend Synth to support Illumos zones!

[kraileth]

@hoper-a11y This sounds super exciting - looking forward to try out more in that regard!

[jrmarino]

People not need pkgsrc there, jrmarino. pkgsrc is Joyent's official ports system for SmartOS and the official way of installing prebuilt binary packages (by Joyent) on SmartOS.

So my point was that if Ravenports installed on SmartOS without issue (and theoretically the solaris 10u8-based repo should) then Joyent clients would have then 2 options, not one.

[hoper-a11y]

People not need pkgsrc there, jrmarino. pkgsrc is Joyent's official ports system for SmartOS and the official way of installing prebuilt binary packages (by Joyent) on SmartOS.

So my point was that if Ravenports installed on SmartOS without issue (and theoretically the solaris 10u8-based repo should) then Joyent clients would have then 2 options, not one.

Yes, that's right. If you don't mind building for SmartOS too. The solaris 10u8 based repo should not be used but a repo specially build for current version of SmartOS running on current SmartOS should be preferred. Solaris 10u8 is too old and many of the software can't be build there, so it's nothing better than pkgsrc for the users to try ravenports, pkgsrc is even has more software available.

Update: as they said there, they use SmartOS 20161222 for maximum portability, not Solaris 10u8. So this could be the reason why they have more packages working on pkgsrc than ravenports which still build on Solaris 10u8 (it's just too few packages now!).

https://pkgsrc.joyent.com/install-on-illumos/

[jrmarino]

I don't know about "too few". We had less than 100 build failures and less than 400 missing packages at one time. And there are PLENTY of pkgsrc ports that do not build on any solaris-based OS.

[hoper-a11y]

I don't know about "too few". We had less than 100 build failures and less than 400 missing packages at one time. And there are PLENTY of pkgsrc ports that do not build on any solaris-based OS.

Just test it. This guy successfully install a whole desktop with just pkgsrc.

https://geekblood.wordpress.com/2017/10/26/installing-x11-and-a-desktop-environment-on-omnios/

Could we do the same with ravenports? Currently, we can't.

This pretty much say anything. But the main business of both pkgsrc and raven is the server space. Joyent seemed to provide smf's service files with their packages. I don't know about ravenports, though. But having smf's manifest already installed is a big advantage.

BTW, I have to agree with you about pkgsrc's instability. Sometimes a packages built, sometimes not. I used to install a package from pkgsrc. After I tried with a new installation, that packages silently disappear. It's annoying, but understandable. If the package doesn't built for Joyent, we no longer have the binary. It's just that. I also agree with you about the number of broken ports on pkgsrc. I admit I have never able to build the same packages as Joyent does. Perhaps they have their own patches that not upstreamed. But the stock pkgsrc has never able to let me possible to install the whole desktop like the guy above did with Joyent's repo. It always fails with some python expat stuffs and I have never possible to build a working xorg from pkgsrc let alone a full desktop system. With me, Joyent's binary packages repo is like magic, I don't know how they could do that!

This is the reason why I go for ravenports. Solaris' IPS pkg is too unstable with network bottleneck, it just hangs and do nothing until I cancel it with Ctrl+C. It also needs very much memory to work properly and it's slow to query for package. Tribblix's ZAP packages are just plain SVR4 packages converted by a program of ptribble from OpenIndiana's IPS packages and it package manager, zap, doesn't handle package dependency but use the metapackage concept (overlays) to manage package. I found it's inefficient and unreliable. So I put my hope on ravenports! But if that hope turned to be a waste of time, I could revert back to DilOS, they use Debian's APT+DPKG. I'm mostly happy with it, only one minor problem is it lacked of too much packages, nearly all desktop related stuffs are missing or have umet dependencies so can't be installed. So I still hope ravenports to be success!

[kraileth]

@hoper-a11y Please be more specific about which packages for your workflow RP is lacking. I'd be interested to know. Regarding desktop environments, I hope that it's not KDE Plasma or Gnome. We've already got a mostly complete Xfce (I intend to add most of the missing components over the next couple of months) and if you were to say MATE, I could tell you that this one is on my list, too. But also when it comes to programs that you're missing - just tell us or edit the wishlist yourself. It will take some time to satisfy most people's needs, especially since the priority right now is stability and probably better Solaris support and not just package count. But it'll be helpful to know what people actually need so that we can try to provide it sooner or later.

BTW: I think that a simple desktop might already be possible with e.g. the openbox window manager. Probably not everybody's cup of tea, but certainly a good start. My guess is that once the separation of Sol10 and modern illumos happens, more desktopy packages will become available much easier.

[jrmarino]

This pretty much say anything. But the main business of both pkgsrc and raven is the server space. Joyent seemed to provide smf's service files with their packages. I don't know about ravenports, though. But having smf's manifest already installed is a big advantage.

It's something I've wanted to do and it won't take much. It's just one of those lower priority items. I can same the same for systemd support and init scripts too.

I think we were pretty close on the desktop benchmark. Not kde5 or anything like that, but xfce4 was pretty close, and there are simpler windows managers/etc. Again, it was mostly a lack of testing and really because solaris is mostly a one man show (although Kraileth has started contributing a bit).

I've made progress with the ada-rewrite of ravensw/pkg this weekend. It's going to be slow but it should work really well. And with a new API from repology, we should get vulnerability support. I had actually ripped that out, but it ooks like it can go back in.

[kraileth]

@jrmarino We haven't talked about it, yet, I think, but I've had this on my mind for some time now. What about adding subpackages like :rcd, :smf, :sysv, ;systemd, :openrc, :$whatever-init-system-somebody-cares-for-and-volunteers-to-maintain? Or would you prefer to put them all into :single / :primary?

If we go for the cleaner separation, I think in the long run that can become quite convenient since you've forked pkg and thus have full control over the package manager now. It could come with a configuration option which init scripts to install by default (if it exists for some package). Might make sense to add that configuration line in the bootstrap packages, so e.g. Linux users can choose between raven-linux-bootstrap-sysv.tar.gz, raven-linux-bootstrap-systemd.tar.gz, etc.

Something like this could make a nice, clean approach to provide packages for various operation systems.

[jrmarino]

I don't think it's worth separate packages. most of these are single line items so using %%ONLY-LINUX%% in the manifest would be fine. i don't think trying to support systemd + sysv is doable tbh. We definitely don't have the manpower and there was a fork over this (devuan). If it's easy to maintain a debian fork to keep away from systemd, we've got no chance here.

[kraileth]

I partly agree. SMF is probably the easiest case and means one line in the manifest as well as a conditional copy in post-install. With rc.d it's already some more systems that might benefit from an init script being available (DF and FreeBSD already, OpenBSD and NetBSD should they ever become supported platforms).

Linux is definitely the most complicated case. But it's not Linux in general - the actual offender is Systemd. There'd be nothing wrong with providing SysV init scripts as well as ones for e.g. Runit which some distros use. If Systemd was simply another init system there probably wouldn't be much of a problem at all: We could simply ship a Systemd unit file as well and everyone would choose the init system to his or her liking. The trouble with Systemd is that it wants to actually own the system and do a lot more than what init is supposed to do. Therefore some software might need to know at compile-time if Systemd is available or not and adapt accordingly. This is also the reason why distros like Devuan were created: You can also run upstream Debian and replace Systemd with something else - but all the packages you might want to install will expect that you're using Systemd and thus come with a dependency on it. Some people loathed this so much that they forked Debian. That's basically the story - and I can understand both sides here.

But there's more. What about OpenRC? It's still the standard init system of Gentoo and a few others which want to avoid Systemd. And it would be useful to get the adapted service files available for FreeBSD packages as well, since TrueOS (and thus e.g. GhostBSD) have switched to OpenRC (the last attempt by M. Wilke to get it into vanilla FreeBSD seems to have stalled, but maybe we'll get it one day).

I know that having some critical component like an init system out of base would be less than ideal, but still it might even make sense for us to package OpenRC and investigate if it can be used on Solaris/illumos, too. If so it would be an option for people that deliberately run different operating systems but would still like to have a few things (like packages and init) be the same cross-platform.

[jrmarino]

we already install bsd init scripts unconditionally. maybe that's wrong. I don't care too much if we install install scripts that are ignored if not used (e.g. OpenRC system ignores bsd init scripts because they aren't in correct place). Where it gets dicey is creating init scripts (which we've obviously done already for BSD). For SMF support we'd certainly have to do that. Then the "what about me?" happens.
At some point it would have to be a port maintainer to decide who gets supported and makes it happen. Right now the vast majority of RP is unmaintained from the POV.

[kraileth]

The ideal solution would be one that's both completely flexible and very easy to work with. But it's good to see that you seem to favor a practical approach over some theoretical "best way" (i.e. leaving it to the maintainers in the end who will do the work, anyway - and who will be more willing to participate in RP if their specific needs can be fulfilled). Still I think we should discuss this some more when the main issue (stable ravensw) is a thing of the past. Because as things are right now, it's nice that we package e.g. the Apache webserver for Linux and Solaris, but nobody likes to run it from the command line these days. So to make things really useful, we need to offer something to do service management.

Regarding SMF I have no idea since I'm completely uneducated in that area. But for OpenRC we could get service files from TrueOS ports and Gentoo's portage tree. At some point I want to take a look at some diffs to figure out how different they are or if they are mostly the same. SysV init scripts should also not be too hard to collect, but that's not something that I'd like to do. Same for Systemd unit files.

I thought about subpackages again and also came to the conclusion that creating separate ones for various init systems might be overkill. You're right, some small files probably don't hurt anybody. But maybe we could put them into a well-known subpackage like :init? If you are open to teaching ravensw a few tricks that pkg doesn't know, it could even treat such a subpackage specially and only extract the correct init script for the init system that it's configured for (if that one is present in the package, of course). Maybe they'd even be installed implicitly when :primary is being installed and they are available for that program.

If we can agree on something like :init I'd volunteer to go over all the ports and see where there's rc.d scripts that would need to be split out. And while I feel that we should post-pone any real decisions on this not so little topic, I'd rather do something like this first step as long as we still have a relatively small port count.

[hoper-a11y]

@hoper-a11y Regarding desktop environments, I hope that it's not KDE Plasma or Gnome. We've already got a mostly complete Xfce (I intend to add most of the missing components over the next couple of months) and if you were to say MATE, I could tell you that this one is on my list, too.

Hey. I'm not crazy. KDE Plasma is obviously impossible now. I'm a fan of XFCE until they made some changes that caused serious screen tearing for me. I switched to MATE since then. If some day I could have a full MATE Desktop setup from ravenports, it's awesome!

[hoper-a11y]

Hey. All of you. You seemed to have misunderstanding about these init shell scripts. The fact is all of them called RC script but they are incompatible! You can't use them interchangeable! For example, SysV init script had to have LSB header. And the OpenRC on Linux and the OpenRC on other systems are not similar. At least I know the one on Linux is heavily patched, e.g: to support LSB header.

You don't believe it but... the easiest init system to deal with is systemd! Just write a simple service unit file that wraps the RC script and you will be fine. For example:

[Unit] SourcePath=/etc/init.d/wibble [Service] ExecStart=/etc/init.d/wibble start ExecStop=/etc/init.d/wibble stop

(I'm not tested this example, I took it from there: https://unix.stackexchange.com/questions/233468/how-does-systemd-use-etc-init-d-scripts)

BTW, IMHO, the easiest way to solve the problem is to use a secondary init system, in addition to system's primary init system. RC scripts could remain as RC scripts and being managed by a secondary RC based init system. e.g: http://smarden.org/runit/useinit.html (Just for example, I don't want you to switch to runit, hehe!)

Disclaimer: all of the information above are just from my little research with google, none of them really tested by me. You have to verify it yourselves.

[hoper-a11y]

The above is for an easy route, though. Doing so ravenports could keep it as a portable solution. But if it want to compete with the native ports/package manager of the system (e.g: Joyent's pkgsrc/pkgin on SmartOS), it have to be more native (providing smf service files is a must).

[jrmarino]

i agree with that, but it's not like pkgsrc provides SMF files. I don't think I've seen any solution that does -- probably OpenIndiana packages but I actually don't know.

[hoper-a11y]

i agree with that, but it's not like pkgsrc provides SMF files. I don't think I've seen any solution that does -- probably OpenIndiana packages but I actually don't know.

Then you should check it again: https://github.com/joyent/pkgsrc/tree/trunk/www/apache24/files/smf

[jrmarino]

i guess if you kill 1 person you're a murderer. Okay, that's one example. There's 15,000 ports according to repology. what percentage of the ones where SMF makes sense actually have them? either way, good. those are easy to steal so we don't need to recreate.

[hoper-a11y]

what percentage of the ones where SMF makes sense actually have them?

I think at least the server related stuffs like the amp stack, nginx... where it's most useful to be able to manage the daemons with SMF services.

[hoper-a11y]

Yeah. The /dev/urandom bug is gone with the latest version of ravensw on OI. But it also imported the disease of the Linux version of ravensw: segmentation fault. I think I would better continue using the old pkg and pkg-static, they have /dev/urandom bug but this bug is pretty much harmless and they does really work.

[hoper-a11y]

The old pkg and pkg-static are more stable than ravensw on Illumos and they worked pretty well.

I just tried to setup Joyent's pkgsrc repo on my DilOS system, everything went straight but ironically it failed when calling pkgin:

reading local summary... processing local summary... processing remote summary (https://pkgsrc.joyent.com/packages/SmartOS/trunk/x86_64/All)... stack smashing detected : terminated Illegal Instruction (core dumped)

In my surprise ravenports worked very well on DilOS like that they were born for each other. John, might I ask you to add DilOS to first class support of the Illumos target? ravenports worked too well on this OS!

Another thing I observed is the compilers packaged by both pkgsrc and ravenports don't work.

Test.cpp is just a helloworld style program.

This is gcc 9 installed by ravenports gcc9-complete-standard:

/raven/toolchain/gcc9/bin/g++ test.cpp -o testx /raven/toolchain/bin/ld:/raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0/../../../libgcc-unwind.map: file format not recognized; treating as linker script /raven/toolchain/bin/ld:/raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0/../../../libgcc-unwind.map:1: syntax error collect2: error: ld returned 1 exit status

But what I surprised most is clang 10 installed from ravenports clang10-complete-standard package seemed to work. It compiled test.cpp just fine. But when I tried to run the program it failed with:

ld.so.1: testxx: fatal: libstdc++.so.6: open failed: No such file or directory Killed

Despite how hard I messed with LD_RUN_PATH and crle I can't get it to pick up the correct library in /raven/toolchain/gcc9/lib/amd64.

So I tried to compile this way:

clang -Wl,-rpath,/raven/toolchain/gcc9/lib/amd64 test.cpp -o test

And it failed with:

Undefined first referenced symbol in file _ZNSolsEPFRSoS_E /var/tmp/test-a5e738.o _ZSt4cout /var/tmp/test-a5e738.o _ZNSt8ios_base4InitD1Ev /var/tmp/test-a5e738.o _ZNSt8ios_base4InitC1Ev /var/tmp/test-a5e738.o _ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc /var/tmp/test-a5e738.o _ZSt4endlIcSt11char_traitsIcEERSt13basic_ostreamIT_T0ES6 /var/tmp/test-a5e738.o ld: fatal: symbol referencing errors. No output written to test clang-10: error: linker command failed with exit code 1 (use -v to see invocation)

Well. Solaris/Illumos used their own linker. These compilers which use the GNU linker of course failed to link.

Conclusion: compilers packaged by pkgsrc and ravenports are pretty much useless!

[hoper-a11y]

Update: I think clang successfully compiled the test because it invoked the Solaris/Illumos linker /usr/bin/ld but not the linker shipped with ravenports! It's the only way I could think of.

[jrmarino]

it's supposed to use the system linker. It's the only linker that works. gnu ld and gnu gold are garbage on solaris, and I've never gotten llvm's lld tested in any way.

[jrmarino]

I don't think you have the latest ravensw. I just tried building it (to redo the bootstrap) and it was missing a definition. It can't build without it. So hold off on "it's worse that pkg-static". lets try the test again once the bootstrap is redone and I produce a new set of packages.

[jrmarino]

In fact, in the repository I see 1.11.1_4 and 1.11.1_6 is the latest. I suspect you have _4.

[hoper-a11y]

Update: On OpenIndiana, pkgsrc seemed to be well tested and worked very well. It's even possible to compile the test.cpp with their packaged compilers (installed from gcc9 and clang-10).

gcc9 package has some trouble with the library path. These symlinks fixed this:

ln -s /usr/gcc/7/lib/amd64/libstdc++.so.6 /usr/lib/amd64/libstdc++.so.6 ln -s /usr/gcc/7/lib/amd64/libgcc_s.so.1 /usr/lib/amd64/libgcc_s.so.1

I created an alias to work better with gcc9 (the path is too long to type again and again):

alias g++9='/opt/local/gcc9/bin/g++'

This is the output of g++9 -v:

Using built-in specs. COLLECT_GCC=/opt/local/gcc9/bin/g++ COLLECT_LTO_WRAPPER=/opt/local/gcc9/libexec/gcc/x86_64-sun-solaris2.11/9.3.0/lto-wrapper Target: x86_64-sun-solaris2.11 Configured with: ../gcc-038ec0d78529e36af568643e799bb1c01e9f1ad0/configure --with-local-prefix=/opt/local/gcc9 --enable-languages=c,c++,fortran,go,objc,lto --enable-libssp --with-gnu-as --with-as=/opt/local/bin/gas --without-gnu-ld --with-ld=/bin/ld --disable-nls --disable-libitm --prefix=/opt/local/gcc9 --build=x86_64-sun-solaris2.11 --host=x86_64-sun-solaris2.11 --infodir=/opt/local/gcc9/info --mandir=/opt/local/gcc9/man Thread model: posix gcc version 9.3.0 (GCC)

This is the output of clang -v:

clang version 10.0.0 Target: x86_64-pc-solaris2.11 Thread model: posix InstalledDir: /opt/local/bin

On OpenIndiana, the compiler itself is a 32 bit binary and target 32 bit by default, to have 64 bit we have to specify it with -m64. Joyent's packaged compilers are 64 bit binary and target 64 bit by default (a big plus for me since it's more convenient to build software from source).

The output of ldd on the binary compiled by g++9:

libstdc++.so.6 => /opt/local/gcc9/x86_64-sun-solaris2.11/lib/amd64/libstdc++.so.6 libm.so.2 => /lib/64/libm.so.2 librt.so.1 => /lib/64/librt.so.1 libgcc_s.so.1 => /opt/local/gcc9/x86_64-sun-solaris2.11/lib/amd64/libgcc_s.so.1 libc.so.1 => /lib/64/libc.so.1

The output of ldd on the binary compiled by clang 10:

libc++.so.1 => /opt/local/lib/libc++.so.1 libc++abi.so.1 => /opt/local/lib/libc++abi.so.1 libm.so.2 => /lib/64/libm.so.2 libc.so.1 => /lib/64/libc.so.1 libpthread.so.1 => /lib/64/libpthread.so.1 librt.so.1 => /lib/64/librt.so.1 libgcc_s.so.1 => /opt/local/gcc9/x86_64-sun-solaris2.11/lib/amd64/libgcc_s.so.1 libumem.so.1 => /lib/64/libumem.so.1 libunwind.so.1 => /opt/local/lib/libunwind.so.1 libdl.so.1 => /lib/64/libdl.so.1

Yeah. It seemed Joyent has managed to get LLVM's libc++ worked on Illumos. A big cheer for them!

I will update ravenports compilers status on OpenIndiana soon.

[hoper-a11y]

The pkg program is very good on SunOS. I don't have any segmentation faults.

The full output when installing gcc9:

pkg install gcc9-complete-standard Collecting entropy via /dev/urandom has failed. Updating Raven repository catalogue... Raven repository is up to date. All repositories are up to date. The following 8 package(s) will be affected (of 0 checked):

New packages to be INSTALLED: gcc9-complete-standard: 9.3.0 gcc9-ada_run-standard: 9.3.0 gcc9-libs-standard: 9.3.0 gcc9-compilers-standard: 9.3.0 gcc9-cxx_run-standard: 9.3.0 binutils-single-ravensys: 2.34 gcc9-fortran_run-standard: 9.3.0 gcc9-infopages-standard: 9.3.0

Number of packages to be installed: 8

The process will require 504 MiB more space. 132 MiB to be downloaded.

Proceed with this action? [y/N]: y [1/8] Fetching gcc9-complete-standard-9.3.0.tzst: 100% 614 B 0.6kB/s 00:01
[2/8] Fetching gcc9-ada_run-standard-9.3.0.tzst: 100% 13392895 B 1.0MB/s 00:13
[3/8] Fetching gcc9-libs-standard-9.3.0.tzst: 100% 321185 B 107.1kB/s 00:03
[4/8] Fetching gcc9-compilers-standard-9.3.0.tzst: 100% 80342824 B 2.0MB/s 00:40
[5/8] Fetching gcc9-cxx_run-standard-9.3.0.tzst: 100% 1743471 B 249.1kB/s 00:07
[6/8] Fetching binutils-single-ravensys-2.34.tzst: 100% 40150660 B 2.1MB/s 00:19
[7/8] Fetching gcc9-fortran_run-standard-9.3.0.tzst: 100% 2034344 B 226.0kB/s 00:09
[8/8] Fetching gcc9-infopages-standard-9.3.0.tzst: 100% 391055 B 97.8kB/s 00:04
Checking integrity...Collecting entropy via /dev/urandom has failed. done (0 conflicting) [1/8] Installing gcc9-libs-standard-9.3.0... [1/8] Extracting gcc9-libs-standard-9.3.0: 100% [2/8] Installing gcc9-cxx_run-standard-9.3.0... [2/8] Extracting gcc9-cxx_run-standard-9.3.0: 100% [3/8] Installing binutils-single-ravensys-2.34... [3/8] Extracting binutils-single-ravensys-2.34: 100% [4/8] Installing gcc9-ada_run-standard-9.3.0... [4/8] Extracting gcc9-ada_run-standard-9.3.0: 100% [5/8] Installing gcc9-compilers-standard-9.3.0... [5/8] Extracting gcc9-compilers-standard-9.3.0: 100% [6/8] Installing gcc9-fortran_run-standard-9.3.0... [6/8] Extracting gcc9-fortran_run-standard-9.3.0: 100% [7/8] Installing gcc9-infopages-standard-9.3.0... [7/8] Extracting gcc9-infopages-standard-9.3.0: 100% [8/8] Installing gcc9-complete-standard-9.3.0... [8/8] Extracting gcc9-complete-standard-9.3.0: 100%

The full output when installing clang 10:

pkg install clang-complete-standard Collecting entropy via /dev/urandom has failed. Updating Raven repository catalogue... Raven repository is up to date. All repositories are up to date. The following 9 package(s) will be affected (of 0 checked):

New packages to be INSTALLED: clang-complete-standard: 10.0.0_1 clang-compiler-standard: 10.0.0_1 zlib-shared-standard: 1.2.11 ncurses-primary-standard: 6.2_1 llvm-single-standard: 10.0.0_1 libxml2-single-standard: 2.9.10 libiconv-shared-standard: 1.16 libexecinfo-single-standard: 1.1,1 clang-extra-standard: 10.0.0_1

Number of packages to be installed: 9

The process will require 2 GiB more space. 288 MiB to be downloaded.

Proceed with this action? [y/N]: y [1/9] Fetching clang-complete-standard-10.0.0_1.tzst: 100% 463 B 0.5kB/s 00:01
[2/9] Fetching clang-compiler-standard-10.0.0_1.tzst: 100% 71923321 B 2.5MB/s 00:29
[3/9] Fetching zlib-shared-standard-1.2.11.tzst: 100% 51694 B 51.7kB/s 00:01
[4/9] Fetching ncurses-primary-standard-6.2_1.tzst: 100% 3127642 B 391.0kB/s 00:08
[5/9] Fetching llvm-single-standard-10.0.0_1.tzst: 100% 194634044 B 3.0MB/s 01:04
[6/9] Fetching libxml2-single-standard-2.9.10.tzst: 100% 1119739 B 160.0kB/s 00:07
[7/9] Fetching libiconv-shared-standard-1.16.tzst: 100% 723226 B 144.7kB/s 00:05
[8/9] Fetching libexecinfo-single-standard-1.1,1.tzst: 100% 13302 B 13.3kB/s 00:01
[9/9] Fetching clang-extra-standard-10.0.0_1.tzst: 100% 30478061 B 1.3MB/s 00:24
Checking integrity...Collecting entropy via /dev/urandom has failed. done (0 conflicting) [1/9] Installing libiconv-shared-standard-1.16... [1/9] Extracting libiconv-shared-standard-1.16: 100% [2/9] Installing zlib-shared-standard-1.2.11... [2/9] Extracting zlib-shared-standard-1.2.11: 100% [3/9] Installing ncurses-primary-standard-6.2_1... [3/9] Extracting ncurses-primary-standard-6.2_1: 100% [4/9] Installing libxml2-single-standard-2.9.10... [4/9] Extracting libxml2-single-standard-2.9.10: 100% [5/9] Installing libexecinfo-single-standard-1.1,1... [5/9] Extracting libexecinfo-single-standard-1.1,1: 100% [6/9] Installing llvm-single-standard-10.0.0_1... [6/9] Extracting llvm-single-standard-10.0.0_1: 100% [7/9] Installing clang-compiler-standard-10.0.0_1... [7/9] Extracting clang-compiler-standard-10.0.0_1: 100% [8/9] Installing clang-extra-standard-10.0.0_1... [8/9] Extracting clang-extra-standard-10.0.0_1: 100% [9/9] Installing clang-complete-standard-10.0.0_1...

[hoper-a11y]

Here is the result of ravenports compilers on OpenIndiana.

The path to g++9 is too long so I use this alias:

alias g++9='/raven/toolchain/gcc9/bin/g++'

The output of g++9 -v:

Using built-in specs. COLLECT_GCC=/raven/toolchain/gcc9/bin/g++ COLLECT_LTO_WRAPPER=/raven/toolchain/gcc9/libexec/gcc/x86_64-raven-solaris2.10/9.3.0/lto-wrapper Target: x86_64-raven-solaris2.10 Configured with: /construction/gcc9/gcc-9.3.0/configure --enable-languages=c,c++,ada,fortran --with-local-prefix=/raven --with-gmp=/raven --with-mpc=/raven --with-mpfr=/raven --with-libiconv-prefix=/raven --enable-shared --enable-threads=posix --enable-checking=release --enable-libquadmath --disable-nls --disable-multilib --disable-libsanitizer --disable-libvtv --disable-libmpx --disable-libcilkrts --with-pkgversion=Ravenports --enable-obsolete --enable-symvers=no --without-gnu-ld --with-gnu-as --with-as=/raven/toolchain/bin/as --with-ld=/raven/toolchain/bin/ld --prefix=/raven/toolchain/gcc9 --localstatedir=/var --mandir=/raven/toolchain/gcc9/share/man --infodir=/raven/toolchain/gcc9/share/info/ --build=x86_64-raven-solaris2.10 Thread model: posix gcc version 9.3.0 (Ravenports)

The output of clang -v:

clang version 10.0.0 Target: x86_64-raven-solaris2.10 Thread model: posix InstalledDir: /raven/bin Found candidate GCC installation: /raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0 Selected GCC installation: /raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0 Candidate multilib: .;@m64 Selected multilib: .;@m64

When compiled with g++9, it failed with:

g++9 test.cpp -o test9 /raven/toolchain/bin/ld:/raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0/../../../libgcc-unwind.map: file format not recognized; treating as linker script /raven/toolchain/bin/ld:/raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0/../../../libgcc-unwind.map:1: syntax error collect2: error: ld returned 1 exit status

Again, ld error. It seemed it use the gnu ld by default.

Clang, on the other hand, successfully compiled the binary and the binary also run fine.

Here is the output of ldd on the resulted binary:

libstdc++.so.6 => /usr/lib/64/libstdc++.so.6 libgcc_s.so.1 => /usr/lib/64/libgcc_s.so.1 libc.so.1 => /lib/64/libc.so.1 libm.so.2 => /lib/64/libm.so.2

So the only answer possible is clang is actually invoked the Solaris/Illumos /usr/bin/ld linker!

[hoper-a11y]

Convenient scripts I used to test pkgsrc and ravenports on both OS:

pkgsrc.sh

!/bin/sh

export PATH=/opt/local/sbin:/opt/local/bin:$PATH export MANPATH=/opt/local/man:$MANPATH bash

raven.sh

!/bin/sh

export PATH=/raven/bin:/raven/sbin:$PATH bash

Just use sudo ./pkgsrc.sh or sudo ./raven.sh and exit when you are done.

[jrmarino]

Again, ld error. It seemed it use the gnu ld by default.

I don't see how that's possible. It is configured to use sun linker. Unless you've got some internal paths that force it to gnu ld.

And ravenports uses the gcc it builds as the ports compiler, so it obviously works -- at least on the build platform but should everywhere.

[jrmarino]

also, i'm not sure why you are posting pkgsrc stuff here. What's the intention with that?

[hoper-a11y]

Again, ld error. It seemed it use the gnu ld by default.

I don't see how that's possible. It is configured to use sun linker. Unless you've got some internal paths that force it to gnu ld.

And ravenports uses the gcc it builds as the ports compiler, so it obviously works -- at least on the build platform but should everywhere.

At least please have a look at the output of my reports:

g++9 test.cpp -o test9 /raven/toolchain/bin/ld:/raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0/../../../libgcc-unwind.map: file format not recognized; treating as linker script /raven/toolchain/bin/ld:/raven/toolchain/gcc9/lib/gcc/x86_64-raven-solaris2.10/9.3.0/../../../libgcc-unwind.map:1: syntax error collect2: error: ld returned 1 exit status

It's obviously using the gnu ld linker: /raven/toolchain/bin/ld and I didn't mess with the paths at all.

[jrmarino]

What makes you think /raven/toolchain/bin/ld is the gnu linker?

[hoper-a11y]

also, i'm not sure why you are posting pkgsrc stuff here. What's the intention with that?

The purpose of my test is testing the usefulness of compilers packaged by both pkgsrc and ravenports. I don't have any other intentions. It's just convenient when I tested ravenports (my main purpose) why don't have a look at pkgsrc, too? It's just that. If you are not pleased I will not post anything pkgsrc again. Sorry.

[jrmarino]

i'm only concerned about ravenports. I would assume anything pkgsrc produces would be useful. and by "useful" I mean functions. If a ravenports compiler doesn't function, I'd like to understand why (as long as it's understood there's been limited testing on the sunos platform in general)