universal Linux executable

doug-moen commented 5 years ago

The Curv project should provide pre-built Linux executables, so that users don't need to build their own executables.

The 3 main alternatives for distribution-independent, universal Linux executables are AppImage, FlatPak, and Snappy.

@ivocavalcante has started experimenting with Snappy builds.

doug-moen commented 5 years ago

@ivocavalcante wrote the following on the Curv mailing list (synopsis).

Hi all, I'm working on a Snap package for Curv.

I just have the name "Curv" reserved on Snapcraft store (which I'll gladly hand, if you decide to manage Snapcraft) and uploaded first release. It can be installed using "sudo snap install curv".

Curv is now live on Snap store: https://snapcraft.io/curv

doug-moen commented 5 years ago

I (@doug-moen) am having problems with the Snappy package.

First problem: it is built with "strict confinement", which means that Curv cannot open files outside of my home directory. This is a problem for me, since I keep all of my Curv source on a ZFS volume outside of my home directory. The error message doesn't tell me what was going wrong, and Ivo had to explain the home directory restriction to me. That's a support problem.

$ /snap/bin/curv orbit.curv
ERROR: "orbit.curv": No such file or directory

This could be fixed by disabling strict confinement.

Second problem, Curv cannot use OpenGL for rendering.

$ /snap/bin/curv -x cube
3D shape 2×2×2
GLFW error 0x10007: GLX: Failed to create context: GLXBadFBConfig
ABORT: GLFW create window failed

Ivo asked for snap connections curv output. Here it is:

$ snap connections curv
Interface       Plug                 Slot             Notes
desktop         curv:desktop         :desktop         -
desktop-legacy  curv:desktop-legacy  :desktop-legacy  -
home            curv:home            :home            -
opengl          curv:opengl          :opengl          -
x11             curv:x11             :x11             -

This shows that Curv has an OpenGL slot, so OpenGL should work.

Ivo asked for curv --version output. Here it is, for both a working Curv executable, and the Snappy package:

$ curv --version
Curv: 0.4-209-ga7c07d1-dirty
Compiler: gcc 5.4.0 20160609
Kernel: Linux 4.4.0-137-generic x86_64
GPU: Intel Open Source Technology Center, Mesa DRI Intel(R) Ivybridge Desktop
OpenGL: 3.0 Mesa 18.0.5

$ /snap/bin/curv --version
Curv: 0.4
Compiler: gcc 7.4.0
Kernel: Linux 4.4.0-137-generic x86_64
GPU: Intel Open Source Technology Center, Mesa DRI Intel(R) Ivybridge Desktop
OpenGL: 1.4 (3.0 Mesa 18.0.5)

Note that the snap version of Curv is reporting OpenGL 1.4. For rendering, Curv 0.4 attempts to create an OpenGL context with a version 3.3 core profile, and it is this specific configuration that is failing on my system. curv --version also creates an OpenGL context, but it doesn't specify a version, and that is succeeding.

doug-moen commented 5 years ago

Why can't Curv use OpenGL on my system? I looked around to see if other open source OpenGL projects are running into similar issues.

OpenSCAD has a snappy package, but they recommend that people do not use it, because of "known display issues". The OpenSCAD snap doesn't work on my machine -- it opens a window, but the window stays black. OpenSCAD also distributes an AppImage, and this is the format that they recommend that people use on Linux machines. The OpenSCAD AppImage works fine on my machine.

doug-moen commented 5 years ago

I downloaded blender from the Snap store, and it works on my machine. So it is possible for snap packages to work on my machine.

One problem. When I installed blender initially, I got this error message:

lux 2249 $ sudo snap install blender
[sudo] password for doug: 
error: This revision of snap "blender" was published using classic confinement
       and thus may perform arbitrary system changes outside of the security
       sandbox that snaps are usually confined to, which may put your system at
       risk.

       If you understand and want to proceed repeat the command including
       --classic.

This error message is not a good user experience for Curv users.

The standard Linux distribution for Blender (downloaded from blender.org) is a tar file that expands into a directory containing an executable, plus subdirectories containing plugins and other data files.

The blender executable is somehow distro-independent. Most libraries are statically linked (with the exception of low level libraries, such as X11, OpenGL and libc). It's not an AppImage.

Blender provides 2 Linux downloads: 64 bit, and 32 bit. There is also a snap store link lower down the page (but snap doesn't support 32 bit).

The reason that Blender is normally distributed as a directory: This allows you to install multiple versions of Blender and switch between them. Blender allows you to install 3rd party plugins, and the directory distribution format allows each version of blender that you have installed to have its own plugins, which avoids version conflict.

ivocavalcante commented 5 years ago

The main reason I didn't go for classic confinement was the message you saw. I still believe we can have Curv working on strict - it works on my machine, and the machines I tested at work, including a Quadro FX5800 -, but we would adopt the strategy of having a confined and classic tracks for Curv; user selects whatever he finds best.

We could look into other formats too; I'm more favorable to Flatpak instead of AppImage. Will give it a try, once done with Snap.

doug-moen commented 5 years ago

Snappy won't work for all users. Here are the problems I see:

It's not open source. The snappy client software is open source, but the server is proprietary and only Ubuntu can run a server. Users who care about open source, and see it as an ethical issue, or who run distros like Trisquel Linux that exclude non-free software, won't be able to use a snappy package. I've read you need to sign a CLA with Canonical to upload packages to snapcraft.io, and that's another barrier that will exclude some potential developers.
It requires root to install a snappy package. This is a problem in educational environments, where you want to run Curv on a shared Linux machine owned by your institution. By comparison, an AppImage can be stored on a thumb drive and run directly off the thumb drive without installing the program in a standard location.
Strict confinement doesn't work for users like me, who need to run Curv programs that aren't stored in their home directory. Classic confinement produces a scary error message on installation.

AppImage doesn't have these drawbacks. I use AppImage packages and haven't had a problem with them. AppImage packages are just files, so they are really easy to deal with. If you want to have several versions of Curv installed, it's trivial with AppImage. AppImage supports optional sandboxing, and in-place updating is available if you use AppImageUpdate, so it has feature parity with Snappy and FlatPak. I am not a fan of Canonical being a middleman between the Curv project and its users. I'd prefer users to download Curv executables directly from a Curv controlled web site.

The Blender strategy also seems attractive, because it doesn't introduce any new dependencies (on Snappy, AppImage or FlatPak). Just statically link as many dependencies as you can. It's a bunch of work to statically link all my remaining dependencies, but I'm going to have to do a lot of that work anyway to build a WebAssembly module (needed for Sebastien's visual programming environment). So I am considering this as a future option.

I know little about FlatPak, so I don't have an opinion about it right now.

ivocavalcante commented 5 years ago

I am not a fan of Canonical being a middleman between the Curv project and its users. I'd prefer users to download Curv executables directly from a Curv controlled web site.

I can see that. :-) Don't like it either, but I think the format has potential - especially in IoT devices.

Well, I understand your point. Thing is, I prefer having some sort of system managing the packages I have on my machine than having a bunch of AppImage files stored; that's a matter of taste, so no point in debating.

AppImage is really convenient (I use some myself, notably PrusaSlicer), but I have my share of problems with it (maybe due to some misuse of mine). Anyway, it's perfectly understandable your prefence in using it.

Since I was already working with Snap, I prefer to finish the job - either have a fully working version or abandon it, after trying all options. Hope you haven't a problem with it - since the package is under my credentials, it won't be seen as an official release, I expect.

Anyway, when I finish, I can take a look at AppImage, if you already haven't.

ivocavalcante commented 5 years ago

Meanwhile, could you please test a new version I uploaded to beta channel? I hope it works in your computer.

$ snap install curv --beta

or, if it's already installed,

$ snap refresh curv --beta

doug-moen commented 5 years ago

The new version you uploaded to beta channel has the same problem as before: OpenGL doesn't work.

Since blender works, why not look at how blender does their snap build. It might provide clues why their snap works on my machine: https://github.com/dfelinto/blender/tree/c3ef1f8db3c420612aecf40359533cb043615208/release/freedesktop/snap

doug-moen commented 5 years ago

Here's the contents of /snap/blender/33/meta/snap.yaml on my system:

name: blender
version: '2.80'
summary: Blender is the free and open source 3D creation suite.
description: "Blender is the free and open source 3D creation suite. It supports the\n\
  entirety of the 3D pipeline\u2014modeling, rigging, animation, simulation,\nrendering,\
  \ compositing and motion tracking, and video editing.\n\nBlender is a public project,\
  \ made by hundreds of people from around the\nworld; by studios and individual artists,\
  \ professionals and hobbyists,\nscientists, students, VFX experts, animators, game\
  \ artists, modders, and\nthe list goes on.\n\nThe standard snap channels are used\
  \ in the following way:\n\n       stable - Latest stable release.\n    candidate\
  \ - Test builds for the upcoming stable release.\n"
architectures:
- amd64
confinement: classic
grade: stable
apps:
  blender:
    command: command-blender.wrapper
license: GPL-3.0

Why not try giving me a version of curv with classic confinement, and see if that fixes the problem?

doug-moen commented 5 years ago

I'm looking for more snaps that both use OpenGL, and work correctly. No luck so far. My theory is that snappy's model of how to perform OpenGL confinement is buggy, and doesn't work for all configurations.

I tried snap install meshlab. /snap/bin/meshlab just core dumps after printing a bunch of AppArmor error messages. The snap uses strict confinement.

I tried voxelizer. Same failure as Curv. It uses strict confinement.

ivocavalcante commented 5 years ago

Using classic confinement isn't straightforward; the snap must go through a vetting process, to ensure (from what I read):

Ensure improvement of the Snap format, by investigating the reason why a particular Snap needs special rights.
Some degree of security, since the package won't receive classic until formally approved.

That being said, I can provide you a binary package with classic confinement - or you could build it yourself, the recipe is here, on snap dir (master branch):

https://github.com/ivocavalcante/curv

I agree that OpenGL interface is broken, should be easier to use.

ivocavalcante commented 5 years ago

I've updated the store page, stating that the package isn't related to the project and there are know issues with some NVIDIA setup.

doug-moen commented 5 years ago

All of my previous testing was using Intel integrated graphics. I just switched to my nVidia GTX 1050, and now it works.

Maybe it's a problem specific to the Intel driver. Or maybe it's a problem with system configurations where there are multiple GPUs available. Maybe the snap opengl implementation makes a mistake in detecting which GPU is active, for example.

ivocavalcante commented 5 years ago

Or maybe it's a problem with system configurations where there are multiple GPUs available. Maybe the snap opengl implementation makes a mistake in detecting which GPU is active, for example.

Probably. Snap team seems to be drafting better GPU support (including CUDA, Vulkan, etc.), but for now, that's what we have. I intend to keep package on store, since it works for me (and others, prob.). Do you have a problem with it?

doug-moen commented 5 years ago

I understand the attraction of making Curv available in a popular Linux app store: ease of installation, automatic updates, centralized management of your installed apps.

My main issue with the snappy package is that it is broken on my system in multiple ways. It's probably broken on other people's systems, the bugs can't be fixed, and I don't want to have to support it.

The conflict between Curv and snappy's confinement system might get worse in the future. I'll be switching to Vulkan at some point to support faster and fancier rendering. A package manager will need network access and a place to cache packages. The forthcoming visual programming system might need plugins.

If the goal is to put Curv onto a popular app store, then flatpak and flathub look like a better solution. From what I've read about FlatPak so far, the sandboxing creates problems (like Snap), but it's easier for developers to disable, and most flatpaks are shipped with sandboxing disabled. So, a Curv flatpak should work correctly. (Note, I haven't tried flatpak yet.)

If you want to keep the snap store version of Curv around, I want it made clear that the graphics are broken on some systems, and it isn't officially supported. If you find yourself unable to update the snappy package in the future, due to personal circumstances, or due to changes to Curv that break the snappy package on all systems, I would appreciate it if you could remove the package at that time. Or notify me by email if that isn't possible.

mosheliv commented 4 years ago

Hi, I am actually interested in trying curv, but a bit intimidated by the build process and does not want to "soil" my nice pristine server with all the required packages without knowing it will work for me. Is there a docker image/file by chance that someone made? If not, I can try and make one.

ivocavalcante commented 4 years ago

Well, you can try the Snap package, for a start. It's available here:

https://snapcraft.io/curv

There are some limitations, inherent to the technology, but you might get an ideia.

doug-moen commented 4 years ago

You can try the Snap store version of Curv created by Ivo.

There are a couple of reasons why you might not want this. You might not want to install Snap on your system for social reasons (some Linux distros, like Mint, are advising people to avoid Snap). Or the Curv snap package might not run properly (it doesn't work properly on my system). But if you are running Ubuntu anyway, then it is easy to try the snap package.

I think the best alternatives to Snap are Flatpak and Appimage. Unlike Snap, Curv packaged with these technologies could be made to run without technical limitations on any modern Linux system. Nobody has packaged Curv using either of these technologies, but I would consider adding support.

Nobody has reported making a docker image, and docker is not a general solution that would work for everybody. The problem with docker, AFAIK, is that Curv requires GPU-accelerated rendering (otherwise the rendering is noticeably slower), and accessing the GPU from docker is difficult. AFAIK it is only possible if you have an nvidia GPU, and then you need a special docker driver from nvidia. Here is a github project for packaging Blender in a docker package (the challenges are similar to packaging Curv): https://github.com/jtomori/docker_blender_gpu

mosheliv commented 4 years ago

Working with gpu in docker is actually easier than outside of docker... i am dealing a lot with deep learning and docker is by far the most heavily used technology to overcome all the cuda drivers incompatibilities. I'll give it a try and see.

On Fri, Jul 31, 2020, 01:04 Doug Moen notifications@github.com wrote:

You can try the Snap store version of Curv created by Ivo.

There are a couple of reasons why you might not want this. You might not want to install Snap on your system for social reasons (some Linux distros, like Mint, are advising people to avoid Snap). Or the Curv snap package might not run properly (it doesn't work properly on my system). But if you are running Ubuntu anyway, then it is easy to try the snap package.

I think the best alternatives to Snap are Flatpak and Appimage. Unlike Snap, Curv packaged with these technologies could be made to run without technical limitations on any modern Linux system. Nobody has packaged Curv using either of these technologies, but I would consider adding support.

Nobody has reported making a docker image, and docker is not a general solution that would work for everybody. The problem with docker, AFAIK, is that Curv requires GPU-accelerated rendering (otherwise the rendering is noticeably slower), and accessing the GPU from docker is difficult. AFAIK it is only possible if you have an nvidia GPU, and then you need a special docker driver from nvidia. Here is a github project for packaging Blender in a docker package (the challenges are similar to packaging Curv): https://github.com/jtomori/docker_blender_gpu

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/curv3d/curv/issues/83#issuecomment-666351450, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC7IWCYI3HJUB6CZHPO7EZTR6FVVPANCNFSM4IMSYS3Q .

doug-moen commented 4 years ago

Thanks for the tip about using CUDA inside of docker. I may need to use that at some point.

mosheliv commented 4 years ago

My docker works nicely, just having problems with display but it works without it... Working on it...

On Sat, Aug 1, 2020, 10:17 Doug Moen notifications@github.com wrote:

Thanks for the tip about using CUDA inside of docker. I may need to use that at some point.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/curv3d/curv/issues/83#issuecomment-667409374, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC7IWCZVTOZSEUMV2HQTDMTR6M7HXANCNFSM4IMSYS3Q .

mosheliv commented 4 years ago

this is the docker file. I am running it on a remote headless server so it was a bit of a pain to get X to work but for everyone else it shouldn't be bad following https://medium.com/@l10nn/running-x11-applications-with-docker-75133178d090 or similar advice. I couldn't check it as I have no display connected to this host

FROM nvidia/cuda:10.0-runtime-ubuntu18.04                                                                                                                                

RUN apt-get update && apt-get install -y libxcursor-dev -y clang cmake git-core libboost-all-dev libopenexr-dev libtbb-dev libglm-dev gedit dbus-x11 \                   
        libxinerama-dev libxrandr-dev libglu1-mesa-dev libgles2-mesa-dev libgl1-mesa-dev libxi-dev                                                                       

RUN git clone --recursive https://github.com/curv3d/curv; cd curv; make; make install

to build: save the above in Dockerfile in an empty directory run : docker build -t curv . this should take some time and build the docker image to run curv with no graphics: docker run curv curv -x 2+2 if you want to run curv scripts, you probably need to do: docker run -v pwd:/work curv curv /work/myscript.curl

good luck!

yawor commented 4 years ago

+1 for AppImage for universal package. What's really important, user don't need to install anything on the system to use it. Most distributions do not ship with snap or flatpak managers/runtimes. I don't have any need to install them on my Arch Linux system. With AppImage I can just download and run an application if it's not available natively in Arch repos or it takes a long time to build it myself (like FreeCAD 0.19 for example). Docker may also be good, but as an additional format. It could be useful for continuous deployment on github for automatic builds on push and then deploy updated models to thingiverse or prusaprinters. The good thing about docker image is that, it's so easy to build it, anyone can do it if needed.

doug-moen commented 4 years ago

I agree that AppImage is the best choice for some users. I personally prefer AppImage for some of the software I use. Also, there can be more than one distribution method! Not everyone has the same requirements.

If someone wants to create a github actions script to build an AppImage for Curv, I would accept that as a contribution.

A-G-D commented 3 years ago

I just made an AppImage build script for curv and integrated it to github actions. I tested the build script locally (Linux Mint 20.2) as well as in github actions and was able to produce a working AppImage. Here is the latest build artifacts from github actions.

The AppImage doesn't contain all the standard metadata requirements yet - it is made to 'just work' for now but maybe it can already be a useful start.

@doug-moen , will you accept a PR?

doug-moen commented 3 years ago

@A-G-D Thanks! Please submit a PR.

doug-moen commented 3 years ago

Thanks to @A-G-D we now have a Curv AppImage build in github actions. Due to the nature of AppImage, this will only run on "sufficiently recent" versions of "major" distros. It may not work on niche distros or on older versions of popular distros. It doesn't work on Ubuntu 18.04, for example. The workaround is to upgrade your distro or compile your own binary.

Right now, the most universal app distribution format is Flatpak. Flatpak apps run on a wider range of Linux systems than AppImages because Flatpak has its own runtime, and doesn't rely on the base system having a sufficiently recent glibc etc. And Flatpak is supported by a wider range of distros than Snap (28 for Flatpak, vs 22 for Snap).

ivocavalcante commented 3 years ago

@doug-moen , while I have a lot of respect for the work you've done with Curv - I was looking for something with better performance than OpenSCAD, and Curv has that and much more - I really can't avoid myself from coming here and saying that your statements about Snaps are not true.

Nonetheless, I have absolutely no intention or desire for starting a package format debate. I've always felt uncomfortable in maintaining my Curv package given your opinions against Snap. This statement just reinforced this feeling, therefore I'll unlist the package from the store. Hope you can find time to create a Flatpak version yourself, or somebody jumps in and fill the gap.

doug-moen commented 3 years ago

Whoops, sorry @ivocavalcante, I should not have posted something inflammatory to the discussion. I edited the comment to make it more factual. And yes, it's true that I have allowed my antipathy towards Snap to colour my decisions in running the Curv project. Probably not the best idea, since I don't really want to insult and alienate people. Although I don't use Snap myself, I don't really want to prevent other people from using it. You don't need to unlist Curv from the Snap store on my behalf. For what it's worth, I will accept Snap-related PRs. The caveat is that I won't be testing these changes manually, due to my local setup, although it might possible to test using github actions.

lf94 commented 1 year ago

Can we close the issue?

curv3d / curv

universal Linux executable #83