Many build issues on FreeBSD

  MODE     production
realpath: illegal option -- -
usage: realpath [-q] [path ...]
find: illegal option -- t
usage: find [-H | -L | -P] [-EXdsx] [-f path] path ... [expression]
       find [-H | -L | -P] [-EXdsx] -f path [path ...] [expression]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'md5'
  CHECK    Configuration dependencies...
  GEN      out/config-cache.mk
  UPDATE   out/config-stamps.sha256
  MODE     production
realpath: illegal option -- -
usage: realpath [-q] [path ...]
find: illegal option -- t
usage: find [-H | -L | -P] [-EXdsx] [-f path] path ... [expression]
       find [-H | -L | -P] [-EXdsx] -f path [path ...] [expression]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'md5'
  CXX      out/sfi/testsfidl.o
  GEN      out/ebeast/vue-docs.md
  GEN      out/images/beast-logo.png
  GEN      out/images/beast-splash.png
  GEN      out/images/beast-mime.png
  GEN      out/images/beast.png
  GEN      out/images/bse-mime.png
  da:       726 translated messages, 80 fuzzy translations, 792 untranslated messages.
  ar:       0 translated messages, 932 fuzzy translations, 666 untranslated messages.
  az:       28 translated messages, 149 fuzzy translations, 1421 untranslated messages.
  bg:       507 translated messages, 95 fuzzy translations, 996 untranslated messages.
  cs:       1309 translated messages, 131 fuzzy translations, 158 untranslated messages.
  ca:       1342 translated messages, 118 fuzzy translations, 138 untranslated messages.
  en_CA:    1319 translated messages, 132 fuzzy translations, 147 untranslated messages.
  el:       178 translated messages, 524 fuzzy translations, 896 untranslated messages.
  eo:       82 translated messages, 62 fuzzy translations, 1454 untranslated messages.
  de:       1309 translated messages, 13 fuzzy translations, 276 untranslated messages.
  en_GB:    1430 translated messages, 90 fuzzy translations, 78 untranslated messages.
  es:       1395 translated messages, 95 fuzzy translations, 108 untranslated messages.
  fi:       500 translated messages, 68 fuzzy translations, 1030 untranslated messages.
  eu:       885 translated messages, 183 fuzzy translations, 530 untranslated messages.
  hr:       229 translated messages, 150 fuzzy translations, 1219 untranslated messages.
  fr:       1430 translated messages, 90 fuzzy translations, 78 untranslated messages.
  it:       1282 translated messages, 90 fuzzy translations, 226 untranslated messages.
  ja:       1183 translated messages, 198 fuzzy translations, 217 untranslated messages.
  nb:       231 translated messages, 37 fuzzy translations, 1330 untranslated messages.
  mn:       151 translated messages, 355 fuzzy translations, 1092 untranslated messages.
  oc:       177 translated messages, 25 fuzzy translations, 1396 untranslated messages.
  ne:       1330 translated messages, 124 fuzzy translations, 144 untranslated messages.
  nl:       1343 translated messages, 116 fuzzy translations, 139 untranslated messages.
  pa:       81 translated messages, 18 fuzzy translations, 1499 untranslated messages.
  pt:       814 translated messages, 293 fuzzy translations, 491 untranslated messages.
  ru:       715 translated messages, 228 fuzzy translations, 655 untranslated messages.
  pt_BR:    1184 translated messages, 169 fuzzy translations, 245 untranslated messages.
  sl:       1320 translated messages, 119 fuzzy translations, 159 untranslated messages.
  rw:       91 translated messages, 799 fuzzy translations, 708 untranslated messages.
  sq:       115 translated messages, 870 fuzzy translations, 613 untranslated messages.
  sr:       660 translated messages, 385 fuzzy translations, 553 untranslated messages.
  sv:       544 translated messages, 390 fuzzy translations, 664 untranslated messages.
  sr@Latn:  660 translated messages, 385 fuzzy translations, 553 untranslated messages.
  GEN      out/data/bse.pc
  GEN      out/po/intltool-merge.cache
  GEN      out/data/bse.mime
  te:       192 translated messages, 17 fuzzy translations, 1389 untranslated messages.
Traceback (most recent call last):
  File "./docs/filt-docs2.py", line 7, in <module>
  uk:       146 translated messages, 131 fuzzy translations, 1321 untranslated messages.
    from pandocfilters import toJSONFilter, Emph, Strong, SmallCaps, Para, Header
ModuleNotFoundError: No module named 'pandocfilters'
  GEN      out/data/beast.applications
  GEN      out/media/Demos
  GEN      out/media/Instruments
  zh_CN:    151 translated messages, 473 fuzzy translations, 974 untranslated messages.
  GEN      out/media/Effects
  GEN      out/media/Samples
  CXX      out/sfi/sfidl.o
  GEN      out/aidacc/IdStrings.py
  GEN      out/aidacc/Parser.py
  GEN      out/aidacc/TmplFiles.py
  FETCH    minizip-2.9.0
  GEN      out/bse/sysconfig.h
  GEN      out/bse/bseenum_arrays.cc
  GEN      out/external/minizip/mz_zip.h
unzip: Failed to open '2.9.0.zip'
gmake[1]: *** [bse/Makefile.mk:392: out/external/minizip/mz_zip.h] Error 1
gmake[1]: *** Waiting for unfinished jobs....
cat: out/bse/conftest_spinlock.txt: No such file or directory
rm: out/bse/conftest_spinlock.txt: No such file or directory
gmake[1]: *** [bse/Makefile.mk:515: out/bse/sysconfig.h] Error 1
^CError running filter docs/filt-docs2.py:
user interrupt
gmake[1]: *** [config-uname.mk:94: out/sfi/sfidl.o] Interrupt
gmake[1]: *** [config-uname.mk:94: out/sfi/testsfidl.o] Interrupt
gmake[1]: *** [ebeast/Makefile.mk:236: out/ebeast/vue-docs.md] Error 83
*** Signal 2

realpath is called with wrong arguments.
find is called with an invalid option.
md5 module doesn't exist in python.
FETCH from the internet isn't allowed during build.
Too many levels of symbolic links

Hi @yurivict.

I'm afraid FreeBSD builds are not possible atm.

1) FETCH for external packages is required, because we cannot have every dependency replicated in our repository. A cache in $HOME is supported though, so only the first build attempt really needs downloads.

2) Beast now has a hard dependency on Electronjs, and AFAICS that is not (yet) available for FreeBSD. So that needs to be fixed first. Alternatively, maybe some future version of Firefox will again support a standalone browser engine that can be used as desktop app, but without a standalone modern browser engine, Beast cannot be brought to live.

3) realpath and find and other tools used in the Makefiles use GNU specific options atm, porting to a pure POSIX command/option set is only useful once (2) is solved (and involves significant work in some cases).

I think this is the Electronjs upstream bug that would have to be fixed first;

https://github.com/electron/electron/issues/3797

FETCH for external packages is required

All major packaging systems disallow fetch during build for security reasons, rpm/debian/bsd

Beast now has a hard dependency on Electronjs

NodeJS is hard-banned by all major packaging systems because NodeJS leads to insecure packages and they just don't package such software.

You made several design decisions that prevent most users from ever seeing your software because you are asking them to compile it by hand instead of installing it via a package. Compilation by hand is too hard for most users.

FETCH for external packages is required

All major packaging systems disallow fetch during build for security reasons, rpm/debian/bsd

As I wrote under (1) above, you could prefetch the downloaded 3rd party sources, and combined with the beast sources + Electron could build Beast offline.

Beast now has a hard dependency on Electronjs

NodeJS is hard-banned by all major packaging systems because NodeJS leads to insecure packages and they just don't package such software.

First, NodeJS doesn't lead to insecure software per se. Blindly installing npm deps can have that effect, yes, but that's not what the Beast build does. (And realistically, the same can happen with other language environments also.)

Second, Beast doesn't even need the NodeJS portion of Electron, just a modern web engine to display the UI, which is why I wrote that Firefox could take on this part in the future also.

Third, this is not the place to start a generic argument against use of Electron. If you care, https://electronjs.org/apps lists hundreds of apps that are based on Electron. It may take distributors some time - or years - to figure how and why they want to package Electron apps, but it will happen eventually due to user demand.

You made several design decisions that prevent most users from ever seeing your software because you are asking them to compile it by hand instead of installing it via a package. Compilation by hand is too hard for most users.

You're right that compilation is too hard for users. We don't advocate that users should compile Beast to get it working. Instead, we provide a pre-built AppImage, that works on Ubuntu and Fedora systems. Distributors/packagers are welcomed to build Beast for platforms beyond that, and that process is actually quite easy, as GNU-Make will tell you what dependencies are missing and you could even inspect the CI Docker images to understand how the build works.

Regarding the basic design choices that you talk about, we originally based Beast on Gtk+ and I invested heavily into that toolkit. As a result, ten years ago we had Debs, Rpms and earlier even BSD packages. The foundation wasn't good enough to keep up with feature requests and UI development demands though.

Changing that has allowed to move Beast into new and more interesting directions, and that at a previously unimaginable pace. For now, the focus is on reaching feature completeness for a 1.0 version, and once that's achieved we can possibly dial back on one or two aspects to ease packaging and integration. For FreeBSD specifically, a proper desktop application webengine is required first - the need of which is also recognized by other FreeBSD users as demonstrated in https://github.com/electron/electron/issues/3797

It may take distributors some time - or years - to figure how and why they want to package Electron apps, but it will happen eventually due to user demand.

Electron apps package NodeJS code. npm downloads NodeJS code directly from GitHub, typically from hundreds of individual GitHub accounts. This subjects users to the danger of some of these accounts to go rogue and deliver malware to them, since NodeJS technology doesn't have any safeguards against this and such unsafe behavior is done rather by its design, there's little chance that major packaging systems would adopt them. You can see that the Atom editor for example isn't packaged by Debian or any RPM packaging systems (https://repology.org/project/atom/versions). They can't subject their users to such dangers, and they shouldn't.

Perhaps Electron can be used w/out NodeJS, but it brands itself as ElectronJS, and your project too has the npm part in it.

Electron apps package NodeJS code.

Electron contains the google-chrome rendering engine libchromiumcontent, the google-chrome javascript engine V8 (sandboxed) and nodejs (a V8 based JS language environment, not sandboxed) in a single binary. That is similar to e.g. Firefox and and any language interpreter like Python or the JVM languages.

npm downloads NodeJS code directly from GitHub, typically from hundreds of individual GitHub accounts.

npm is a package manager and package repository for web applications and nodejs applications. packages are downloaded from npmjs.org. web applications are sandboxed like a website displayed in a browser. only nodejs applications and npm packages for nodejs could go rouge the way CPAN packages, PIP modules, JVM code, C, Go, Ruby, C++, Rust or shell code could after being downloaded. There is nothing special about npm here, other than it being used more widely, so its potentially a more attractive target. It has had dependency fallouts in the past, which is why procedures have been adapted now to prevent this in the future and make package name spoofing harder. Most Python, C, C++, etc code is hosted on Github these days btw, but whether Github is used for hosting or not is really not related.

This subjects users to the danger of some of these accounts to go rogue and deliver malware to them, since NodeJS technology doesn't have any safeguards against this and such unsafe behavior is done rather by its design, there's little chance that major packaging systems would adopt them.

I don't follow that argument, Electron has sandbox functionality, C, C++, Ruby, Python, Haskell, etc code that is pulled from Github and packaged doesn't.

Perhaps Electron can be used w/out NodeJS, but it brands itself as ElectronJS, and your project too has the npm part in it.

That is just not related. Most websites use npm packages and are used by billions of people, every day. Claiming that npm is bad or insecure or unusable or anything alike is just not credible. Regarding Beast, the package installation uses just one npm package, vue-2.6, and that comes with 0 dependencies. We used to also need jquery but could get rid of it. And I have probably studied more of the Vue code than I have studied the various C/C++ dependencies we pull in (those are pulled from Github btw).

@yurivict wrote:

This subjects users to the danger of some of these accounts to go rogue and deliver malware to them, since NodeJS technology doesn't have any safeguards against this

There is no point in picking on npm (nodejs) specifically when it comes to malicious code being introduced via dependencies. Whatever language / runtime environment you use, always check your dependencies closely and pay close attention to name spoofing / typosquatting.

there's little chance that major packaging systems would adopt them. You can see that the Atom editor for example isn't packaged by Debian

FYI, here is the Debian bug for packaging Electron: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842420

And here is the Wiki page tracking the progress: https://wiki.debian.org/Javascript/Nodejs/Tasks/electron

FYI, here is the Debian bug for packaging Electron: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842420

There's nothing wrong with Electron itself. It has been ported to FreeBSD and is long available in ports. The problem is that Electron is used as a Trojan Horse to drive NodeJS packages.

Whatever language / runtime environment you use, always check your dependencies closely and pay close attention to name spoofing / typosquatting.

No. Other projects have a more centralized nature with upstream devs having control over the content of used dependencies. In NodeJS npm just downloads the latest versions of hundreds/thousands of GitHub projects without anybody being able to even track what versions are used n particular cases. There is no easy way to freeze dependencies, to have reproducible builds, to fingerprint files, etc. This creates an ecosystem prone to security violations.

tim-janik / beast

Many build issues on FreeBSD #132