Running Shoes on FreeBSD

[IanTrudel]

There are two obstacles for running Shoes on FreeBSD:

1. Building Shoes on Linux instructions are a little light to adapt to FreeBSD. Might be easy, might be hard.
2. FreeBSD provides a very good Linux compatibility where Shoes for Linux may work right out-of-the-box but it doesn't like the Makeself installer. Would it me possible to get a tarball instead? @ccoupe

[ccoupe]

Now compiling llvm/clang ... Plenty of time time to think. I'll need to di the x11 and a WM asetup and probably a bunch of other stuff but DO I WANT that linux compatibly? Not yet. I'll want build ruby from source and see what RUBY_PLATFORM is. The Shoes rake file setup is pretty flexible. FWIW, I dont like asking freebsd users to download and install linux stuff - although freebsd users must be used to conflicts so they aren't "newbies".

[IanTrudel]

Nobody will complain about a native FreeBSD version of Shoes. :)

Though at least we got Linux version working. It's a start.

You may enjoy FreeBSD Handbook and FreeBSD Porter's Handbook. FreeBSD is known for its extensive documentation.

[ccoupe]

After way too many re-installs, reboots and head scratching I do have a freebsd/xfce that seems to be stable in virtualbox. I tried a gnome3 installation first - gawd, I do hate that desktop. Bad Cecil!

Next step is to get an nfs file system mounted through virtualbox. Note to self: When upgrading ubuntu - pick the xfce sub-distro.

[IanTrudel]

I did the exact same thing. Turns out XFCE is outdated and hardly maintained. Now running on MATE and it's great. Probably something like pkg install mate should do the trick.

[ccoupe]

Turns out XFCE is outdated and hardly maintained

Good. They won't screw it up in the future! I don't need a lot. Geany is installed and a couple of terminals is all I that I need.

I built a ruby 2.3.4 and put it in /opt with this build script

$ more bsd-ruby.sh 
#! /usr/bin/env bash
# copy this to the ruby dir or soft link it.
# execute it instead of ./configure
export dest="/opt"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --enable-shared \
  --enable-load-relative \
  --disable-install-doc \
  --without-tk --without-tcllib --without-tcltk \
  --prefix="${dest}"
$ 

Missing gdbm and readline but we don't use them. Still, they could and should be there. Builds pretty fast.

If only my PATH had the mojo to find pkg_config and gcc.

[IanTrudel]

The big question is are you using libraries provided in FreeBSD ports? This is definitively the preferred way on FreeBSD. I suspect that all the libraries are almost readily available in the ports to be compiled for Shoes. May need some configuration but that's it.

[ccoupe]

So far I'm just using pkg install <x> I'm using the default /bin/sh and only changed PATH to add the /opt/bin to find ruby and that does work (ruby and rake). What pkg install do I iinvoke to get gcc and pkg-config installed ? Some how the the ruby build knows how to do that.

[IanTrudel]

You don't need to do all that. Are you trying to install ruby in a different directory? man pkg is your friend.

   -r ⟨root directory⟩, --rootdir ⟨root directory⟩
             pkg will install all packages within the specified ⟨root
             directory⟩.

Take note that when you compile ports, they all go in a sandbox directory /usr/ports/distfiles. So let's say a port depends on ruby, compiles it but does not install it, the port will use the compiled files in /usr/ports/distfiles/lang/ruby or something.

What pkg install do I iinvoke to get gcc and pkg-config installed ?

Everything is installed by default to facilitate building ports including cc. Normally pkg-config comes along gtk as you already know. So it should be all there for you. Otherwise take a look into the ports (rubygem-pkg-config is available as binary but pkg-config is not, though it should still be on your system).

/usr/ports$ make search name=pkg-config
Port:   rubygem-pkg-config-1.2.7
Path:   /usr/ports/devel/rubygem-pkg-config
Info:   pkg-config implementation for Ruby
Maint:  ruby@FreeBSD.org
B-deps: indexinfo-0.2.6 libedit-3.1.20170329_2,1 libexecinfo-1.1_3 libffi-3.2.1_1 libunwind-20170113_1 libyaml-0.1.6_2 pkgconf-1.3.7,1 ruby-2.3.4_2,1 ruby23-gems-2.6.12
R-deps: indexinfo-0.2.6 libedit-3.1.20170329_2,1 libexecinfo-1.1_3 libffi-3.2.1_1 libunwind-20170113_1 libyaml-0.1.6_2 ruby-2.3.4_2,1 ruby23-gems-2.6.12
WWW:    https://github.com/ruby-gnome2/pkg-config

Port:   devel/pkg-config
Moved:  devel/pkgconf
Date:   2012-07-26
Reason: pkg-config has been replaced by pkgconf

In a FreeBSD world, we would have to create a new port for Shoes and use other ports as dependencies. The Shoes port Makefile would contain whatever dependencies and flags that need to be passed down to them (such as ruby) and so on.

[ccoupe]

Progressing after pkg install devel/pkgconf only to hit can't find 'pty.h'. Sigh. If I remember, bsd and sysv differ in their terminal handling. So tesi.c may need some help. It was going good up till then.

[IanTrudel]

You refer to libungif in your last commit. Please, take note that it is mentioned giflib supersedes libungif on FreeBSD.

[ccoupe]

Ungif was not the problem - fortunately. It's working if you don't load any shoes native gems. Really close.

[ccoupe]

I've got something that works well enough to do more testing.

$ ./shoes --ruby -e 'puts RUBY_PLATFORM'
x86_64-freebsd11.1
$ 

There is Shoes ruby code that needs updating for that answer (image.rb, download.rb for two). I've got a tgz of the binary. There are issues related to the newer gtk3 version that freebsd 11.1 provides.

If you want to build from source then clone branch 'bsd'. You'll really, really want the pre-built gems . Of course you need a ruby built with --enable-load-relative but that isn't too difficult and the script is up-thread. Lastly, you'll need a x86_64-bsd-custom.yaml file. Mine is

Ruby: /opt
Deps: 
Gemloc: /usr/home/ccoupe/gems/rb23
Extloc: /usr/home/ccoupe/gems/rb23
Exts:
Gems:
InclGems: 
  - chipmunk-6.1.3.4
  - sqlite3-1.3.13
  - mini_portile2-2.2.0
  - nokogiri-1.8.0
  - rb-readline-0.5.4
  - byebug-9.1.0
  - ffi-1.9.18
  # picky needs:
  - activesupport-5.1.3
  - concurrent-ruby-1.0.5
  - i18n-0.8.6
  - multi_json-1.12.1
  - picky-4.31.3
  - rack_fast_escape-2009.06.24
  - thread_safe-0.3.6
  - tzinfo-1.2.3
  - url_escape-2009.06.24
  - yajl-ruby-1.3.0
  # typhoeus
  - ethon-0.10.1
  - typhoeus-1.1.2
Debug: true

Known bugs - gtk whines about 'Allocating size to GtkLabel', native gems may not compile from Shoes, ssl is untested but expect problems.

[IanTrudel]

It's a very good first round. The native version is notably faster than Linux. Samples are working in general, including funnies. There are few quirks when playing around samples and it crashed on me with downloader and video-player (no vlc installed) but it's similar to Windows behaviour (non repeatable crashes when messing around).

[ccoupe]

New beta (tgz) to match target name change and the gems need a new name to match. I also merged branch bsd into branch rb23 which is closer to master.

Recent commit added the build output showing the gtk3 deprecation issues in freebsd. It is possible to test the gtk version when compiling so it's possible we could write substitutions someday.

[ccoupe]

I've got makeself (sudo pkg install makeself) working to create a freebsd.install - just like linux has. My freebsd window manager doesn't support the xdg-desktop so no menus for me but Shoes is installed in ~/.shoes/walkabout so, rake package works. I don't have a master build environment that can handle a weekly build (pita - much to test and much to go wrong) so it's still the red-headed child of shoes. Download here

[IanTrudel]

The installer works like a charm and FreeBSD native binary seems up to par. Spinner works. Did few other tests here and there. I'm not a big fan of Shoes in menu education category. Cobbler still reports x86_64-freebsd11.1 instead of FreeBSD. Pretty good in any case. You did great @ccoupe !

[dredknight]

@BackOrder unless you have any complex app I can drop you my skillwheel somewhere to use it for tests.

[IanTrudel]

@dredknight it sounds like a very good idea.

[ccoupe]

x86_64-freebsd11.1 is Ruby's choice (probably via uname) which means they (freedos) or the user has to change it somehow. rake x:setup:y really just means 'use the files in make/x/y/'

There are windows mangers without a visible Programming menu. The user can delete the one they don't like. I wonder how Geany set a menu on XFCE w/o xdg-desktop? perhaps freedos can handle both in the ports?

[IanTrudel]

RE: Ruby/x86_64-freebsd11.1

Alright.

RE: Menu

Programming menu entry is fine (and makes sense). The Education menu entry is the one that I am not fan of.

RE: freedos

Are you really using FreeDOS? haha

I wonder how Geany set a menu on XFCE w/o xdg-desktop?

Do you mean xdg-desktop-menu ? Seems to be working here on MATE desktop.

[IanTrudel]

perhaps freedos can handle both in the ports?

Eventually we should have a Shoes port. This is the best way to deliver an app to every FreeBSD user.

[ccoupe]

Eventually we should have a Shoes port

Yeah, been thinking about how to it. virtualbox-cli startup from a script. wait 60 secs or so then login. The VM would need port forwarding so I can log in via ssh and pass over of the command lines to the build. It's very Similar to how I build the Pi version. I just have to remember/relearn the ssh chants.

BTW and slight off topic, I'm thinking of dropping the 32 bit linux build and the osx 10.9 build (10.9 is a real pain for me to maintain - although the work for getting 10.10 back to working is not negligible)

[IanTrudel]

RE: building port on FreeBSD VM

You already have an ecosystem that you are familiar with. This should work well. The port itself shouldn't be too much troubles (creating a one-time Makefile [download/set other ports as dependencies/patching], SHA256 on the fly, perhaps a couple of other things, upload to FreeBSD repo when ready to release).

RE: Linux 32-bit and OSX 10.9

I thought you had already dropped Linux 32-bit support. Nowadays even embedded devices run 64 bit. So it should be fine dropping 32-bit support entirely. You still can offer a legacy download on the main website.

MacOS X on the other hand is a little more complicated question. My first reaction would be a straight no but it seems Apple has been pushing macOS Sierra aggressively including offering it for free to existing MacOS X users.

The following chart is dating back 2016. source

The market share as of today says MacOS X 10.9 gets 0.26% but 10.10 (0.72%), 10.11 (1.09%) and 10.12 (3.59%). So it would seem that keeping MacOS X 10.9 support for another year would be in order.

[ccoupe]

On my Linux host I have this ~/bin/nbuild-shoes-bsd script

#!/bin/bash
VBoxHeadless --startvm "freebsd 11.1" &
echo "waiting"
sleep 40
echo "logging in"
ssh -p 3022 ccoupe@127.0.0.1 <<'ENDSSH'
echo "Build Shoes FreeBSD"
export PATH=/opt/bin:$PATH
export NFS_ALTP=/usr/home/ccoupe/build/
cd Projects/shoes3
pwd
echo $PATH
rm -rf build-target
rake bsd:setup:freebsd
rake
rake package
cp /usr/home/ccoupe/build/pkg/*freebsd*.install pkg
rake clobber
ENDSSH
ls -ld /home/ccoupe/Projects/shoes3/pkg/*freebsd*.install
VBoxManage controlvm "freebsd 11.1" poweroff

And it works. Once. The second attempt suggests that Rakefile.CLOBBER is incomplete. Close, but not there yet.

[dredknight]

I thought you had already dropped Linux 32-bit support.

True. We do not have 32 bit linux systems as well. Hek even windows 32 bits is less than 0.1% compared to the total sum of devices and they are mainly used for deprecated software.

@BackOrder here you go. Dump the archive into a folder and run it from main.rb. I still have not found a way to compact all the files so it make take sometime to extract it.

[IanTrudel]

Skillwheel is not exactly NIX friendly. Filenames are most often incorrect. For example, require 'code/Tooltip' but the file is named tooltip on the filesystem. Same goes with various images and such ¬ some files on disk start with a capital letter (or not) and the file in the ruby code is the other way around. NIX and MacOS X are all case sensitive. Take a look at the startup screen below.

Feedback for @ccoupe ¬ opening any file using Shoes main window will cause the following error in Shoes Console. It also opens two windows.

Error in <unknown> line 0 | 2017-09-04 04:39:03 -0400
wrong number of arguments (given 0, expected 1..3)
eval:1:in `instance_eval'
eval:1:in `block in <main>'
/usr/home/ian/.shoes/walkabout/lib/shoes.rb:138:in `app'
/usr/home/ian/.shoes/walkabout/lib/shoes.rb:138:in `ask_open_file'
/usr/home/ian/.shoes/walkabout/lib/shoes.rb:138:in `show_selector'
/usr/home/ian/.shoes/walkabout/lib/shoes.rb:167:in `block (4 levels) in splash'

[dredknight]

@BackOrder that sucks big time... Here you go a version where all (i hope so) cases are fixed. If there are still issues and they are not that much could you paste me a log so I can resolve them?

[IanTrudel]

@dredknight not even starting anymore.

Error in <unknown> line 0 | 2017-09-04 14:49:45 -0400
main.rb:90: invalid multibyte char (US-ASCII)
main.rb:90: invalid multibyte char (US-ASCII)
main.rb:90: syntax error, unexpected end-of-input, expecting ')'
...G}/heroes/#{ch_heroes[hero]}/аdditional.txt")[0]
...                               ^
/usr/home/ian/.shoes/walkabout/lib/shoes.rb:351:in `eval'
/usr/home/ian/.shoes/walkabout/lib/shoes.rb:351:in `visit'
eval:1:in `<main>'

[IanTrudel]

@dredknight you can create a separate issue and we can work out the problems.

[dredknight]

@BackOrder thanks actually I think I found it. For some reason some of the files have weird hidden characters. By retyping them by hand the issue disappears. What I do not understand is why do you see that error on FreeBSD but not on Windows. I will give you another version of the app once it is fixed.

[IanTrudel]

invalid multibyte char (US-ASCII)

That would be it.

[dredknight]

Thanks for the help! I do not understand how such thing can just appear on 2 random places (once in the code and once in a file) without reason. Never happened before.

Here is a working version of the wheel with fixed letters cases. Let me know how it goes !

[IanTrudel]

@dredknight no problem, mate ! Your app is undoubtedly cool. :)

So there are still several problems. There are most likely more. I would recommend to lower case everything both in the filesystem and the code. See if it helps.

Error in <unknown> line 0 | 2017-09-04 16:46:50 -0400
No such file or directory @ rb_sysopen - text/en/factions/haven/Name.txt
main.rb:51:in `read'
main.rb:51:in `block (2 levels) in set_main'
main.rb:50:in `clear'
main.rb:50:in `block in set_main'
main.rb:49:in `each'
main.rb:49:in `each_with_index'
main.rb:49:in `set_main'
main.rb:252:in `hero_pane'
main.rb:645:in `block in <main>'
eval:1:in `instance_eval'
eval:1:in `block in <main>'

Error in <unknown> line 0 | 2017-09-04 16:47:51 -0400
Shoes could not find the file pics/heroes/Unknown.png.

Error in <unknown> line 0 | 2017-09-04 16:47:51 -0400
Shoes could not find the file pics/heroes/Unknown.png.

Error in <unknown> line 0 | 2017-09-04 16:47:54 -0400
Shoes could not find the file pics/heroes/Unknown.png.

Error in <unknown> line 0 | 2017-09-04 16:48:14 -0400
No such file or directory @ rb_sysopen - text/en/factions/Neutral/name.txt
main.rb:352:in `read'
main.rb:352:in `creature_pane_2_book'
main.rb:347:in `creature_pane'
main.rb:621:in `block (2 levels) in main_menu'
main.rb:352:in `read'
main.rb:352:in `creature_pane_2_book'
main.rb:347:in `creature_pane'
main.rb:621:in `block (2 levels) in main_menu'

Error in <unknown> line 0 | 2017-09-04 16:48:22 -0400
Shoes could not find the file pics/spells/STONE_SPIKES.png.

Error in <unknown> line 0 | 2017-09-04 16:48:22 -0400
Shoes could not find the file pics/spells/LIGHTNING_BOLT.png.

Error in <unknown> line 0 | 2017-09-04 16:48:22 -0400
Shoes could not find the file pics/spells/STONESKIN.png.

[dredknight]

There is that challenge that some of the strings I use to get the files are also used as filters so they are required to be semi-upper or full upper case. I never thought this can create such a void of problems.

yet another version

[IanTrudel]

Here are some more...

Error in <unknown> line 0 | 2017-09-04 17:14:08 -0400
Shoes could not find the file pics/spells/ICE_BOLT.png.

Error in <unknown> line 0 | 2017-09-04 17:14:08 -0400
Shoes could not find the file pics/spells/FROST_RING.png.

Error in <unknown> line 0 | 2017-09-04 17:14:18 -0400
Shoes could not find the file pics/spells/FORGETFULNESS.png.

Error in <unknown> line 0 | 2017-09-04 17:14:18 -0400
Shoes could not find the file pics/spells/WASP_SWARM.png.

Spells > World of light (also Teleportation, Divine Vengeance, Resurrection, First of Wrath, Vessel of Shalassa, Town Portal, Instant Travel)

Error in <unknown> line 0 | 2017-09-04 17:14:34 -0400
invalid byte sequence in US-ASCII
main.rb:11:in `[]'
main.rb:11:in `block in read_skills'
main.rb:7:in `each_line'
main.rb:7:in `read_skills'
main.rb:447:in `spell_pane_effect'
main.rb:430:in `block (3 levels) in spell_pane_pages'

All the artifacts are problematic (images are all caps).

[ccoupe]

So it would seems that keeping MacOS X 10.9 support for another year would be in order.

Or until I have to update the 10.9 deps to include a new library. It's a gruesome process now.

I did trace my build problem to how rake clobber behaves linux vs bsd. Could be I did something unspecified or it could be a bug in rake. Easy workaround in the shell script. I'll wire freebsd into the app packing code and this issue will be finished.

[ccoupe]

The port itself shouldn't be too much troubles (creating a one-time Makefile [download/set other ports as dependencies/patching], SHA256 on the fly, perhaps a couple of other things, upload to FreeBSD repo when ready to release).

You might have noticed that Shoes does not have linux distro packaging (.deb, .rpm, and what ever suse and arch use), It won't have a bsd port either. For the same reason - too many hurdles for me. Yes, getting listed at all the popular places would increase uptake (as would paying for Windows 10 and OSX developer certificates and paid membership in their developers "programs"). Not for me.

A more interesting side project, @BackOrder would be to clone the linux-shoes so developers could create a freebsd port of their app with Shoes hidden from view. Currently only supports deb and rpm but theiremight be port support - uses gem npm. Add in the gui work @dredknight and I did on the exe-shoes GUI and you might have something freebsd developers would like.

[dredknight]

@BackOrder , try this one. Artifacts should be working now. I tried some fixes on the spells but I am not very sure as there are called from two places.

[IanTrudel]

@dredknight

The following still needs fixing...

Spells > Dark magic > Cures of Netherworld Spells > Light magic > World of light, Teleportation, Resurrection Spells > Summoning magic > Fist of Wrath Spells > Adventure magic > Vessel of Shalassa, Town Portal, Instant Travel

Also, if you change language for polish (not found) in settings, it will render the whole thing UI unsuable. If polish is not present, it shouldn't be listed. You could list languages according to directories found in text directory.

[dredknight]

Thanks @BackOrder that is a good idea . I actually removed the polish folder before sending you the release so no errors from those should be presented. I forgot it will crash it.

Here you go this should work -> fixed version (do not use Polish it will still crash xD)

[IanTrudel]

The images seem fine but there are some other problems (all the list above). The following is the error list from Spells > Light magic > World of light, Teleportation, Resurrection.

Error in <unknown> line 0 | 2017-09-05 17:47:58 -0400
invalid byte sequence in US-ASCII
main.rb:11:in `[]'
main.rb:11:in `block in read_skills'
main.rb:7:in `each_line'
main.rb:7:in `read_skills'
main.rb:448:in `spell_pane_effect'
main.rb:431:in `block (3 levels) in spell_pane_pages'
main.rb:11:in `[]'
main.rb:11:in `block in read_skills'
main.rb:7:in `each_line'
main.rb:7:in `read_skills'
main.rb:448:in `spell_pane_effect'
main.rb:431:in `block (3 levels) in spell_pane_pages'
main.rb:11:in `[]'
main.rb:11:in `block in read_skills'
main.rb:7:in `each_line'
main.rb:7:in `read_skills'
main.rb:448:in `spell_pane_effect'
main.rb:431:in `block (3 levels) in spell_pane_pages'

Error in <unknown> line 0 | 2017-09-05 17:48:00 -0400
undefined method `*' for nil:NilClass

Error in <unknown> line 0 | 2017-09-05 17:48:02 -0400
invalid byte sequence in US-ASCII
main.rb:11:in `[]'
main.rb:11:in `block in read_skills'
main.rb:7:in `each_line'
main.rb:7:in `read_skills'
main.rb:448:in `spell_pane_effect'
main.rb:431:in `block (3 levels) in spell_pane_pages'

[ccoupe]

The numerous (overwhelming) GTK warnings started in Gtk 3.20.0 and supposedly there is a patch that some linux/bsd distributions may or may not apply in a timely manner. Gtk 3 actively discourages pixel counting layout like Shoes does in favor of their own container/constraint layout manager scheme. (OSX/Cocoa encourages this too). Accommodating that is not a maintenance effort - it's rewrite of shoes layout and likely the Shoes dsl/language. Time that would be better spent converting to QT based system? Yes, I am unhappy with gtk3 and wish I never moved Shoes to to it but here we are.

[dredknight]

@BackOrder could you try 2 things for me ? Open main.rb and remove the first line that says - # coding: utf-8. Try again. Then if it still spits the same errors ( very probable) insert the following code at the very beginning of the file.

~~ export LANGUAGE=en_US.UTF-8 export LANG=en_US.UTF-8 export LC_ALL=en_US.UTF-8 ~~ P.S. Scratch that, the main.rb does not even start with those settings. I have to dig deeper not sure what is the issue that cause these errors..