search

chaosforgeorg / doomrl

D**m, the Roguelike
https://drl.chaosforge.org/
GNU General Public License v2.0
1.08k stars 131 forks source link

How to make the GNU/Linux binary? #3

Closed mdtrooper closed 1 year ago

mdtrooper commented 7 years ago

I have been trying with:

$ fpcmake
Processing Makefile.fpc
 x86_64-linux requires: 
 Writing Makefile
$ make
Makefile:197: *** The Makefile doesn't support target x86_64-all, please run fpcmake first.  Stop.

And with:

$ bash mk_unix_dist.sh 666 linux ""
cp: cannot stat ‘../bin/musicmp3.lua’: No such file or directory
cp: cannot stat ‘../bin/musiccdmp3.lua’: No such file or directory

But it is not.

mdeguzis commented 7 years ago

I've been hunting this down as well. Backporting a few package to SteamOS so I can make full use of fpc/lazarus. From what I understand so far, you need to use lazarus, with fpc included, to do this.

See: https://github.com/ChaosForge/fpcvalkyrie/issues/2

subtract1 commented 7 years ago

I was unable to build on Linux. I can build on Windows. But I can still help you past this bit.

You're missing the regular DoomRL resources--sounds and music. Those aren't on this repository; you'll need to grab an actual copy of DoomRL and move the resources over. You can build the engine without them but it won't get far.

I don't usually build with the scripts; I've always just loaded the relevant project in Lazarus and built from there.

CyberKiller40 commented 7 years ago

Hi

I helped with Linux builds for AliensRL and DiabloRL in the past, these games use more or less the same engine and you'll find my notes on building for Linux on the command line (without the Lazarus IDE) in their docs. Hopefully this will give you something to start here with DRL.

Note: for DRL you'll want to convert the music (and sounds) to vorbis and modify the config files accordingly - mp3 support in SDL is usually nonexistent on GNU/Linux. Be sure to drop the embedded covers in the files though, otherwise e.g ffmpeg will convert them as theora and these won't work, only plain ogg vorbis works.

mdeguzis commented 7 years ago

hmm sounds like fun :) I'll see if I can hack away at this on the weekend. Thank you @CyberKiller40 . I assume you mean the GitHub repositories for those projects. avconv of ffmpeg can handle the ogg conversion fine I believe (as a separate step of course). I hope to get a native build for SteamOS/Ubuntu up next week possibly. Backporting Lazarus to SteamOS (yes I know it's mainly Jessie, but I prefer a proper backport) is giving me a headache. If it continues long enough I'll just snag the jessie-backports binaries.

CyberKiller40 commented 7 years ago

I'm not sure if they're on github, last time they were on sourceforge, or somewhere else. But the Linux notes were AFAIR included with the binaries so you'll get them with the official downloads. And like I said - you don't need Lazarus, only the freepascal compiler which should be in your distro's repos.

As for audio conversion - ffmpeg handles it fine, but if you get a resulting ogg with vorbis+theora inside, then the game will crash. Only ogg with just vorbis is working so you need to add a switch to ffmpeg to drop the video part (-vn). You can use my script for this: http://paste.opensuse.org/88656780 (a nicer script which downloads the original ogg versions of music files can be found on somewhere the game's forums - you won't lose the quality then)

found it: https://sourceforge.net/p/diablorl/code/HEAD/tree/trunk/COMPILING.linux - you'll need to rework this for DRL, but it should give you something to start

mdeguzis commented 7 years ago

Thanks!

mdeguzis commented 7 years ago

Unfortunately I get the same error as OP regarding x86-64-all. Yes I did not add in the music, but the compilation doesn't even start, which I would think would at least be a good initial test.

Steps to reproduce:

apt-get install build-essential fpc lua5.1 liblua5.1-0-dev
git clone https://github.com/ChaosForge/doomrl
cd doomrl
git clone https://github.com/ChaosForge/fpcvalkyrie
FPCDIR=/usr/lib/fpc/2.6.4/ fpcmake
make

Makefile produced: http://paste.debian.net/901237/

I may be able to work out what the Makefile is doing wrong. Maybe :)

This is where it is evaluated:

ifeq ($(findstring $(FULL_TARGET),$(MAKEFILETARGETS)),)
$(error The Makefile doesn't support target $(FULL_TARGET), please run fpcmake first)
endif

The full target alludes to: x86_64-all, where as the string set at the top of the Makefile is MAKEFILETARGETS=x86_64-linux Running OS_TARGET=linux make gets you a bit further, so I believe it will take a few adjustments to the Makefile. In reading the man page for fpc make, I noticed this:

-r If this option is given, fpcmake will recursively scan subdirectories for src/makefile.fpc files and generate makefiles for them.

makefile.fpc is what the src dir has. Now I hit a snag, where processing src/makefile.fpc requires 'rtl` which I am trying to find the package/package config for.

epyon commented 7 years ago

I never built using makefiles. Building for Linux was done using the makefile.lua (custom build script). AFAIR, you have to have lua in path (possibly 5.1), and run "lua makefile.lua lq" or "./makefile.lua lq" - lq can be substituted for hq or all.

Also copy config-linux.lua to config.lua, edit it and replace the full path to VALKYRIE_ROOT to the path of your FPC Valkyrie sources, with the trailing slash.

mdeguzis commented 7 years ago

Ok this gets farther. I'll try the lua approach.

apt-get install build-essential fpc lua5.1 liblua5.1-0-dev
git clone https://github.com/ChaosForge/doomrl
git clone https://github.com/ChaosForge/fpcvalkyrie
cd doomrl
FPCDIR=/usr/lib/fpc/2.6.4/ fpcmake -r
OS_TARGET=linux make

Output for fpc attempt: http://sprunge.us/ObKY

epyon commented 7 years ago

I'm pretty sure you need FPC 3.0.0 for the current sources.

mdeguzis commented 7 years ago

Well.. I just happened to backport 3.0.0 to my SteamOS repo :) But it didn't make a difference. I'll try with a fresh clone.

FPCDIR=/usr/lib/fpc/3.0.0/ fpcmake -r
OS_TARGET=linux make
subtract1 commented 7 years ago

I'm rooting for you md. Right now I have to test things through Wine...

I ran through lazarus directly on Mint. There were no x86 binaries in the repo and I got stuck on an 'internal error' so I'm stuck waiting for someone else to figure it out.

mdeguzis commented 7 years ago

Ok, installed a crapton of fp-units packages after trying some of the easy to spot units packages,. But, after having no luck, I used the verbose options of fpc to dig into the one pascal file unit failure.

fpc -vut src/doomrl.pas: log file

As you can see, tons of the units I installed are found, but not vpkg or vsystems. Still checking out this path of reasoning here.

Searching file src/doomrl.pas... found
(DOOMRL)   Registering new unit SYSUTILS
(DOOMRL)   Registering new unit VSYSTEMS
(DOOMRL)   Registering new unit VDEBUG
(DOOMRL)   Registering new unit DOOMBASE
(DOOMRL)   Registering new unit DFOUTPUT
(DOOMRL)   Registering new unit VLOG
(DOOMRL)   Registering new unit VUTIL
(DOOMRL)   Registering new unit VOS
(DOOMRL)   Registering new unit DFDATA
(DOOMRL)   Registering new unit DOOMMODULE
(DOOMRL)   Registering new unit DOOMNET
(DOOMRL)   Registering new unit DOOMIO
(DOOMRL)   Load from DOOMRL (implementation) unit SYSUTILS

Since some units, such as rtl are found just fine, and the fact I could not find vpkg/vsystems anyway via locate, I think there is just some missing piece of the puzzle or a package is missing entirely to provide these two units.

I believe this is a result of a missing path to fpcvalkyrie, even though it is in the path above doomrl/, as vsystems unit code is found in fpcvalkyrie. This folder designation is defined in the lua makefile, so I do agree with the above that using lua to kick off the compilation is proper for the project.

epyon commented 7 years ago

Did you use makefile.lua as suggested? It runs fpc with the proper paths.

mdeguzis commented 7 years ago

I was just about to write that lol. lua makefile.lua lq would not pick up LINUX as the OS. Is there a verbose output for lua? I didn't see it in the man page. $HOME expands to /home/desktop.

mdeguzis commented 7 years ago

I tried OS=LINUX lua makefile.lua lq as well. That results in this: 

Free Pascal Compiler version 3.0.0+dfsg-9~bsos+1 [2016/12/08] for x86_64
Copyright (c) 1993-2015 by Florian Klaempfl and others
vutil.pas(798,1) Error: Can't create object file: tmp/vutil.o (error code: 2)
vutil.pas(798,1) Fatal: Can't create object tmp/vutil.o
Fatal: Compilation aborted
Error: /usr/bin/ppcx64 returned an error exitcode
Autodetected OS - LINUX
epyon commented 7 years ago

did you copy config-linux.lua to config.lua and change the VALKYRIE_ROOT to your Valkyrie path? Also, might be problem of write'ability - create a tmp folder in doomrl root (ergo bin, tmp and src)

mdeguzis commented 7 years ago

I'll keep a WIP local build script here that you can reference as I test this. set -x is in effect, which produces the log like this: http://sprunge.us/RihO , versbose: http://sprunge.us/PXSW

mdeguzis commented 7 years ago

WIP build script is pretty much there with the last suggestion you gave me. I cat the 2 line config to the config.lua file in the script. Verifed it displays in the same syntax as config-linux.lua.

Making a tmp dir inside the main source root got things a bit further. From what I read, this may be a bug in the compiler. Not sure. You'll see similar mentions in various places, such as fpc/lazarus-based packages in the Arch Linux AUR. The fpc Debian package I backported is from 201609nn. Hmm. I could try compiling the lest fpc from the GitHub source, but i'll look around before doing that.

+ lua makefile.lua lq
Free Pascal Compiler version 3.0.0+dfsg-9~bsos+1 [2016/12/08] for x86_64
Copyright (c) 1993-2015 by Florian Klaempfl and others
vluaext.pas(455,49) Hint: Local variable "lnumber" does not seem to be initialized
vgenerics.pas(28,26) Hint: Parameter "aItem" not used
vgenerics.pas(178,26) Hint: Parameter "aItem" not used
vgenerics.pas(266,26) Hint: Parameter "aItem" not used
vlua.pas(155,17) Hint: Variable "IST" does not seem to be initialized
vrltools.pas(454,1) Warning: Function result variable does not seem to initialized
vrltools.pas(464,1) Warning: Function result variable does not seem to initialized
vrltools.pas(809,30) Warning: Function result variable does not seem to initialized
vrltools.pas(941,3) Warning: Symbol "Split" is deprecated
vrltools.pas(946,5) Warning: Symbol "Split" is deprecated
vrltools.pas(952,5) Warning: Symbol "Split" is deprecated
vluastate.pas(244,9) Fatal: Internal error 2012090607
Fatal: Compilation aborted
Error: /usr/bin/ppcx64 returned an error exitcode
Autodetected OS - LINUX
epyon commented 7 years ago

bin/config.lua is the game configuation, config.lua is the build configuration.

mdeguzis commented 7 years ago

Same thing: http://sprunge.us/IaMS. I put a ending slash on the valkyrie root, otherwise the lua makefile would not be found for valkyrie. I appreciate the help, as I'm sure you have better things to do :) Maybe it's because I am using x86_64 fpc. I noticed the readme notes 32 bit. Of Note: wget will refuse to download your game files via CLI, noting "the certificate of drl.chaosforge.org hasn't got a known issuer".

simon-v commented 7 years ago

I would like to point out, for all that's worth, that there is no need to convert any of the special level tracks to ogg, as they are already available in that format, and are freely downloadable from the music author's website. The same applies to Sonic Clang's tracks, which are redistributed in that very format with the Classic Doom 3 mod.

Why the game never packaged those instead is beyond my understanding.

mdeguzis commented 7 years ago

@simon-v , would you mind linking me those tracks, despite current issues building?

simon-v commented 7 years ago

The special level tracks are at http://simonvolpert.com/music/drla/ — the complete zip is in the bottom of the track listing. The normal level tracks are obtainable from http://www.moddb.com/mods/classic-doom-3/downloads/classic-doom-3-131 (the files themselves are in ./sound/music/ inside cdoom_sounds.pk4, which is in ./cdoom inside the classic_doom_3_1.3.1.zip archive). I might consider repacking and hosting just those files, if there is sufficient interest; From what i know, i don't think Sonic Clang will mind this any.

tung commented 7 years ago

I'm on Ubuntu 16.04 64-bit and, after running into the same issues as the folks above, I've managed to successfully compile DoomRL.

The error that says Fatal: Internal error 2012090607 happens with Free Pascal Compiler version 3.0.0, which is unfortunately the version that Ubuntu 16.04 ships with. I worked around it by manually compiling 3.0.1 from source (which has a fix for that bug: http://bugs.freepascal.org/view.php?id=27580) and using that instead for compiling.

Rough list of steps:

  1. Install fpc 3.0.0 from the standard packages.
  2. Download fpcbuild.zip (80 MB) under "Download Daily Source Snapshot of the Fixes Tree" at http://www.freepascal.org/develop.var
  3. Extract fpcbuild.zip (400 MB uncompressed).
  4. Build fpc 3.0.1 using its makepack script (takes another 400 MB of room). I had to touch doc-pdf.tar.gz and pass in CHECKLIBGDB=no before it worked.
  5. Install fpc 3.0.1 somewhere convenient (~200 MB once installed). I deleted all the fpcbuild stuff once it was done to free up the ~800 MB it was taking.
  6. Clone doomrl repo.
  7. Fill in the doomrl/bin/{mp3,music,wav,wavhq} directories with the binary assets from the standard DoomRL releases.
  8. Clone fpcvalkyrie repo into a directory adjacent to doomrl.
  9. Edit fpcvalkyrie/scripts/lua_make.lua to replace the standard fpc path with the 3.0.1 version, and stub out the logic in make.svnrevision() with a bogus return value since this is a git clone and not a subversion clone.
  10. I had to edit fpcvalkyrie/src/vtextio.pas and comment out {$IFDEF UNIX}Shell('reset'){$ENDIF} because the compiler wasn't recognizing it for some reason.
  11. Return to the doomrl directory and mkdir tmp.
  12. Run lua makefile.lua lq in the doomrl directory.

If everything went right, you'll have something like doomrl-linux-0998-beta1-lq.tar.gz sitting in your doomrl directory, as well as a playable build of the game in the doomrl/bin directory.

If Ubuntu ever decides to package fpc 3.0.1 instead of 3.0.0, all of the custom fpc building and tweaking can just be skipped.

mdeguzis commented 7 years ago

Nice! I'll make up a Debian/SteamOS/Ubuntu package if things work well tomorrow or later tonight. I have some Hadoop things I need to get to today. Thank you very much for confirming a new fpc worked. I was about to try that as well. I have all the build steps already lined p, same as you do, just needed to get over the compiler error. I will upload a 3.0.1 or higher fpc to my PPA as well. Just keep an eye on it.

https://launchpad.net/~mdeguzis/+archive/ubuntu/freepascal

Once confirmed working, i'll put up the game in the games PPA.

mdeguzis commented 7 years ago

There are a ton of Debian patches for fpc. When I have time I will port them. In looking through their patches I saw which file the true version is pulled from, so I source the major.minor.patch version from that. Currently it is 3.1.1. There is no rush on this of course, but for Debian folk, I will provide a backported package at some point. I host non-Ubuntu at packages.libregeek.org. I don't target arm or ppc, so I am cutting those patches (I am not a Debian maintainer). It's nagging at me, so maybe i'll have a package up sooner than later :)

mdeguzis commented 7 years ago

I tried using fpc-svn (which is at 3.1.1.r35149) with no luck. Below is the AUR package I started. Open the PKBUILD to see the WIP steps.

==> Starting build()...
Autodetected OS - LINUX
Free Pascal Compiler version 3.1.1 [2016/12/17] for x86_64
Copyright (c) 1993-2016 by Florian Klaempfl and others
doomrl.inc(11,4) Error: Illegal optimization specified "ASMCSE"
valkyrie.inc(12,4) Error: Illegal optimization specified "ASMCSE"
valkyrie.inc(12,4) Error: Illegal optimization specified "ASMCSE"
valkyrie.inc(12,4) Error: Illegal optimization specified "ASMCSE"
vutil.pas(283,1) Fatal: There were 4 errors compiling module, stopping
Fatal: Compilation aborted
Error: /usr/bin/ppcx64 returned an error exitcode
mdeguzis commented 7 years ago

Specifying {ASMCSE OFF} in the two inc files for doomrl/fpcvalkyrie got me farther. This line seems to be for optimizing the assembler code.

ASMCSE "Use common subexpression elimination at the assembler level"

Log: http://slexy.org/view/s20gzxLBkS

It seems a few files are incomplete, such as version.inc

version.inc(5,29) Error: Illegal expression

This is mainly due to version rev being generated here:

https://github.com/ChaosForge/fpcvalkyrie/blob/master/scripts/lua_make.lua#L265

..when subversion is obviously not in use :)

I then changed this line in the pcvalkyrie/scripts/lua_make.lua. 

        local text = [[// Autogenerated by FPC Valkyrie lua_make.lua
VERSION_STRING = '{!string}';
VERSION_ARRAY  : array[1..{!length}] of byte = ({!array});
VERSION_BETA   = {!beta};
# This line below and this comment were modified
VERSION_REV    = 0000;

Then, I fixed the offending shell line in ../fpcvalkyrie/src/vtextio.pas

  //Unknown to myself, I have no clue why I get an access violation here...
//{$IFDEF UNIX}Shell('reset');{$ENDIF}

I'm still working on things. Lots of old svn code. If I can piece together a solution, I'll add the set of patches to the AUR package. Thank you so much to GPF_1 in #fpc on freenode for helping!

Now on to a final linker error it seems:

Warning: Library lua5.1.a not found, Linking may fail !
/usr/bin/ld: cannot find liblua5.1.a

The linker is failing out, pointing towards doomrl/bin/link.res. Some kind of disagreement with dynamic .so over static .a library. @tung, see anything of note here?

GROUP(
-lc
)
INPUT(
/usr/lib/gcc/x86_64-pc-linux-gnu/6.2.1/crtendS.o
/lib64/crtn.o
)

Current Log: http://sprunge.us/DKed

Update

Ok I fixed things by forcing dynamic linking with fpcvalkyrie/lua. things link fine now. Should have things fixed up today I think for the AUR package if the rest goes well.

subtract1 commented 7 years ago

If you run in godmode you don't need the wads as it reads the scripts directly (well, you do NEED them but it doesn't use them). Don't know if that helps you any but there you go.

mdeguzis commented 7 years ago

I was able to just get past that by temporarily symlinking the lib to the current directory where makewad is. I have a bit of testing to do. Everything builds and I modeled the pkgdir after the Linux tarball that is pushed out, but i'm sure I'll need to make some adjustments. I can't test right now, away from my desktop, using SSH.

The one thing I am missing yet is core.wad for whatever reason. Actually, makewad is just failing, not sure why it can't find the lib (which I'll need to generate the core wad):

mikeyd@archboxmtd bin $ ./makewad
An unhandled exception occurred at $00000000004BF306 :
ELibraryError : Can't load library "lua5.1.so": lua5.1.so: cannot open shared object file: No such file or directory
  $00000000004BF306  LOAD,  line 65 of ../../fpcvalkyrie/src/vlibrary.pas
  $00000000004A6F87  LOADLUA,  line 391 of ../../fpcvalkyrie/libs/vlualibrary.pas
  $00000000004A8378  LUA_OPEN,  line 651 of ../../fpcvalkyrie/libs/vlualibrary.pas
  $00000000004812AF  ADDLUAFILE,  line 110 of ../../fpcvalkyrie/src/vpkg.pas
  $00000000004825C2  ADD,  line 206 of ../../fpcvalkyrie/src/vpkg.pas
  $0000000000482709  ADD,  line 216 of ../../fpcvalkyrie/src/vpkg.pas
mdeguzis commented 7 years ago

fixed it :) sorry for all the posts, this may be helpful to those debugging anyway.

Update: The AUR package should be ready to test, i'll make sure things work before finalizing this. It was a lot of hacky maneuvers to get here. I'll take a break and maybe do a Debian package next week, not sure. Since the game does not have a concept of shared folders, I had to deploy files to /usr/share/games/doomrl and make it owned by group "games" + writable. The other option is to do the silly copy/symlinking the doomrl package does (the unpacked zip for Linux).

Update

Ugh.. when running I get an error:

mikeyd@archboxmtd ~/tmp/doomrl-git/bin $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${PWD} ./doomrl
libpng warning: iCCP: cHRM chunk does not match sRGB
libpng warning: iCCP: known incorrect sRGB profile

Abnormal program termination!

Reason : LuaError: DoomRL.OnLoadBase -- Range check error

If this reason doesn't seem your fault, please submit a bug report at
http://forum.chaosforge.org/, be sure to include the last entries in
your error.log that will get created once you hit Enter.

An unhandled exception occurred at $000000000054DFEF :
ELuaException : LuaError: DoomRL.OnLoadBase -- Range check error
  $000000000054DFEF  ONERROR,  line 441 of doomlua.pas
  $00000000004EA39A  PROTECTEDCALL,  line 1511 of ../../fpcvalkyrie/src/vluasystem.pas
  $00000000004581BB  CALLHOOK,  line 85 of doombase.pas
subtract1 commented 7 years ago

I sincerely hope we get a good linux maintainer who can make DoomRL work and be all multiuser friendly. I know that person won't be me so good luck. We're all counting on you.

mdtrooper commented 7 years ago

@subtract1 Maybe we must to start a bounty software about the DoomRL in GNU/Linux.

subtract1 commented 7 years ago

I think the main issue is we're all C gurus and not pascal gurus :/

mdeguzis commented 7 years ago

I've built plenty of programs (see my libregeek-packaging repo), this is my first that is pascal. I believe with a little nudge/bit of help I can get this. Usually, on a common build system, or something common (make/cmake/qmake etc.), you'll know where the issue lies. In this, you do need some inside knowledge here from the devs or good knowledge of pascal code. We'll just have to wait until after they get the Jupiter Hell business in order. Just a matter of time. I can get the entire project to build with static/dynamic, but I must be missing something. I double checked my produced/packaged files vs. what is distributed via the tarball on the website.

Anyway, Happy Holidays (soon) everyone!

subtract1 commented 7 years ago

Wait, you're trying to get an exact binary match? I can guarantee you the source code that was GPLd is not the code that was used to build 0997; it's more like an alpha 0998 and a few things in the engine have changed (enough to require altered lua scripts)

mdtrooper commented 7 years ago

Well...Did somebody compile the DoomRL?

mdtrooper commented 7 years ago

With the recipe (https://github.com/ChaosForge/doomrl/issues/3#issuecomment-266179749) of @tung is easy more or less to make a binary of doomrl.

But in Debian, you must use the lua5.1 instead the 5.3 for to avoid this bug:

Timestamp   : 24-10-17 02:32:24
Error level : FATAL EXCEPTION
Message     : Fatal exception encountered

An unhandled exception occurred at $000000000052CB84 :
ELuaException : LuaError: DoomRL.OnLoadBase -- Field maxrange not found!
  $000000000052CB84 line 441 of src/doomlua.pas
  $00000000004D0AD0 line 1511 of ../fpcvalkyrie/src/vluasystem.pas
  $0000000000451F27 line 85 of src/doombase.pas