Making Ken Silverman's "Voxlap" voxel graphics engine run on any platform with SDL and C++.
I used to have an outline of things to do. But it became horrendously out of date. This should serve the same purpose, and actually reflects what I want to do. Keep in mind that to some extent all branches are developed simultaneously.
This is what master is now. In short, every block of MSVC inline asm has a block of gcc inline asm to go with it. Nothing else semantically should be changed. I doubt this will work, as sometimes Voxlap used persistent registers.
Currently Voxlap actually contains both SSE2 and 3DN inline assembly, and uses
some run-time stuff to chose which routines to use. This is really weird, and
makes replacing with C slightly more complicated as some functions like
movps_3dn are no longer used. I should remove the "cputype" routine, and
make the choice of architecture compliantly compile time. This should be
rebased off conservative. ("Fatbin" refers to the fact that cputype is used
to compile something analogous to a fat binary.)
This is no-asm, a rush to get something that works and also uses as little compiler-specific code as possible. I have used ken's C alternatives even when vector built-ins would be more efficient, or said alternatives don't even work right (kv6draw). all the point4d vector arithmetic functions should be moved to a separate header. This should be rebased off no_fatbin.
This is where I want to end up. This will be completely architecture independent. Pure C is used where appropriate, but if something is better done with SIMD/vector built-ins I use those instead. By the time I get to this, no-asm should be bug free. And if I haven’t bothered to redo v5.?asm in no-asm, I should re-do it now with vector-built-ins. Probably a lot of the little inline functions used for assembly can be manual inlined and discarded at this point. I may drop MSVC support here, as I don't really want to have to redo all the SIMD stuff for two compilers. Also I might do the minimal pre-processor stuff needed to allow for building as Pure C if I have not done so already (this involves making a few static struct members into global variables, so it will change the API). This should be rebased off no_fatbin, not no-asm.
So, the basic idea is to get Ken Silverman's Voxlap voxel graphics engine to run on just about any platform that supports C++ and SDL (DirectX is still kept operational for windows). This all boils down to 2 fundamental things: getting the code to compile in GCC, and making sure that now windows libraries are in use.
The compiler part is pretty easy, except for one thing: the assembly code. Voxlap uses both a single file of routines in assembly, and copious amounts of inline assembly within it's C++ files. Now, most of this assembly should be perfectly portable, but every compiler collection has a different format for both external and internal assembly.
I say format, because converting from one to another is a fairly trivial task, but not quite trivial enough where I can confidently do it with my limited assembly knowledge in an error-free manor. I have tried various scripts to convert assembly (inline and external), but the bottom line is that I don't really trust them. Most significantly, v5.asm's NASM port won't work with MSVC (though it should), so why should I trust it with GCC? Dealing with the inline assembly is even simpler, but the conversion tools are correspondingly more primitive, and I can't test them anyways until v5.asm works.
Therefore I HAD opted to dispense with assembly code altogether. Voxlap5 (the main Voxlap code) is littered with commented out replacement C for various assembly routines. Not all of it works as well as the assembly -- I think Ken mainly used it as a guide for writing the assembly and didn't debug it as the program evolved -- but it's certainly better than nothing. As to the external assembly, there is a macro that's supposed to let the program run without it, but it leaves in a few missing linkages -- it was probably also ignored as the program evolved. For better or worse, the inline assembly and external assembly often relies on each other. On one hand, this means taking out one can help get rid of the other. On the other hand, it means it's very difficult to replace the assembly incrementally and debug as you go.
I learned of another assembly, JWASM, on IRC which is both cross-platform and Supports NASM syntax as close as possible. This gave me the opportunity to go back to my original plan of porting all the assembly. This of course came with the added benefit of leaving the original program as intact as possible, and utilising the inline assembly that was admittedly still faster than the C. For ARM ports it will still have to go but this seems to be the easier solution for now. Now my plan is to work along both methods. I don't know whether converting or removing inline assembly will get to a finished port faster, so I've made a git branch for each method to work on both concurrently.
I was able to quickly fix the few syntax errors that existed, and get a working Game.exe with MSVC and JWASM. But JWASM in itself did nothing to prevent the link errors with MSVC external assembly together is often made easier. The cacophony of link errors, was a bit hard to decipher, and in order to better separate assembly-caused linking errors from other issues, I decided to get the whole thing to be compiled as C. C, unlike C++, has no name mangling, and in addition C-derived objects files made by different compilers have a better chance of linking correctly (especially on windows). I figured this would open up new doors testing.
I now have everything compiling and linking on Windows and Linux, MSVC and gcc, and SDL on both platforms. Unfortunately however, when running a gcc build on any platform some text is displayed on the screen. When I don't use the -ffast-math flag, it will in addition crash due to arithmetic overflows. Because this is thus strictly a compiler-caused issue, my guess is there is still some mistake in the inline-assembly I converted to C. It is now clear to me that in order to get this working, I will have to remove all inline assembly. Because in many cases the C is less efficient, I will try to add it back in after I have fixed most bugs. But the only way to rigorously cross- reference the MSVC and gcc builds/bugs, I must avoid as much compiler-specific syntax as I can. I imagine once I reach the point where it is safe to again experiment with assembly, asm and no-asm versions will be developed with some independence, and the benchmarks will state which branch to focus on.
I can't really dive into this until the former step is complete, but it seems that again a good bit of the work is already done.
Voxlap itself is a surprisingly self-contained beast, primary because it's a software render so it only uses DirectX to copy frame buffers and other trivial stuff. The back-end it currently uses Winmain, is not Voxlap-Specific, but rather used by a couple of Ken's things (maybe even the build engine).
On the flip side, the example game, and probably other games based around Voxlap don't just use voxlap's header, but also directly use Sysmain.h, Winmain's interface. So to have a genuinely useful Voxlap port, seemingly unused features of Sysmain.h must also be ported.
Thankfully, available on Ken's site is SDLmain, a non-Voxlap-Specific port of Winmain. I'm not sure how complete it is, but I'd hope it has the most basic stuff that Voxlap itself requires available. The more progress I've made, the more I been able to validate the SDLmain basically works as advertised. Now that I have the Voxlap building with SDLmain and MSVC, I am almost sure it will hardly need modification.
The other example programs besides 'game' use some GDI and other Windows- specific APIs not abstracted in Winmain, so I will not focus on porting them for the time being. I did recently however add Slab6 to my code base, as it shares both Winmain code that later ended up in Voxlap. To fully incorporate it I will need to deal with these other windows dependencies, so doing so is definitely on the agenda.
A true port should change as little of the old program as possible in order to maintain maximal compatibility. At the same time, porting is a perfect time to introduce deep architectural changes into a program. I believe a few obvious improvements that would not effect existing programs ability to interface with Voxlap to much extent.
Voxlap is a library, nothing more, nothing less. As such it should be able to be both statically and dynamically linked with another program. Currently, only static linking is tested to work, but with a couple trivial changes, dynamic linking should be too.
Voxlap is C++, but in name only. In practice it contains little-to-none Object-Oriented components, and only a few C++ idiosyncrasies prevent it from being compiled as C. In fact, it even contains a couple of preprocessor conditionals to test for C or C++ completion. Ideally it should fully support being compiled as C or C++ in order to work in many situations.
Based on my concept of a "true port" I should keep the license the same as the original: basically attribution + non-commercial only without Ken Silverman's explicit permission. I previously stated here my work could be licensed with the LGPL provided Ken's restrictions are also followed, but I now see that is not possible due to the GPL's requirement that commercial distribution must be permitted.
As before, Ken's licensing restrictions must be obeyed. If obtains permission to use Ken's code under different terms, he or she must also contact me and obtain permission if they want to use my modifications.
Freenode IRC network: It about as indispensable a resource as exists for programming these days. #C and especially #asm in particular for this project. Many people on the network offered excellent advice, and copious amounts of patience and time, but I'd especially like to thank:
The Voxlap engine includes all the tools you need to create your own game using a high-speed software voxel engine. I have included 4 applications:
I compiled the code with Microsoft Visual C/C++ 6.0. It should work with later versions of Visual C. If you don't have a copy of MSVC, you can download the VC 2003 Toolkit from Microsoft. (See note in requirements).
After unzipping, the first thing you should do is look at this file :) First, I'll describe my way of compiling. I like to compile at the command line. To set up your VC environment, run VCVARS32.BAT (as described at the top of MAKESTUF.BAT) Then in MAKESTUF.BAT, check to make sure your directories are set up correctly. Make sure your environment variables (lib and include) point to the right path. You'll need them to point to the Visual C directories as well as the directories from the DirectX SDK. Once this is done, you are ready to run MAKESTUF.BAT. I put an in-line makefile at the top of each source file. This is why there is no "makefile" in the directory. Cool trick, eh?
You can compile the examples from the VC GUI environment as well, but you'll need to create your own project if you do so. Here are some hints on setting it up to compile Voxlap:
Be sure to specify these non-default compiler settings:
/J default char type is unsigned/TP compile all files as .cpp (I incorrectly use .c to save typing
:)Add these libraries to your linker settings:
To compile V5.ASM, you'll need an assembler. I use MASM. I'm not quite sure how to set this up in the VC environment. To make things easier, I included V5.OBJ. You can add this to your project if you have trouble assembling V5.ASM.
To compile with VC Toolkit 2003:
Change the linker option: /MD to /ML (see top of GAME.C, SIMPLE.C, VOXED.C, KWALK.C)
To fix the "MSVCRT.LIB not found" error, in WINMAIN.CPP, change: #define USETHREADS 20 To: #define USETHREADS 0
-Ken S.
WINMAIN.CPP uses SSE or 3DNow! instructions, so if you are compiling with VC6, you will need to install the Processor Pack (VCPP.ZIP). VCPP will only install over VC6 SP5 Professional edition, although I have had some success in fooling the installation program :) Fortunately, this is not an issue with VC7. You can also download a "free" version of the VC compiler: google for "c++ toolkit 2003". Unfortunately, it's missing a few files, such as WINDOWS.H and NMAKE.EXE, but they can be found elsewhere - either from the Platform SDK, or by googling for them.
Quick&dirty batch file showing how to compile things at the command prompt.
Minimalist sample code showing how to interface Voxlap. Study this code before looking at GAME.C. It should also be easier to start a new project from this code.
A big ugly testing ground for many Voxlap features. Think of this as the example code for most functions in VOXLIB.TXT.
Include file for VOXLAP5.C, V5.ASM, KPLIB.C
Voxlap engine C routines
Voxlap engine assembler routines
Ken's Picture LIBrary. Decoding for: PNG,JPG,GIF,TGA,BMP,PCX and ZIP. This library can be used independently from the engine. Voxlap depends on this library for various functions.
Windows layer (setup screen, sound, input). This code is independent of Voxlap and can be replaced if you want to use your own windows layer, such as SDL. I've been maintaining WINMAIN.CPP for years, so it is rather stable now.
Include file for my windows layer (WINMAIN)
Game data. You do not need to extract this data to run GAME.EXE, SIMPLE.EXE, or VOXED.EXE. You will need to extract it to select models for loading in KWALK though.
My VXL voxel editor. I made UNTITLED.VXL using VOXED. There are a lot of keys to learn - so have patience. As a convenience, you can view the help file (VOXEDHLP.TXT) inside the editor by pressing F1.
An animation editor for .KV6 voxel sprites using .KFA animations. You'll need to extract files from VOXDATA.ZIP to see my example models.
Description of the VXL format
Help file for KWALK
Help file for VOXED
Help file for VOXLAP5, KPLIB, and WINMAIN.
This help file
/win Run in a window/3dn Force 3DNow! code. Crashes if not Athlon series./sse Force Pentium III code. Crashes if not P3+/AthXP+!/sse2 Force Pentium 4 code (ignored currently)/#x#x# Specify resolution. Max resolution is 1024x768/fps Toggle fps meter/fallcheck Toggle fall checking/sideshademode Toggle cube face shading/faceshade=#,#,#,#,#,# Specify shade offsets for each of the 6 faces.
Example: /faceshade=0,5,10,15,20,25/monst Toggle monsters active/inactive./curvy Toggle KV6 sprite in front of view (cycles through
all 3 bending/twisting modes) Try it!/curvy=[KV6_file] Toggle and select KV6 sprite in front of view.
Example: /curvy=caco/curvystp=# Set stepsize for /curvy mode/light Add a light source (lightmode 2 only)/lightclear Clear all light sources/lightmode=# Specify lightmode (0=no lighting,1=normal
lighting, 2=point source lighting)/scandist=# Set maximum radius to scan for VXL raycasting/fogcol=# Set RGB fogcol. Example: /fogcol=0xff0000 sets fog
color to bright red. /fogcol=-1 to disable/kv6col=# Set RGB color to blend with all KV6 sprites with
(default:0x808080)/anginc=# Select raycast width (affects framerate/detail
significantly) Default:1, Typical range:1-8/vxlmip=# Select raycast distance where VXL (world map)
MIP-mapping begins. (Default:192)/kv6mip=# Select distance where KV6 (sprite) MIP-mapping
begins. (Default:128)/health=# Set player health (Default:100)/numdynamite=# Set number of dynamite (choose a high number :)/numjoystick=# Set number of joysticks (choose a high number :)/showtarget=# Toggle target in center of screen/showhealth=# Toggle health displayKASCI9x12.PNG: Tom added Polish characters to the bottom of my font :)
Tom has been a great help to me over the past few years. I thank him for his many ideas, help, and keeping me motivated to work on Voxlap for as long as I did. He has written many things for Voxlap in addition to the Voxlap Cave Demo and his untitled game from 2002 which was never released.
KPLIB.C (1%): JonoF upgraded this library to be compatible with Linux for his Build port.
Any derivative works based on Voxlap may be distributed as long as it is free of charge and through noncommercial means.
You must give me proper credit. This line of text is sufficient:
VOXLAP engine by Ken Silverman (http://advsys.net/ken)
Make sure it is clearly visible somewhere in your archive.
If you wish to release modified source code to your game, please add the following line to each source file changed:
// This file has been modified from Ken Silverman's original release
I am open to commercial applications based on Voxlap, however you must consult with me first to acquire a commercial license. Using Voxlap as a test platform or as an advertisement to another commercial game is commercial exploitation and prohibited without a commercial license.
Full voxlap source code released. Changes since 11/09/2004:
Voxlap library released.
If you do anything cool with Voxlap, I would love to hear about it! I prefer that you contact me in either of these ways:
Write about it on my personal forum ("advsys.net/ken" topic) at: http://www.jonof.id.au
You can write me a private E-mail (address can be found at the bottom of my main page). Please use this for license requests.
If you have any questions or feature requests about other things in Voxlap, it doesn't hurt to ask, although keep in mind that I haven't really worked on the engine since 2003.
Ken's official website: http://advsys.net/ken