a path towards moving from Allegro to SDL

[ericoporto]

ags/wiki/SDL-Migration-proposition.md

The document is in the wiki so people can write in and clarify! If you are worried about losing your edit, do it using git, the document is SDL-Migration-proposition.md on the root directory.

• https: git clone https://github.com/adventuregamestudio/ags.wiki.git
• ssh: git clone git@github.com:adventuregamestudio/ags.wiki.git

---

I wrote above a document proposing a path toward migrating from Allegro, meaning, removing Allegro dependency completely, and using instead mostly SDL. It may turn out we need something else too.

Since last ags3-sdl2 port did not get merged, this tries to detail a plan, with ags4 in mind and focus on complete removal of Allegro.

Please comment , also accepting people to write the code. My central idea was to have code flowing towards this goal into AGS repo.

[sonneveld]

@ivan-mogilko

I found and marked several of your old threads with different works (streams, strings, asset loader for switch iirc, and so on) and probably will be using them as a reference (if that's okay legal-wise).

I give you permission to use any code from my ags repo forks. Considering that the exact commits won't be against my name, could I request that I be mentioned in the Credits as the author of the original SDL port code?

Something due mentioning, we were discussing trying to port without having allegro wrap over SDL. Could you elaborate, because I forgot, what were the reasons you went with allegro wrap, was that for simplicity sake, or something more serious?

My first attempts at porting the engine to SDL removed the allegro library altogether. However I found that because we relied heavily on the different allegro blending methods, as well as different drawing routines (line drawing etc), I had to copy and paste a lot of that code verbatim. It also meant that the engine didn't have full functionality until I had ported all the different allegro methods to use SDL.

For reference, you can see that attempt here https://github.com/sonneveld/ags/commits/ags3--sdl2--old--2016-10-06 (I think I have an earlier branch from 2012 somewhere too. That was based on the original ags code before any refactoring)

The later attempts kept allegro in the loop. By implementing the SDL support as drivers, it reduced the surface area of the allegro api that I needed to port. I wanted to commit this cut down allegro layer first, to ensure that compatibility remained before removing allegro altogether.

Either way, we would still need to reimplement the blending and drawing implementations (or drop them if you're dropping compatibliity). Early on I believe we needed the allegro fixed point maths library for the path finding code but that has been rewritten. Also colour conversion code (i.e. 32bit rendering at 8 bit and in the other direction)

it's just that I am concerned that it may be an overkill to bring whole ffmpeg purely to play videos (while we still are using SDL_sound etc for sound).

That's fair. There is definitely an overlap when it comes to audio codecs. However SDL_Sound has a much less restrictive license, so we can include that while leaving the GPL licensed ffmpeg as an optional feature.

In regards to using thirdparty libraries in general, it's all a trade off. AGS only has a small team of developers and I see the time would be better spent on the editor and implementing actual features for creating adventure games. Supporting multiple systems and audio/video codecs can be delegated to libraries. Game engines like godot have the luxury of extra developers which is why they can afford to ignore libraries like SDL and write drivers per OS.

@ericoporto

for ags3.6 I think I would keep allegro even if only for the bitmap operations, while having mouse, keyboard, and window and others on SDL2.

Agreed. The bitmap operations are the main thing that might just need to be copy/pasted or reimplemented into the ags project.

[ivan-mogilko]

I see... so, presumably, traditional bliting/blending and primitive drawing operations are not covered by SDL to full extent, or do not produce same result? I will have to research this when I get to them because never compared sdl and allegro before. AFAIK allegro uses common memory format for bitmaps, so it should not be a problem to incorporate its functions withing sdl-based program...

Anyways, so far this kind of matches the plan I was imagining, where first step is sound (most easy to replace), next step is window + events loop, and then we'll see how to deal with raw drawing.

[ivan-mogilko]

A tiny update, I started wip branch for step 2 (it's on top of the audio branch), which includes SDL window handling and input. https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--try-sdl2-window It's kind of rough so far, and I might post more details later. Unfortunately, there are some choices done by Nick in regards to key handling which seem bit weird, which may also mean that I dont realize some problem that he was trying to handle, so I will need more time to research this.

As of now:

• allegro is installed with "SYSTEM_NONE", which prevents it from creating now unnecessary window and grabbing input. Still need allegro to initialize, as all bitmap code and some utilities depend on it.
• old OpenGL and Direct3D renderers work fine in SDL window, but 1) need to update OpenGL renderer to create context through SDL - this way the code might become universal, without need to split between Windows/Linux etc; 2) software renderer must be replaced with one using SDL_Renderer for final present (Sonneveld's branch has one ready, so it's mostly matter of copying I hope).
• there's a silly mistake that the window may appear partially behind the desktop border, this is because I resize it after creation and forgot it also needs re-aligning on desktop.
• did not copy support for resizable (by user) window yet (want to complete the main bulk of functionality first).
• input does not really work yet, but at least event processing is going and you can drag window around and close it (no freezing).
• after latest WIP commit there's a strange issue with audio stopping playing after few seconds (but audio thread continues to run well).

[ivan-mogilko]

Remade https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--try-sdl2-window

• added fully working mouse code (or so it seems).
• left keyboard out for this time. There are few things that I still do not understand completely and wanted to read some more on how SDL handles it to be certain what the code does. But I hope to get it back in a couple of days.
• fixed dissapearing music (trivial missing break in the case statement, and engine kept thinking that user alt+tabbed outside the window)
• PS. not certain what to do with mouse speed config; if we completely remove exclusive fullscreen that may not be necessary to have, if we keep it as a third option (e.g. for Direct3D on Windows) - then ... . anyway I may disable it for now.

[rofl0r]

if we completely remove exclusive fullscreen

what do you mean by that ?

[ivan-mogilko]

After an unfortunate delay, the branch is updated with working SDL key input. https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--try-sdl2-window

I spent time mostly researching different approaches, but in the end decided to use Nick's code only partially and keep original simple virtual keycode value throughout the engine, only changing the way it's received - from buffered SDL events rather than polling allegro (from the look of the code, @sonneveld wanted to already begin replacing this old keycode with reading sdl data).

The main reason I did this is that IMO there has to be a larger engine redesign to make direct use of events convenient. I was also concerned that otherwise their presence may confuse other contributors. Ideally, the event polling should pass events to some subscribers (engine components and game states, whatever you call them). Right now, because of how engine is structured, we have to save received event in a queue and then pass it "out" through a sequence of functions by demand. Also, I am simply not certain what would be more convenient: to have SDL_event read everywhere, or invent some custom struct to pass only data we may need. So leaving this for next time. NOTE: if we add unicode support, we will have to replace that simple keycode with some struct containing optional utf8 char array; SDL has special events for receiving unicode interpretation of key presses, and they provide a ready utf8 string, so may be useful. Haven't tried them in practice though.

Remaining tasks on this stage:

• complete renderers update;
• fix notable bugs.

---

if we completely remove exclusive fullscreen

what do you mean by that ?

There are three possible gfx modes: windowed, "desktop fullscreen" (borderless window covering whole desktop) and "exclusive fullscreen". I dont know if third one is supported on Linux oses, on Windows it's a part of DirectX functions, and supposedly can improve perfomance as all video resources are given to single task (as a downside switching away is slower as system has to unlock the mode first). People were telling me that "desktop fullscreen" becomes a better default on Windows too.

[sonneveld]

@ivan-mogilko only changing the way it's received - from buffered SDL events rather than polling allegro

If you mean removing a keyboard buffer altogether, I'm not sure that's the right way to go. The reason why I buffered the events is because we have code in ags_getch and ags_clear_input_buffer essentially wrapping allegro's readkey, which returned the next key from the keyboard buffer. (Reference https://liballeg.org/stabledocs/en/alleg006.html#readkey )

Simply polling will mean you might miss key events between frames.

edit: oh another reason for keeping track of SDL events was because the gui text input needs actual ascii chars (upper and lower case chars) instead of scancodes. SDL doesn't have a mapping from SDL_KeyboardEvents to ascii. It prefers you use the keyboard input events SDL_TextInputEvent instead. So where we need "ascii", I ended up using keyevents for backspace/tab/esc/enter and textinputevents for the rest. You can see that in asciiFromEvent that is used whenever code requires ascii chars.

[ivan-mogilko]

@sonneveld

If you mean removing a keyboard buffer altogether,

No, I keep the buffer mechanic of course, perhaps did not express myself clearly but this is what I am saying in quoted sentence: replaced allegro poll with getting a key from buffered SDL events (that we buffer).

oh another reason for keeping track of SDL events was because the gui text input needs actual ascii chars (upper and lower case chars) instead of scancodes. SDL doesn't have a mapping from SDL_KeyboardEvents to ascii.

Hmm, I spent some time figuring out what your code means before realizing clearly which problem it tries to solve (as you explained in above comment). After that decided to try an experiment, and found that it may be possible to map from SDL keydown event to ascii without SDL_TextInputEvent, by checking for all mod states, including SHIFT. This is why for the time being I added only keydown events handling, with plans to utilize SDL_TextInputEvent later for unicode support. Basically, what I do is convert SDL keycode + mod combination into internal engine enum that merges both ascii and these special allegro-based values (Alt + xxx): https://github.com/ivan-mogilko/ags-refactoring/commit/0c9ed8401f99426f8e7de7483c1a0f9952727919#diff-0213334fb1d5c42579e84c9f63ec0ec4b7480382fbcb164c4d661f1341691595R38

EDIT: Yes, I expanded eAGSKeyCode enum in the engine to include both small and capital letters (this was merely formality at this point), and only convert it to the script keycode by capitalizing letters before sending them to script callbacks, thus to match AGS script requirements.

This was successfully producing both small and capital letters in text fields for me.

But I must admit that dont know if this is 100% correct and will work for everything (still have to test on linux etc), so I am ready to switch to TextInputEvent if current code fails in some circumstance.

[sonneveld]

Ah fair enough.. I think I misunderstood.

Because it looked like the combined scancode/ascii was only used internally, I decided to bite the bullet and use the actual sdl scancodes where I could (that way I could use the SDL key defines everywhere internally). As far as I could tell, ags scripts only exposed the mapped scancodes, and not the combined ascii codes anywhere so you might need to be careful you don't expose that accidentally. Translated ascii is only used for gui and parsers so it made sense to make it understand SDL textinput instead.

edit: ahh, you map back from the combined back to the normal scancodes for scripts. I think I thought that was too much converting back and forth for me (plus it wouldn't work once you go unicode). It seemed much easier to use SDL events as much as possible, only converting when necessary. i.e to ascii when adding to a string to ags key enum when passing to an ags script function.

edit2: Looking at your commit, does it deal with special characters? (e.g mapping shift-3 to # on us keyboards). This was the main reason why I decided not to convert scancodes to ascii characters myself. I could have done the mapping for my own keyboard, but I didn't want to have to write lookup tables for different keyboards, which is why I deferred to SDL textinput events.

[ivan-mogilko]

edit2: Looking at your commit, does it deal with special characters? (e.g mapping shift-3 to # on us keyboards). This was the main reason why I decided not to convert scancodes to ascii characters myself.

Ah. No, it does not. For some reason I thought this will work because there are SDLK_ constants for these too and so these will be covered. Guess I misunderstand how these sym codes work then.

It seemed much easier to use SDL events as much as possible, only converting when necessary

I tried that at first, but got worried that this will cause confusion to people, because whole rules for using these conversions is unclear. I hoped to do this after inventing some better looking way to handle these. My thought is, given we utilize all the necessary SDL events, it might be possible to receive an old style ags code at all times.

[sonneveld]

For some reason I thought this will work because there are SDLK_ constants for these too and so these will be covered

I went through the same thought process! But I believe those SDLK constants are for keyboards that don't require a shift to produce those keys. You might see there isn't a one to one mapping between those and ASCII characters.

[ivan-mogilko]

@sonneveld ok, I restored SDL_TEXTINPUT handling: a7aafb82feb37dddec308a63fc891833ee8077f7

It seems generally working, will have to tidy up later maybe, and add mors comments to prevent same questions in the future.

[ivan-mogilko]

Updated branch with number of fixes, OpenGL renderer uses SDL code to be more universal, and software renderer rewritten to use SDL_Renderer. https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--try-sdl2-window

Software renderer now works like this: evertyhing is drawn to virtual screen as usual, but then it's converted into SDL_Texture and put on screen by SDL_Renderer, which may be accelerated if supported. For example, on Windows it creates Direct3D (according to log).

There are certain differences from original code by @sonneveld, one notable is that I explicitly define the final rect where texture should be scaled to, thus allowing letterbox/sideborders modes. Also, I kept old fade-in/fade-out functions, mostly because of plans to be able to have a "debug overlay" over any game effects in the future. There is also a questions of proper renderer architecture with support for render layers of different resolution, and so on, so I try to keep changes minimal whenever possible for now.

The Hqx filters are removed completely, as these are slow and it's unknown if they are used by anyone anymore (esp. considering software renderer was working terribly slow on Linux for many people). In theory it might be possible to have them inserted before final texture scale, but that's better left for later consideration.

EDIT: my plans are now to

• clean this recent branch and rebase again with squashed commits (where makes sense);
• update makefile and try building and running on Linux;
• then move to stage 3, where experiment with unlinking Allegro lib completely and see what remains to be replaced (possibly having to move some bitmap code from allegro to the engine).

[ivan-mogilko]

Also, I found a bug in SDL :) https://bugzilla.libsdl.org/show_bug.cgi?id=5329

[ivan-mogilko]

Rebased / cleaned branches:

Stage 1: sound https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--sdl2-port1--sound Stage 2: window, rendering and input (based on top of Stage 1) https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--sdl2-port2--windowinput

@sonneveld I am adding following to some commits: Based on the work of @sonneveld (Nick Sonneveld), hope this is right way to reference you.

[ivan-mogilko]

Made a quick test by removing allegro from the project, and following are things that AGS still uses from allegro:

• Bitmaps and software blitting & blending functions; the easiest is probably to extract and copy necessary parts of allegro source (that does not seem too bring much extra code with itself, but I only made an approximation so far).
• File system functions, such as "find files" and path utilities; I guess as an option we could use physfs for these if we plan on using physfs in the future in general.
• Unicode string functions and conversions, used literally in one or two places in the engine, but also used extensively in alfont. It does not seem like SDL has any unicode string utilities (except unicode keyboard support). Maybe we could extract these from allegro too, or find some simple library.
• Alfont depends on bitmap and unicode allegro functions. I havent researched SDL's TTF support yet, so idk whether it's better or worse or suitable at all for us. EDIT: font support is not a part of core SDL, but there's SDL_ttf library: https://www.libsdl.org/projects/SDL_ttf/ which also uses FreeType internally.
• Apeg is used to play theora videos, and depends on some allegro parts; bitmaps for certain, but I did not do full research yet.
• FLIC video playback is done through allegro.

[ericoporto]

Just a note about playing Theora Videos, a library appears to also exist from Icculus: https://icculus.org/theoraplay/

He also ships a video player using it with SDL2 here: https://hg.icculus.org/icculus/theoraplay/file/tip/test/sdltheoraplay.c

Unfortunately I couldn't find documentation on it or the library.

Also, copying the bitmap related sources from allegro4 is fine, they are not that many files and they don't interact with the "outside", they are basically math in arrays.

[ivan-mogilko]

Well, I pushed a first wip attempt of embedding a gfx part from allegro, it's a little dirty atm and I only included headers: https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--sdl2-port3--removeallegro

[ivan-mogilko]

I checked TheoraPlay very quickly, but it seems to use its own thread types inside, which may not be consistent with our code.

For now pushed @sonneveld 's changes to apeg, which make it use OpenAL instead of allegro sound interface. They seem to do the trick, only had to additionally replace allegro timers with SDL's, and fix a mistake that made video play x2 faster than the sound...

As far as I can see, file/path functions are the only ones remaining to be replaced...

[ivan-mogilko]

Hmm, so, I decided to not use physfs at the moment, as it turned out that it can only replace findfile functions, but not other path-related ones. There may be other libs for working with paths (e.g. I found this: https://github.com/likle/cwalk), but they will have different api and will require more changes, so we might just keep using allegro's for now.

It looks like we will need to have a stripped allegro library in some form, with following contents:

• bitmaps, drawing, blending and palette functions;
• functions for loading/saving bitmaps and pcx (latter is for that silly preload splash screen); EDIT: ah, SDL has ways to load and save BMPs though
• playing FLIC;
• PACKFILE, stripped to only supporting user vtable (used only for FLIC iirc);
• fixed math functions;
• file path and find-file functions;
• unicode functions (used very sparingly, but maybe we can use these later in full capacity for utf8 support);

Perhaps we may have this stripped library in its own repository, optionally added to project as submodule. Then we may get rid of full allegro dependency already.

[rofl0r]

It looks like we will need to have a stripped allegro library

do you have an estimate how big that will be ? if it's just a handful of files (like less than 3 KLOCish) it would IMO be a lot less hassle if it was in the main repo rather than a subrepo.

[sonneveld]

It looks like we will need to have a stripped allegro library in some form

This fairly stripped down already https://github.com/adventuregamestudio/ags/tree/ags3--sdl2/libsrc/allegro Includes most of the code you're talking about, with an sdl2 subsystem. The CMakefiles have been cut down for easy inclusion as well.

[ivan-mogilko]

@sonneveld

It looks like we will need to have a stripped allegro library in some form

This fairly stripped down already https://github.com/adventuregamestudio/ags/tree/ags3--sdl2/libsrc/allegro Includes most of the code you're talking about, with an sdl2 subsystem. The CMakefiles have been cut down for easy inclusion as well.

Okay, I spent several days researching what could be cut off or replaced, because at first I wondered if remaining code will be small enough to be added into engine directly. But maybe I can reuse your library, except it may be cut even further imo. Primarily I hope things like drivers could be removed, but also things like video bitmaps (as we only need them as memory ones), and things like that.

Something that bothers me though, your lib version has only windows and unix platform code? What about others (android, ios, osx), do they still need platform-specific code in allegro part?

I will check this out today.

@rofl0r

It looks like we will need to have a stripped allegro library

do you have an estimate how big that will be ? if it's just a handful of files (like less than 3 KLOCish) it would IMO be a lot less hassle if it was in the main repo rather than a subrepo.

I'm afraid it may be more than that amount, but dont know precisely yet.

[ivan-mogilko]

@sonneveld So, as an experiment, I pushed a temp change with your stripped allegro lib embedded in the engine, https://github.com/ivan-mogilko/ags-refactoring/commits/ags3--sdl2-port3--removeallegro and I continued removing stuff there, aiming at having no single "driver" or "system" object in the library, as imho our goal is to have only functions that work with data passed into them:

• timers, mutexes, threads etc,
• removed all bitmap types except "memory",
• removed gfx driver, screen bitmap and any mentions of those Note I did not clean up the headers for unix etc platforms, but they may be done in same way as I did with the Windows header.

After all this, the noteable problem is FLIC playback. Unfortunately its API was not open enough and implementation was relying on own Allegro's timer (which I removed). So, instead I deleted playback functions from the library code and exported "speed" (basically fps) variable. My plan is to try and recreate playback function in the engine while only using Allegro to read FLIC frames. The function in question is quite simple (https://github.com/adventuregamestudio/lib-allegro/blob/allegro-4.4.3.1-agspatch/src/fli.c#L845), and curiously Allegro seem to export necessary variables too (https://github.com/adventuregamestudio/lib-allegro/blob/allegro-4.4.3.1-agspatch/include/allegro/fli.h#L51).

Finally, there is a number of Windows-only variables that export DirectDraw and DirectInput interfaces. These were used for two things:

• playing AVI videos through Windows API (non portable);
• exposing these interfaces by plugin API (again, non portable); These are mostly useless now when we have SDL backend, and I think we actually may just drop their support and maybe provide better alternative later if needed.

[rofl0r]

I continued removing stuff there

i'd go about it the other way round: remove allegro completely, then copy into the tree the bare minimum the compiler complains about as missing.

[ivan-mogilko]

i'd go about it the other way round: remove allegro completely, then copy into the tree the bare minimum the compiler complains about as missing.

I tried doing this at first, and it caused me lots of headache tbh, as the "missing" parts were going on and on, and I had to search for their location in original allegro, and for their dependencies too.

[ivan-mogilko]

To amend above, in Allegro API things are arranged in files/sections pretty neatly, so cutting off might be easier as you know that you don't need this or that section. On other hand, I would not want to cut literally every unused function from otherwise used section (like particular drawing functions), because there's still a chance some of these might come handy at some point, and it's just simplier to keep them. Still there are maybe few more things that could be removed now, so might do that.

[ivan-mogilko]

Okay, I cautiously hope I'm done with this part. https://github.com/ivan-mogilko/ags-refactoring/commits/ags3--sdl2-port3--removeallegro

• Removed even more code from allegro: soft 3d drawing mostly (I had a crazy idea that if we add simple 3D drawing API into AGS, then we might utilize those for software mode, but they may be returned later).
• Removed non-portable video playback based on DirectX from Windows version (it required directx interfaces provided by Allegro, but it's not clear if we could easily reimplement this since we use SDL2,.. and besides it's non-portable).
• Removed (or rather deprecated) non-portable parts in plugin API, which returned directdraw and directinput for the same reason as above. Frankly I don't remember any contemporary plugin that was using them.
• Reimplemented FLIC playback function in the engine: it's roughly 20 lines of code, and still using allegro for reading FLIC frames.

There may still be some errors and compatibility issues, but guess we'll have to deal with them as they are found, after merging SDL into main branch.

---

So, what should be done now is:

• Finally decide on how to have all these library sources:
  • stripped allegro
  • SDL
  • SDL_Sound
  • MojoAL (couple of smaller files)
• Cleanup & fix SDL branch, using final library locations
• Update CMake files, make sure it builds at least on Linux too
• PR, merge
• Update CI scripts (separate task)

[ericoporto]

About CMake files, if there's interest, I can do them and PR to where the branch is, making sure it builds on Linux. It would be a version of the CMake files here, but without the extra libraries I added: https://github.com/ericoporto/ags

[rofl0r]

Finally decide on how to have all these library sources: stripped allegro SDL SDL_Sound MojoAL (couple of smaller files)

my preference would be to add stripped allegro and MojoAL sources to the tree (MojoAL unfortunately has no releases, so one can't pin down a specific version easily), and "add" SDL* via a script that downloads release tarballs for windows builds, so only the script has to be checked in, just like we did for allegro in the past https://github.com/adventuregamestudio/ags/blob/master/libsrc/download.sh#L20

[ivan-mogilko]

Remade the branch: https://github.com/ivan-mogilko/ags-refactoring/tree/ags3--sdl2-port-full

Contains stripped allegro and mojoAL source, but requires SDL and SDL_sound to be added separately. For example, for MSVS project it replaces AGS_ALLEGRO_INCLUDE/LIB env variable with AGS_SDL_INCLUDE/LIB and AGS_SDL_SOUND_INCLUDE/LIB.

@ericoporto you mentioned that you can update CMakefiles, although I would prefer to add patches or cherrypick commits from somewhere instead of PR, because prs create merge commits.

[rofl0r]

prs create merge commits

not necessarily, the github "merge PR" button has a dropdown option "rebase and merge" which omits the merge commit. it can also be done manually by adding a remote for the other repo, fetch the remote, checkout the PR branch, then rebase that branch on top of the branch it's supposed to be merged into, then checkout the base branch and run git merge pr-branch, which, when successful, shows "fast forward" which means no merge commit was created (optimally one tests this on a backup copy of the git repo in case something goes wrong).

[ericoporto]

Hey @ivan-mogilko, I did a bit of the work on building on Linux with CMake here ericoporto/ags/tree/ags3--sdl2-port-full

I tested a bit with some games and it appears to work. There is a small catch for now that I don't know how to solve that SDL_Sound adds a custom target on it's cmake and names it uninstall, and CMake targets should be unique. When I had it's sources in the repository, my approach was to prefix it. I don't have an obvious solution just yet, so I mailed it's mailing list asking for help.

You can use my source files directly without referring to it's commits or me :). If commits are intended to be ported over, I need to ask how they should be separated - I imagine one for CMake everything, and other for Linux build fixes, and one to up C++ from 11 to 14 in the Engine (for make_unique).

[rofl0r]

and one to up C++ from 11 to 14 in the Engine (for make_unique)

if a C++-14 feature has slipped in, maybe one should try to find a backwards-compatible alternative instead of forcing all users to upgrade their compiler.

[ericoporto]

make_unique -> https://en.cppreference.com/w/cpp/memory/unique_ptr/make_unique

I have a polyfill in one of my branches, mentioned above in the conversation. But upgrading is painless.

Microsoft specifies c++14 as it's default here-> https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=msvc-160 We don't specify a version of cpp on the vcxproj file used, so it picks the default there.

[rofl0r]

upgrading is painless

it depends. if you have latest windows os and a MSDN subscription, then certainly it is easy to update your MSVC (if the huge download is not a problem with your internet connection and disk space). if you don't, then it might become a problem. it's also a problem if you use e.g. centos stable or ubuntu 16.04 LTS which doesn't provide a GCC with C++14. also in case of MSVC, usually a compiler update also "updates" the list of supported target OS, for example it might remove support for windows XP, vista, or even 7.

[morganwillcock]

Unfortunately it might also raise the glibc requirement too, which might remove binary compatibility with older environments. So even being able to build it isn't necessarily enough.

[ericoporto]

I just tested in Ubuntu 16.04 and it builds and runs fine there. Here's GCC features per version: https://gcc.gnu.org/projects/cxx-status.html CentOS case, apparently it has a way to target old glibc from newer gcc -> https://stackoverflow.com/questions/52609269/c-project-compiled-with-modern-compiler-but-linked-against-outdated-libstdc/52611144#52611144 In case of Windows I don't know. Linked MS docs says it's default since 2015, so we are probably using it there.

it might also raise the glibc requirement too

Sure. Our current solution is to build on a older Linux distribution (debian) and hope it will work on modern distros. Our Dockerfile doesn't specify which debian, but assuming Jessie, it has gcc 4.9.2, which supports c++14 - according to above docs, I have not tested.

---

I recommend using the polyfill function below for now, because the single feature in use is really simple -> https://github.com/ericoporto/ags/blob/ags4--sdl2-sound-1/Common/util/nostd_make_unique.hpp

[morganwillcock]

it might also raise the glibc requirement too

Sure. Our current solution is to build on a older Linux distribution (debian) and hope it will work on modern distros. Our Dockerfile doesn't specify which debian, but assuming Jessie, it has gcc 4.9.2, which supports c++14 - according to above docs, I have not tested.

It is passed as an environment argument to the build system, so in theory it can build on multiple versions just by adding more container sources, but yes it is set to use Debian Jessie. From memory, I think we were quite lucky with the versioning for this one, in that the glibc version wasn't raised (so everything stayed working with the pre-built binaries that come with the editor) and it added a compiler feature that was needed for some builds.

[ivan-mogilko]

TBH I was quite lost why there's this issue of C++14 sprung out, as I thought we currently do not use any C++14 in the code... but then I found I brought it with one of the sonneveld's commits and did not notice (facepalm).

If it's desirable to move to the next C++ version we will have to first research where it is supported (and it's particular features, as new standards were sometimes not implemented fully at once), and make sure we don't loose some systems that are still in use or don't introduce difficult requirements for particular ports. We did this with C++11 in the past.

But ofcourse if it's just make_unique, then it would be much simplier to use our own implementation at the moment. I'd really prefer to not include additional dependencies while we are changing backend, as it's worrisome enough already :). EDIT in our particular case it may be replaced with creating unique_ptr directly in code even.

@rofl0r

it depends. if you have latest windows os and a MSDN subscription

To clarify, we are now relying strictly on free (community edition) MSVC.

also in case of MSVC, usually a compiler update also "updates" the list of supported target OS, for example it might remove support for windows XP, vista, or even 7.

I am not sure if it's connected with plain C++ support, it looks more like dependent on runtime library implementation. Since we moved to MSVS 2015 ags engine requires separate build configuration for Windows XP. I have not tried C++14 and onward on XP though. Windows 7 is still my os at the moment, and I wrote and run C++17 programs on MSVS 2019 without any problems. This is only for the reference.

[ivan-mogilko]

So, I tried fixing linux compilation, using old Makefiles for now, made it compile but not link yet (wip commit): https://github.com/ivan-mogilko/ags-refactoring/commit/718ebe6207ebe385eb6b6173b4b4fd800afd9762

There seem to be a problem with SDL_sound. Ubuntu only has old packages, iirc mine is 16.04 LTS, and it has only SDL_sound 1.2, based on SDL1, while our code requires SDL_sound 2.x. I was not yet able to find out if they have newer formal releases. It's possible to download their latest Mercurial revision by hand (https://hg.icculus.org/icculus/SDL_sound/archive/default.zip), but not sure if it's an optimal way and how to include it in the makefile rather than requiring user to set variable externally prior to running make.

[ericoporto]

Icculus mercurial have links to download specific changesets too. Click on files at right of a changeset and then click on the format you want to download the files on top. The download step could be either a download script or a step in the makefile - if downloading source and building is desired.

[rofl0r]

while our code requires SDL_sound 2.x

isn't SDL2_sound already part of SDL2 ? i can only find SDL2_mixer as an external library.

[ivan-mogilko]

isn't SDL2_sound already part of SDL2 ? i can only find SDL2_mixer as an external library.

It's an add-on to SDL: https://www.icculus.org/SDL_sound/

[rofl0r]

sigh. these clowns haven't published a release since they started work on SDL2 version, but one can download a tarball of latest mercurial as https://hg.icculus.org/icculus/SDL_sound/archive/9262f9205898.tar.bz2 (replace .tar.bz2 with .zip for windows unzip program)

edit:

It's possible to download their latest Mercurial revision by hand (https://hg.icculus.org/icculus/SDL_sound/archive/default.zip),

this "default" download shouldn't be used, as it is always pointing to latest commit, i.e. contents and checksum can and will change.

[sonneveld]

sigh. these clowns haven't published a release since they started work on SDL2 version

There's no need to be disrespectful to anyone here. Ryan C Gordon's providing his work for free and we're hoping to build on top of that work.

[ivan-mogilko]

I pushed last changes needed to build the engine on Linux (https://github.com/ivan-mogilko/ags-refactoring/commit/72f4bbe66bff70c4eb165733216d34eb3be7e51d) (without CMake changes yet).

There was another issue, the SDL version our sound code requires is at least 2.0.8, and one coming with Ubuntu 16.04 is 2.0.4, so I had to build necessary SDL version myself too along with SDL_sound. I don't know if this is related, or I screwed up somewhere else, but when running engine I get errors from SDL where it cannot find display and audio device, so could not test this at runtime yet.

[ivan-mogilko]

@ericoporto tried applying your changes to cmake (commit: https://github.com/ivan-mogilko/ags-refactoring/commit/ef9a178fdfcf0a0112f9a2277445ee759eb3002a), but it does not work for me failing at "configuration" step, and because I dont know cmake very well, cannot figure what exactly goes wrong.

EDIT: also, judging by the console output. SDL_sound could not find OpenAL for some reason ("missing: OPENAL_LIBRARY OPENAL_INCLUDE_DIR ")

[rofl0r]

I pushed last changes needed to build the engine on Linux (ivan-mogilko@72f4bbe)

i tried to compile this on linux, but i get

In file included from ../libsrc/allegro/include/allegro/internal/alconfig.h:40:,
                 from ../libsrc/allegro/include/allegro/base.h:41,
                 from ../libsrc/allegro/include/allegro.h:25,
                 from libsrc/apeg-1.2.1/apeg.h:4,
                 from libsrc/apeg-1.2.1/adisplay.c:16:
../libsrc/allegro/include/allegro/platform/alucfg.h:34:39: fatal error: allegro/platform/alunixac.h: No such file or directory
 #include "allegro/platform/alunixac.h"
                                       ^

it seems to be still expecting system-wide installed allegro, even though it is now in-tree. probably adding the right -Iinclude to Engine/Makefile would fix that.

edit nope, the headers aren't there, they would need to be copied into the tree too, or their usage removed.

[ivan-mogilko]

Sorry, I missed that one "alunixac.h". It looks like that header has to be generated (by allegro cmake) depending on system. It contains macros telling what system has enabled. Many of these definitions are no longer necessary for a stripped library, but some still do, like ones that tell endianess. Maybe there's a way to detect these without cmake, or provide in cflags.