Any plans on scaling?

[shangjiaxuan]

It seems the repo has seen HUGE refactoring recently focusing on decoupling the modules and perhaps implementing an event based system (If I understand correctly.). Does this mean

How are you planning to implement multi-threading, heap management and perhaps 3d graphics (displaying models and transforms)?

My toy project (mainly focused on learning opengl and basics of sound playback (writing pcm to audio buffers in callbacks)) has stalled on a similar refactoring and rewrite, and would really appreciate if you would like to share some ideas of the current design and some tips on where to help with this project (With this new virus spreading, there's not much else to do anyway.).

(As for the reason my project was stalled: I was trying to implement an abstraction layer over the available threading primitives (atomic spin lock is the farthest one can go without interacting with the platform api for thread scheduling with C++17 without fibers, and the std::mutex on msvc was said to be using sub-optimal implementation that goes to process level (unverified), semaphores and condition variables have similar problem (e.g. most PC have hardware support for watching a word and notifying when it changes, and is exposed by os' api), c++ also does not expose the read-write lock available on modern OS (of course one can implement the states oneself, but that's overhead.)). The types of heaps that may be needed also seemed especially daunting, whether locked in ram, allocation scheme, cache locality, etc.)

[RinGotou]

Yes, in my current roadmap I will try to use a universal event interface to replace ad-hoc SDL2 event handler, but I'm now focusing on basic 2D graphic.(although I'm dealing with the extension system and studying in these days...) This interpreter will not contain multi-threading feature in current backend structure(Maybe I will write a new machine if I can?), but I have some idea about ruby-like Fiber support and need to change some codes in kagami::Machine class(add facilities for switching between contexts). I think...I will upload some documents(about main sturcture and extension system, in Chinese) after getting my postgraduate exam scores. This project is also my final year project and I need to arrange old documents. //sorry for my creepy English...

[shangjiaxuan]

I'm also Chinese, so if you feel better writing Chinese, just do it.

The idea of using fiber over single threaded is that some operations may block, and doing something else instead of wasting CPU may be better, for example slow IO operations, user input, etc. will typically block until the buffer is filled, or input is received (e.g. polling events vs. waiting for events). It can save resource by actively yielding to the other tasks that are not dependent on the current one while waiting for the resource to become available.

A simple implementation, I suppose, would be to use setjmp and longjmp from the C library to implement the context switch (or perhaps boost?), and a scheduler main loop on the fiber-ed thread to switch between the environments (or perhaps use the platform dependent api?).

Edit: With some searching, I found this thread, which in turn lead to this emulator component which claims to do this much faster(with assemblies). However, it is not guaranteed that problems like this one won't emerge, so the choice is yours (Also, C++20 is promising coroutines, using that may be an option).

[RinGotou]

To tell the truth, I don't have a certain implement design now, I'm still learning the details of other project's structure about fiber, these informations can help me to find out a suitable design. Thanks! For graphics/multimedia, I decide to use TOML as UI description language to simplify UI creating experience, and replace SDL_mixer with other sound library(SDL_Mixer doesn't include some feature I'd like to use such as variation/audio length for variable bit rate files/etc.)

[shangjiaxuan]

I just looked into the setjmp and longjmp specifications, and it seems that the stack environment of a callee is not guaranteed to be preserved after a call to longjmp. i.e. one will always need a separate stack for a different co-routine. In fact, I tested the example at Wikipedia, and it only works on msvc x86 target, but not with x64, emitting stack unwind error from ntdll.dll. From this, I suggest perhaps using boost for this specific purpose (dealing with stack pointers will always end up in assembly or platform api)?

For audio, from what I've learned, the high-end industry standard is third-party (mostly paid or gpl-ed) frameworks like Fmod, or openal (there's a software implementation that is LGPL, Apple ships its own version with coreaudio) standard(?), or platform api like XAudio2 (successor to directsound). I suggest using commercial frameworks if you plan on going to consoles and make things commercial, use openal for things that are compatible with lgpl and runs on PC (typically working with other lgpl components like ffmpeg on linux).

The mentioned apis typically works with 3d sound, and define dsp as filters. They typically work with decompressed audio buffers and perform a fixed pipeline that mix the sound to output (a mixer at the core) on a high priority thread. The controlling threads change the state of the buffers by streaming the buffer, issuing commands like play/stop/volume, changing pipeline configurations, etc. Decoding of audio and video are typically handled by the commercial frameworks or some platform api (e.g. media framework on windows).

On platforms where these are not present, the engine must come with its own decoder. Typically these games restrict the bgm to the ogg/vorbis format for license issues (aac audio codec, h264 video codec has licensing issues, wmv is much restricted to the Microsoft platform (VC1 is not that much used), etc. Games typically resort to either a commercial third-party codec and sdk (e.g. the bink format)), use the platform framework (wmv in directshow), or use mpeg1 or mpeg2 formats (patents expired). I was looking into the open-media vp8, vp9, and av1 codecs, but the docs and library support is still limited, and requires much cpu resource in software decode

Hardware vpx video decode is not really well-maintained on the windows platform, even though modern cpus and gpus support it, typically one would need to call the dxva functions manually or use the media foundation transforms. But still, even with the VPx codec installed from microsoft store, one can still not find the VPx decoder from MFTEnumEx, the class is registered in the registry but not under media foundation transform (Latest Win10 Pro installed). Most likely Microsoft reserved the method to retrieve the COM object to themselves with the Movies and TV app, perhaps? With this lacking, one will need to resort to the vendor media sdks. Linux seems much better in this by using the va-api (and on linux one can assume the installation of ffmpeg, and all is smooth.).

I also feel quite complicated in regards to the codec problem, and was trying to just support a few codecs from the start, and see if some abstraction or better alternative comes around. I suggest choosing from flac, vorbis and opus for audio codec, vp8 vp9 and av1 for video codec (they are all permissively licensed with opensource decoder and encoders). Doing the format conversion outside the run-time can greatly reduce the work needed to be done (or you can just use ffmpeg on linux for pretty much all the codec if you are happy with LGPL.).

Edit: I just looked at the sdl mixer docs, and it seems that it handles some of the codecs under the hood, and is licensed under zlib license. However, decoding is still done by using the respective libraries dynamically loaded (LGPL). I suppose you can make sdl write the decoded sound buffer to your own buffers and use sdl for decoding some formats, but that level of abstraction is better handled by ffmpeg, since the decode will always need LGPL and dynamic loading (if without a commercial license, many codec libraries are dual-licensed under gpl(or lgpl) and commercial), but ffmpeg seems much harder to use. (Also note that some codecs charges fees for play time from content providers, etc.)