ProjectM Still Freezing with Certain Presets

[stevenswart]

20th of March 2021

This issue is probably not a duplicate of:

https://github.com/projectM-visualizer/projectm/issues/408

I see that bug has been closed some time ago. However, I am still experiencing freezes with certain presets, so it seems that the fix didn’t completely solve the problem with all presets.

My habit had been to move all problem presets out of my preset directory, and into a subdirectory. After I saw that issue 408 had been marked fixed, and still noticed the freezes, I watched my updates and every time the ProjectM library was updated, I would create a new subdirectory. I am on OpenSUSE Tumbleweed.

As of today, the versions I have installed are:

steven@linux-ll3r:~> zypper info libprojectM3
Loading repository data...
Reading installed packages...

Information for package libprojectM3:
-------------------------------------
Repository     : multimedia:apps
Name           : libprojectM3
Version        : 3.1.12-49.3
Arch           : x86_64
Vendor         : obs://build.opensuse.org/multimedia
Installed Size : 1.1 MiB
Installed      : Yes
Status         : up-to-date
Source package : projectM-3.1.12-49.3.src
Summary        : Run-time library for projectM
Description    : 
    projectM is a music visualizer.
    This package contains its runtime library.

steven@linux-ll3r:~> zypper info projectM
Loading repository data...
Reading installed packages...

Information for package projectM:
---------------------------------
Repository     : multimedia:apps
Name           : projectM
Version        : 3.1.12-49.3
Arch           : x86_64
Vendor         : obs://build.opensuse.org/multimedia
Installed Size : 365.3 KiB
Installed      : Yes
Status         : up-to-date
Source package : projectM-3.1.12-49.3.src
Summary        : A Music Visualizer
Description    : 
    projectM is a music visualizer.

Please see attached zip file Freezes.4.zip.

On my Tumbleweed system, the creation dates I have for the subdirectories are as follows:

freezes Sunday, November 1, 2020 10:19:06 PM SAST new Monday, February 22, 2021 11:15:20 PM SAST newer Sunday, February 28, 2021 12:00:46 PM SAST newest Thursday, March 4, 2021 12:31:15 PM SAST newest2 Tuesday, March 9, 2021 11:51:11 AM SAST

I am still regularly getting crashes and freezes, so it seems that I haven’t removed all of the problem presets yet.

Complete list of freezing presets follows – note that some of these may no longer cause the problem because of the fix to issue 408.

3D function draw template [flexi's learning by doing session 05] nz+ nglumbrephonle hsdv.milk
Cope - fractal explorer 115 [Flexis kaleidoscope] tweaq.milk
Fed + Geiss Colorpox remix.milk
Flexi - truly soft piece of software - topology - cohere.milk
N.O. phathmolesc.milk
RLP + Hexcollie + Jc + martin + Geiss - RLP Presents Death the Kid - Glow.milk
Rozzor and Zylot - Associative Order 20030326232843 - AutoSave .EMPR.milk
ShadowHarlequin - Red + Blue - lineart - [Geiss - Aurora 2 v22 22].milk
Stahlregen & Geiss + ORB - Etheral Waves (3 layers of Rayleigh RMX)_1 - var nz+ qrohre.milk
T Miles - Navigator(lightspeed mix) - greyed-out options.milk
Zylot - Paint Spill (Music Reactive Paint Mix) nz+ infinite broadening of quantum.milk
Zylot-static.milk
amandio c - iterative - procedural maps villianesseth.milk
cope + martin - silicon life - 777 3.milk
fed + EoS + Phat - last point of pointfield.milk
fed - glotarine 3.3.milk
file_03.milk
file_04.milk
file_05.milk
file_06.milk
file_08.milk
file_09.milk
file_1.milk
file_Custom Wave 1.milk
file_aural3.milk
file_aurals.milk
file_auralsneg.milk
file_auralw.milk
file_auralw2.milk
file_auralw2neg.milk
file_auralwneg.milk
file_ball.milk
file_ball2.milk
file_bird.milk
file_boatsh.milk
file_boatwa1.milk
file_boatwa2.milk
file_camel1.milk
file_camel2.milk
file_cohere1.milk
file_cohere2.milk
file_dote1.milk
file_dote2.milk
file_dote4.milk
file_dotef1.milk
file_dotef2.milk
file_green.milk
file_n7771.milk
file_n7772.milk
file_n7773.milk
file_n7774.milk
file_phar.milk
file_pharm.milk
file_prim.milk
file_prma2.milk
file_proph.milk
file_proph2.milk
file_roosta1.milk
file_shifter's word wave.milk
file_stal1.milk
file_stal2.milk
file_stal3.milk
file_stal4.milk
file_tlx1.milk
file_tlx2.milk
file_viper.milk
file_white.milk
file_wroam1.milk
file_wroam2.milk
file_wroam3.milk
file_wroam4.milk
file_wstatics1.milk
file_wstatics2.milk
martin - autumn melancholy - dpt wimpaire.milk
martin - autumn melancholy - interolerate - fb2.milk
martin - fresco flash - i know i want you to want it.milk
martin - mandelbox explorer - wreck diver.milk
martin - mandelbox explorer v1 (Bass Movement Mix).milk
martin - on silent paths_Anandamide_Remix.milk
new
pair up gnothing knew nz3.milk
phledge lasso blaq wholes aren't natural wr felix balls and bowels.milk
suksma + boz eos shifter hexcollie - litigua mierrorr kompleht sanguinoose amberet.milk
suksma + rovastar raron orb - emergent phenomena in evolutionary systemic reprisal.milk
suksma - Eo.S. - matrix cube - biochar geomartyr.milk
suksma - eyes that will not see.milk
suksma - fiShbRaiN - witchcraft (phugly shame on remix) - darning kneedless - hot vessel cadavar lexical decoherence.milk
suksma - fram oil filter ad.milk
suksma - medioloom peril firma.milk
suksma - negative infinity for not flinching - swag.milk
suksma - snarlcasm overrated creativity of violent acts roam3 nz+.milk
suksma - target you initiate.milk
technothrob holotorics - mrt spore addict.milk
va ultramix - 109 [spiral].milk
xtramartin (339).milk
xtramartin (723).milk
xtramartin (87).milk

./new:
file_dote3.milk
martin - autumn melancholy - plow look back.milk
martin - on silent paths roam3-.milk
newer
newest
newest2
suksma - death taughtologistix.milk
suksma - do get your hopes down.milk
the finished produck - greyed-out options roam3.milk
xtramartin (401).milk
xtramartin (581).milk

./new/newer:
RLP + martin - neon space ps2 (Akatsuki Mix V2.0).milk
sewn to slavish memes - biblically knowing your enemy nz+ biblical issue.milk
suksma - satanic teleprompter - geiss shade - squeakers.milk
suksma - water cooled red uranium - colgatedbundy.milk

./new/newest:
martin - castle in the air more mbahlsce.milk
suksma - log spec with phase diff - for mushroomhead covers seal sake.milk

./new/newest2:
Cope - fractal explorer x302.milk
Zylot - Paint Spill (Music Reactive Paint Mix) nz+ latsolute mnegatio.milk
how to fuel the fire of the unknowable.milk
i concentrate on your skin, and to your horror, it begins to melt away.milk
martin - mandelbox explorer v1 nz+14.milk
rce-ordinary + flexi - far away distance (bounce visor composite).milk
suksma - Rovastar - Snapshot Of Space - mash0000 - sparkle humpy dumpy.milk
suksma - negative infinity for not flinching - clv.milk
various - whirygig03 - flx rigidigeridoolittle' - 3 binah understanding black saturn threes.milk
xtramartin (561).milk

Freezes.4.zip

[kblaschke]

Many of these presets have issues with the compute shaders. MilkDrop uses HLSL in the presets, while projectM uses OpenGL, which has a different shading language (GLSL). projectM has a converter built in to translate the programs, but there might still be some specific constructs that aren't converted correctly. The hlslparser project hasn't seen any updates in the last 7 years, so unless someone bothers to dig into it, the situation will not improve.

At some point in the future, projectM might use Vulkan for rendering which now natively supports compiling HLSL shaders to SPIR-V, but there's currently no work done or planned to implement this.

[kblaschke]

Some time has passed since my last comment, and I recently reworked the multithreading code in libprojectM. Here's what I can tell about the state of loading presets asynchronously:

• Before my changes, background loading only worked on Linux/macOS, as pthread was used directly. Now it's using std::thread, so threading will work on most if not all platforms (even emscripten).
• Even with threads enabled, it's not as asynchronous as it is expected to be. Issues are:
  • If a new preset is loaded, the thread is started just before the frame render code, and as soon as the first frame pass is done (e.g. everything except applying the composite shader), projectM will wait for the loader thread to finish before continuing. This will trim off a few milliseconds of potential freezing, but still stall rendering until the preset is fully evaluated.
  • As far as I know, texture loading is not done in the thread.
  • Shader compilation also happens in the main render thread, which is actually the main source of freezes as it consists of translating HLSL to GLSL, then compiling and linking the shader program.
• There was a mutex locking preset switching during frame rendering, which possibly wasn't recursive. Now it is, and should not cause deadlocks if the same threads locks the mutex multiple times.

So, to properly fix these freezes, all work that needs to be done before a preset can be displayed should happen in the loader thread, which can take multiple frames for presets with large textures and complex shaders. Here's what needs to be done:

• When a preset needs to be switched, start the loader thread and create a future variable to check if it's done.
• On each frame, check if the loader thread is done (with or without an error), and either display the preset or load the next in line if it was broken.
• As long as the loader thread runs, either queue up or ignore any further preset switch events.
• In the background thread, make sure the required textures are loaded.
• In the background thread, create a new OpenGL context (hint: that's the hard part as it requires platform-dependent stuff like window creation), compile the shaders and then delete the context again.

[revmischa]

Why do we need a new OpenGL context to compile the shaders? Can we not create a new glProgram in the same context?

[kblaschke]

From what I understand: An OpenGL context is tied to a single thread, so if you want to run OpenGL functions in a separate thread, you need a different context, as the context is keeping the current state machine - shader compilation could interfere with what the driver does in the other thread, e.g. rendering things or uploading data to the GPU. Most resources can be shared between contexts. So to compile a shader asynchronously, is needs to be done in both a separate thread and context.

[kblaschke]

Over two years have passed since this issue was opened, and in the meantime most of the projectM internals were rewritten. I've not seen any crashes or freezes in the past year or so, and no new issues in that regard were posted. Will close this as fixed then.