Can you add a version that runs on Linux servers?

Xudong-Mao commented 3 months ago

I have a server but no interface, I can only use bash, all the OSError when downloading the model, it gives me a headache to find a solution, please can you release a version that can run directly in the terminal as soon as possible.

Tremeschin commented 3 months ago

I'm inexperienced with headless systems, but one user on Discord managed to run it on a headless Ubuntu server successfully this week

You'll need to set SHADERFLOW_BACKEND=headless env var for shaderflow to not use glfw, and he needed to prepend the command with xvfb-run or similar to have a headless OpenGL context created

Not sure what OSError you faced, linux should be the least problematic platform as I develop on it, if you could give more info I can help further!

GentlemanHu commented 2 months ago

I'm inexperienced with headless systems, but one user on Discord managed to run it on a headless Ubuntu server successfully this week

You'll need to set SHADERFLOW_BACKEND=headless env var for shaderflow to not use glfw, and he needed to prepend the command with xvfb-run or similar to have a headless OpenGL context created

Not sure what OSError you faced, linux should be the least problematic platform as I develop on it, if you could give more info I can help further!

I tried with SHADERFLOW_BACKEND=headless xvfb-run depthflow on ubuntu server, seems audio related errors

----------------------------------------------------------------------------------------------------------------------------------------
5 <module> /content/depthflow/DepthFlow/BrokenSource/.venv/bin/depthflow
> from DepthFlow.__main__ import main

5 <module> /content/depthflow/DepthFlow/BrokenSource/Projects/DepthFlow/DepthFlow/__main__.py
> from DepthFlow.DepthFlow import DepthFlowScene

9 <module> /content/depthflow/DepthFlow/BrokenSource/Projects/DepthFlow/DepthFlow/DepthFlow.py
> from ShaderFlow.Scene import ShaderScene

55 <module> /content/depthflow/DepthFlow/BrokenSource/Projects/ShaderFlow/ShaderFlow/Scene.py
> from ShaderFlow.Modules.Audio import ShaderAudio

9 <module> /content/depthflow/DepthFlow/BrokenSource/Projects/ShaderFlow/ShaderFlow/Modules/Audio.py
> import soundcard

4 <module> /content/depthflow/DepthFlow/BrokenSource/.venv/lib/python3.11/site-packages/soundcard/__init__.py
> from soundcard.pulseaudio import *

290 <module> /content/depthflow/DepthFlow/BrokenSource/.venv/lib/python3.11/site-packages/soundcard/pulseaudio.py
> _pulse = _PulseAudio()

101 __init__ /content/depthflow/DepthFlow/BrokenSource/.venv/lib/python3.11/site-packages/soundcard/pulseaudio.py
      # these functions are called before the mainloop starts, so we
      # don't need to hold the lock:
      self.mainloop = _pa.pa_threaded_mainloop_new()
      self.mainloop_api = _pa.pa_threaded_mainloop_get_api(self.mainloop)
      self.context = _pa.pa_context_new(self.mainloop_api, self._infer_program_name().encode())
      _pa.pa_context_connect(self.context, _ffi.NULL, _pa.PA_CONTEXT_NOFLAGS, _ffi.NULL)
      _pa.pa_threaded_mainloop_start(self.mainloop)

      while self._pa_context_get_state(self.context) in (_pa.PA_CONTEXT_UNCONNECTED, _pa.PA_CONTEXT_CONNECTING, _pa.PA_CONTEXT_AUTHORIZING, _pa.PA_CONTEXT_SETTING_NAME):
          time.sleep(0.001)
>     assert self._pa_context_get_state(self.context)==_pa.PA_CONTEXT_READY

  @staticmethod
  def _infer_program_name():
      """Get current progam name.

      Will handle `./script.py`, `python path/to/script.py`,
      `python -m module.submodule` and `python -c 'code(x=y)'`.
      See https://docs.python.org/3/using/cmdline.html#interface-options
      """
      import sys

AssertionError

Same command `SHADERFLOW_BACKEND=headless xvfb-run shaderflow` runs no error

root@ngkqo77b4r:/content/depthflow/DepthFlow/BrokenSource# SHADERFLOW_BACKEND=headless xvfb-run shaderflow
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                                      │
│                █████████  █████                    █████                    ███████████ ████                                         │
│               ███░░░░░███░░███                    ░░███                    ░░███░░░░░░█░░███                                         │
│              ░███    ░░░  ░███████    ██████    ███████   ██████  ████████  ░███   █ ░  ░███   ██████  █████ ███ █████               │
│              ░░█████████  ░███░░███  ░░░░░███  ███░░███  ███░░███░░███░░███ ░███████    ░███  ███░░███░░███ ░███░░███                │
│               ░░░░░░░░███ ░███ ░███   ███████ ░███ ░███ ░███████  ░███ ░░░  ░███░░░█    ░███ ░███ ░███ ░███ ░███ ░███                │
│               ███    ░███ ░███ ░███  ███░░███ ░███ ░███ ░███░░░   ░███      ░███  ░     ░███ ░███ ░███ ░░███████████                 │
│              ░░█████████  ████ █████░░████████░░████████░░██████  █████     █████       █████░░██████   ░░████░████                  │
│               ░░░░░░░░░  ░░░░ ░░░░░  ░░░░░░░░  ░░░░░░░░  ░░░░░░  ░░░░░     ░░░░░       ░░░░░  ░░░░░░     ░░░░ ░░░░                   │
│                                                                                                                                      │
╰───────────────────────────────────────────── Made with ❤️ by BrokenSource, Python 3.11.8 ─────────────────────────────────────────────╯

 Usage: shaderflow [OPTIONS] COMMAND [ARGS]...                                                                                          

 🌵 Imagine ShaderToy, on a Manim-like architecture. That's ShaderFlow.                                                                 
 • Tip: run "shaderflow (scene) --help" for More Options ✨                                                                             
 • Warn: Make sure you trust the File you are running                                                                                   

 ©️ Broken Source Software, AGPL-3.0-only License.                                                                                       

╭─ 🎥 ShaderScenes at (/content/depthflow/DepthFlow/BrokenSource/Projects/ShaderFlow/ShaderFlow/Resources/Scenes/Examples/Demo.py) ────╮
│ audio        Basic audio processing                                                                                                  │
│ bars         Basic music bars                                                                                                        │
│ bouncing     Bouncing Logo animation                                                                                                 │
│ default      The most basic ShaderFlow Scene, the default shader                                                                     │
│ dynamics     Second order system                                                                                                     │
│ life         Conway's Game of Life in GLSL                                                                                           │
│ multipass    Many Layers ('Buffers') done on a single shader                                                                         │
│ nested       Basic scene with two shaders acting together, main shader referencing the child                                         │
│ noise        Basics of Simplex noise                                                                                                 │
│ raymarch     Ray Marching demo                                                                                                       │
│ shadertoy    ShaderToy Default Shader                                                                                                │
│ temporal     Poor's man Motion Blur. If you dislike the effect, definitely don't run this                                            │
│ video        Video as a Texture demo                                                                                                 │
│ visualizer   Radial Bars Music Visualizer Scene                                                                                      │
│ waveform     Audio Waveform Oscilloscope demo                                                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

 • Made with ❤ by Broken Source Software v0.2.0                                                                                         
 → Consider Sponsoring my Open Source Work

GentlemanHu commented 2 months ago

I just comment the error code, can successfully run on server now. Maybe there is a better solution.

https://github.com/GentlemanHu/ShaderFlow https://github.com/GentlemanHu/BrokenSource

pip install git+https://github.com/GentlemanHu/BrokenSource

https://github.com/BrokenSource/DepthFlow/assets/34559079/bbc3fb0e-ba48-4e65-8ee4-7ecbe5702d2f

Tremeschin commented 2 months ago

I've removed the necessity to import the ShaderAudio class on the scene file on latest commits^everywhere, so DepthFlow doesn't require importing soundcard anymore, which might indeed fail on audio server-less systems

_{(it was used to filter only audio modules on the scene to find the longest one, but now all modules shall advertise a duration)}

~

And given a rise of headless rendering interest, I've given some love to it and ironed out some issues with it

A command like this is possible now:

SHADERFLOW_BACKEND=headless depthflow input -i ~/image.jpg main -r -o /tmp/temp.mp4 -f 30 -w 720 -q 95 -s 2 -t 5

Adding a early exit() on Audio.py the program doesn't quit, meaning it's not importing soundcard at all, and the final video is properly rendered on correct aspect ratio to the image, etc

It's only available when running from source at the time of writing this, but I'll work myself to a new minor PyPI wheel release, as there's been a lot of QOL

hucara commented 2 months ago

I've removed the necessity to import the ShaderAudio class on the scene file on latest commitseverywhere, so DepthFlow doesn't require importing soundcard anymore, which might indeed fail on audio server-less systems

(it was used to filter only audio modules on the scene to find the longest one, but now all modules shall advertise a duration)

~

And given a rise of headless rendering interest, I've given some love to it and ironed out some issues with it

A command like this is possible now:
SHADERFLOW_BACKEND=headless depthflow input -i ~/image.jpg main -r -o /tmp/temp.mp4 -f 30 -w 720 -q 95 -s 2 -t 5
Adding a early exit() on Audio.py the program doesn't quit, meaning it's not importing soundcard at all, and the final video is properly rendered on correct aspect ratio to the image, etc

It's only available when running from source at the time of writing this, but I'll work myself to a new minor PyPI wheel release, as there's been a lot of QOL

A headless mode would be highly appreciated. I'm still getting errors like, I guess due to not running it from code?

Exception: (standalone) XOpenDisplay: cannot open display

Tremeschin commented 2 months ago

Exception: (standalone) XOpenDisplay: cannot open display

@hucara

You'll want to run with xvfb-run by prepending the command or setting a valid xserver running on some DISPLAY=:0 (or other number) for X11 - or Wayland equivalent - to be able to create and use OpenGL contexts :) !

^{(Examples below)}

For that, after some hefty research this past days, TLDR; turns out xvfb-run uses software rendering by default, giving terrible rendering performance. But there's a way around it, to find/create GL contexts on ModernGL with EGL

https://github.com/BrokenSource/ShaderFlow/commit/1892b6b983acb2baafbedeb5c14fd8aeaa661645#diff-00377b1f67bd85923fa2febc0605f35dc920c2cdca7ff761fee289454ab25305R447

This line should make it possible to use GPU acceleration for true headless mode, one user on Discord had success with a manual intervention before this commit on a headless server. Example command:

SHADERFLOW_BACKEND=headless xvfb-run depthflow (...)
SHADERFLOW_BACKEND=headless xvfb-run depthflow input -i ~/image.png main -o /tmp/temp.mkv

If you still get low performance (about 5 fps for a 2 channel DDR4 3200 MT/s system), you could try setting the GLCONTEXT_DEVICE_INDEX=N variable to common values, like 0, 1, 2

It'll either use CPU, crash, use GPU, or warn there's no more than N devices for these options

Tremeschin commented 2 months ago

On the same commit referenced above, apparently xvfb-run isn't needed at all - as we're asking for EGL to provide a OpenGL context, which can, and is headless+GPU accelerated on ModernGL's headless "window" backend

One user ran successfully today on a true headless Ubuntu server with NVIDIA Proprietary drivers just by using SHADERFLOW_BACKEND=headless depthflow (...) with nominal performance

I'll have a PyPI wheel soon™ with those new fixes, will be version 0.2.0

Feel free to reopen / create a new issue if it fails with other configurations such as AMD/Intel GPUs on servers !

Tremeschin commented 1 month ago

IMPORTANT: The config env var SHADERFLOW_BACKEND was renamed to WINDOW_BACKEND to 1. Reflect on moderngl-window naming and 2. It wasn't really a "hidden/secondary" engine to shaderflow itself as the name suggested

Tremeschin commented 1 month ago

Commenting more findings, the import soundcard errors go away when importing ShaderAudio class if the pulseaudio packages are installed on the server's package manager (e.g. apt install pulseaudio). Not really surprising tbh, I thought an actual audio server had to be running, but no, just finding the libs is enough

And adding

ENV NVIDIA_VISIBLE_DEVICES="all"
ENV NVIDIA_DRIVER_CAPABILITIES="all"

..to the dockerfile seem to help a lot when the host os is native linux, alongside ENV WINDOW_BACKEND="headless" ofc :)

BrokenSource / DepthFlow