Closed Kaned1as closed 2 months ago
you may be interested in the gstreamer plugin for graphics pipelines https://github.com/projectM-visualizer/gst-projectm
@revmischa thank you, but as I can see it relies very heavy on gstreamer-provided OpenGL context, which I don't have :smiling_face_with_tear:
@revmischa thank you, but as I can see it relies very heavy on gstreamer-provided OpenGL context, which I don't have 🥲
Yes but you could use gstreamer to replace your entire pipeline. It does internally what you're doing - piping ffmpeg outputs and inputs, but in a more natural and holistic fashion. Gstreamer is exactly for what you are trying to do here.
@revmischa I meant that I won't have gstreamer OpenGL context because I eventually plan to use OSMesa on a completely headless machine, without audio and without any GPU, hardware acceleration or framebuffer :smiling_face_with_tear:
Sorry to bother you again, I will try to figure it out by myself
Not exactly sure why the glow is missing, as the rendering output should just contain whatever projectM draws.
I've opened a pull request yesterday adding an API call to render to a custom FBO, allowing to pass in a custom framebuffer with an in-memory texture attachment. This could possibly solve the issue, as you don't have to rely on the default framebuffer and surface. Edit: It's now merged to master.
I'll try out your code over the coming days when I've got some spare time and see what I can find.
Not really related to the issue, but as a note: projectM currently uses the system clock to determine animation speeds in many different locations. If you render frames as fast as projectM can handle it, you'll probably end up with a different frame rate than the targeted 60 FPS, which will result in either slowed down animations (if encoding FPS are higher than real time) or sped up (if FPS are slower, e.g. projectM can't keep up with 60 FPS).
For libprojctM 4.2,we're also planning on adding a new API function to set the exact time of the next frame from the outside, so the rendering won't be tied to the system clock anymore. This way, you'll get stable animation speeds no matter how fast or slow you're encoding the video.
@kblaschke yes! That's exactly what I did! I added a new API function to set fixed fps so that timekeeping is linear. You will probably need my patch to try the code above, I'll attach it in the evening once I'm home. Thank you for looking into this!
Here's the patch for the timekeeper
diff --git a/src/api/include/projectM-4/parameters.h b/src/api/include/projectM-4/parameters.h
index 2d9c70c89..4df06c5b3 100644
--- a/src/api/include/projectM-4/parameters.h
+++ b/src/api/include/projectM-4/parameters.h
@@ -187,6 +187,21 @@ PROJECTM_EXPORT void projectm_get_mesh_size(projectm_handle instance, size_t* wi
*/
PROJECTM_EXPORT void projectm_set_fps(projectm_handle instance, int32_t fps);
+/**
+ * @brief Set the current fixed frames per second.
+ *
+ * If this value is set to non-zero, timekeeping updates frames at a fixed rate.
+ * The application running projectM is expected to fill the audio buffer with
+ * the audio content up to expected for the current frame.
+ *
+ * E.g., a value of 60 fps means the application is expected to fill
+ * 16ms of audio each time render_frame is called.
+ *
+ * @param instance The projectM instance handle.
+ * @param fps The fixed FPS value projectM is expected to work with.
+ */
+PROJECTM_EXPORT void projectm_set_fixed_fps(projectm_handle instance, int32_t fps);
+
/**
* @brief Returns the current/average frames per second.
*
diff --git a/src/libprojectM/PresetFactory.cpp b/src/libprojectM/PresetFactory.cpp
index 3c56876b2..bde847145 100644
--- a/src/libprojectM/PresetFactory.cpp
+++ b/src/libprojectM/PresetFactory.cpp
@@ -18,7 +18,7 @@ std::string PresetFactory::Protocol(const std::string& url, std::string& path)
path = url.substr(pos + 3, url.length());
#ifdef DEBUG
- std::cout << "[PresetFactory] Filename is URL: " << url << std::endl;
+ std::cerr << "[PresetFactory] Filename is URL: " << url << std::endl;
#endif
return url.substr(0, pos);
}
diff --git a/src/libprojectM/ProjectM.cpp b/src/libprojectM/ProjectM.cpp
index ae9390efa..98e5b95fb 100644
--- a/src/libprojectM/ProjectM.cpp
+++ b/src/libprojectM/ProjectM.cpp
@@ -354,6 +354,10 @@ void ProjectM::SetTargetFramesPerSecond(int32_t fps)
m_targetFps = fps;
}
+void ProjectM::SetFixedFramesPerSecond(int32_t fps) {
+ m_timeKeeper->SetFixedFps(fps);
+}
+
auto ProjectM::AspectCorrection() const -> bool
{
return m_aspectCorrection;
diff --git a/src/libprojectM/ProjectM.hpp b/src/libprojectM/ProjectM.hpp
index b84e29335..cf9de3ad9 100644
--- a/src/libprojectM/ProjectM.hpp
+++ b/src/libprojectM/ProjectM.hpp
@@ -143,6 +143,15 @@ public:
*/
void SetTargetFramesPerSecond(int32_t fps);
+ /**
+ * @brief Sets a new fixed frames per second value.
+ * Timekeeping will use this value instead of OS time.
+ *
+ * @param fps The new frames per second value.
+ *
+ */
+ void SetFixedFramesPerSecond(int32_t fps);
+
auto AspectCorrection() const -> bool;
void SetAspectCorrection(bool enabled);
diff --git a/src/libprojectM/ProjectMCWrapper.cpp b/src/libprojectM/ProjectMCWrapper.cpp
index cbf5f6edc..e25a43beb 100644
--- a/src/libprojectM/ProjectMCWrapper.cpp
+++ b/src/libprojectM/ProjectMCWrapper.cpp
@@ -270,6 +270,13 @@ void projectm_set_fps(projectm_handle instance, int32_t fps)
projectMInstance->SetTargetFramesPerSecond(fps);
}
+void projectm_set_fixed_fps(projectm_handle instance, int32_t fps)
+{
+ auto projectMInstance = handle_to_instance(instance);
+ projectMInstance->SetFixedFramesPerSecond(fps);
+
+}
+
void projectm_set_aspect_correction(projectm_handle instance, bool enabled)
{
auto projectMInstance = handle_to_instance(instance);
@@ -374,4 +381,4 @@ auto projectm_pcm_add_uint8(projectm_handle instance, const uint8_t* samples, un
auto projectm_write_debug_image_on_next_frame(projectm_handle, const char*) -> void
{
// UNIMPLEMENTED
-}
\ No newline at end of file
+}
diff --git a/src/libprojectM/TimeKeeper.cpp b/src/libprojectM/TimeKeeper.cpp
index 612b0ab33..f0a5823c3 100644
--- a/src/libprojectM/TimeKeeper.cpp
+++ b/src/libprojectM/TimeKeeper.cpp
@@ -1,5 +1,6 @@
#include "TimeKeeper.hpp"
+#include <math.h>
#include <algorithm>
#include <random>
@@ -16,11 +17,17 @@ TimeKeeper::TimeKeeper(double presetDuration, double smoothDuration, double hard
void TimeKeeper::UpdateTimers()
{
- auto currentTime = std::chrono::high_resolution_clock::now();
+ double currentFrameTime = 0;
+ if (m_fixedFps != 0) {
+ m_secondsSinceLastFrame = 1.0 / m_fixedFps;
+ m_currentTime += m_secondsSinceLastFrame;
+ } else {
+ auto currentTime = std::chrono::high_resolution_clock::now();
+ currentFrameTime = std::chrono::duration<double>(currentTime - m_startTime).count();
+ m_secondsSinceLastFrame = currentFrameTime - m_currentTime;
+ m_currentTime = currentFrameTime;
+ }
- double currentFrameTime = std::chrono::duration<double>(currentTime - m_startTime).count();
- m_secondsSinceLastFrame = currentFrameTime - m_currentTime;
- m_currentTime = currentFrameTime;
m_presetFrameA++;
m_presetFrameB++;
}
diff --git a/src/libprojectM/TimeKeeper.hpp b/src/libprojectM/TimeKeeper.hpp
index 23d17df6c..d320f7f41 100644
--- a/src/libprojectM/TimeKeeper.hpp
+++ b/src/libprojectM/TimeKeeper.hpp
@@ -86,6 +86,10 @@ public:
return m_secondsSinceLastFrame;
}
+ inline void SetFixedFps(int32_t fps) {
+ m_fixedFps = fps;
+ }
+
private:
/* The first ticks value of the application */
std::chrono::high_resolution_clock::time_point m_startTime{std::chrono::high_resolution_clock::now()};
@@ -107,6 +111,7 @@ private:
double m_presetTimeA{};
double m_presetTimeB{};
+ int m_fixedFps{0};
int m_presetFrameA{};
int m_presetFrameB{};
I've actually implemented it a bit differently, as per #740, allowing to pass in the actual frame time in fractional seconds. This is a bit more flexible, as it allows to set frame times with dynamic framerates, which may occur in video capturing or other non-deterministic rendering scenarios. In fixed-FPS use cases, the app can simply increment the frame time with the 1/FPS fraction.
The preset transition class also had to be fixed to use the TimeKeeper class. See PR #817 for details.
One other thing to note here is that the code example in the initial post uses EGL, which projectM currently doesn't support - it requires a GLX/GL Core context, as it internally uses the GLX vendor library for rendering. While it may work in general, some things will be broken. We have the exact same issue with recent Qt versions, as they also switched their GL rendering stack to EGL to support Wayland.
We already have issue #681 for this change, but it'll require some rework of how projectM uses OpenGL, plus potentially adding a build flag to choose between EGL and GLX. You can track the progress there for updates.
Since offscreen rendering support using a custom FBO/texture is now implemented in the master branch and the other potential issue is already tracked, I'll close this one.
Understood! Thank you for spending your time on this!
Please confirm the following points:
Topic
Development and Contributing
Your Request
Hello projectM team! I'm trying to understand what's wrong with my offscreen rendering pipeline. I launch it with the following command:
Here's what I get:
Here's what I expect to get:
As you can see there's a certain "glow" around the edges. Somehow when I render it offscreen it is not present. I'm kind of a rookie in graphics pipelines but I'm trying to implement offscreen rendering for ProjectM right now and before I submit a PR i need to get it right.
Here's my code that I use for offscreen rendering
```cpp #include "projectM-4/core.h" #includeI can also upload the full video or my small changes to timekeeping if you need it. I haven't touched any of GL-related code.