skywolf829 / GSTK

Gaussian Splatting toolkit application. One stop shop for preprocessing your dataset, training your model with human-in-the-loop training, and editing saved GSplat PLY files.

MIT License

4 stars 1 forks source link

Backend toy data #2

Closed skywolf829 closed 7 months ago

skywolf829 commented 7 months ago

To avoid needing a full-functioning backend for testing (some aspects of) the frontend, we need some way to generate toy data and transmit it to the frontend.

Requirements

Frontend

Window with a button, which when pressed, will tell the server to begin sending only toy data. Which data to send is TBD and will change as our features progress.
Include relevant information for the test in the sent message, such as "FPS: 120" to test the backend sending images at 120 FPS. Also consider resolution, loss metric updates, etc. Include as much as possible in the message to the backend for testing useful cases.
Include a Stop button to tell the backend to stop sending toy data

Backend

Listen for the message from the frontend instructing the use of toy data, or to stop sending data.
Once signaled for toy data, the backend should begin producing data at the requested frequency/resolution within the ServerController. May require threading.

skywolf829 commented 7 months ago

Based on the results from this (after implemented) we may need to try different socket libraries or connection type (UDP/TCP). I believe the original GS code uses UDP which may be most appropriate, but the multiprocessing.connection library I'm using is TCP I believe.

skywolf829 commented 7 months ago

Added DEBUG mode in backend and a way to enable it on the frontend through a button on a DebugWindow. This allows the user to test the frontend without needing CUDA, any datasets, or real training to take place.

With DEBUG enabled, the Dataset does nothing, and the Trainer's step() function will generate a random image output (noise shape [1024, 1024, 3]) with a random loss each time. On my mac at full speed, this only reaches about ~100 iterations per second. Hopefully the networking isn't limiting speed, and my mac is just slower.