Access framebuffer data directly?

[hjhinton]

Hello! I've been putting together a little interface in Qt to take data from a local hydra instance and display it on an LED matrix. Right now, I can grab the screen to downsample, but it would be great if the output frame buffer were able to be directly accessed as an object -- is this possible via the js console? It would speed up frame access and also allow for not having to hide the on-screen code editor. Thanks!

[geikha]

You can access an output's pixel values via regl. I've made an extensions that simplifies it:

https://github.com/ritchse/hydra-extensions/blob/main/doc/hydra-pixels.md

[hjhinton]

Amazing, works perfectly. Thank you! For reference, here is a snippet from the PyQt code:

self.webview = QWebEngineView()
self.page = WebEnginePage()
self.webview.setPage(self.page)
self.page.load(QUrl("https://localhost:8000/?")) # local hydra instance
# ... more between here
self.page.runJavaScript(str("shape(3).out()\n render(o0)\nchoo.state.showInfo=false"), self.ready)
self.page.runJavaScript(str('render(o0)'), self.ready)
self.page.runJavaScript(str('window.regl = o0.regl'), self.ready)
self.page.runJavaScript(str('regl.attributes.preserveDrawingBuffer = true'), self.ready)
self.page.runJavaScript(str('window.Output = o0.constructor.prototype'), self.ready)
self.page.runJavaScript(str('''Output.readAll = function () {
                                    return regl.read({
                                        framebuffer: this.fbos[this.pingPongIndex],
                                        x: 0,
                                        y: 0,
                                        width: window.width,
                                        height: window.height
                                    })
                                }'''), self.ready)

[geikha]

nice! feel free to close this issue