search

ubcemergingmedialab / SynesthesiAR

EMLx - A music vizualizer / Synesthesia simulator for the Hololens
MIT License
3 stars 0 forks source link

Set up image to sound processing #11

Open allen6chen opened 4 years ago

allen6chen commented 4 years ago

Using Hololens' PhotoCapture api, get the frame the user is looking at.

CopyRawImageIntoBuffer might help with this - instead of storing the photo info as a texture, we just want to do frame analysis. The link shows iterating over the buffer and storing it as a list of colors. This should be pretty useful for our purposes too - we can take the midpoint color (or an average of several colors if we want) and return that

unoctium1 commented 4 years ago

https://github.com/VulcanTechnologies/HoloLensCameraStream This seems perfect for us actually