search

brownhci / WebGazer

WebGazer.js: Scalable Webcam EyeTracking Using User Interactions
https://webgazer.cs.brown.edu
Other
3.54k stars 536 forks source link

Further speed up mat.js with OpenGL + LinAlg libraries #31

Open Skylion007 opened 8 years ago

Skylion007 commented 8 years ago

Currently, mat.js relies on naive, un-optimized code ported from Java to for use throughout the libraries. We should look into porting some of this code to rely on optimized, high performance code. If anyone has any Javascript libraries they would like to recommend, please list them here.

Skylion007 commented 8 years ago

Here are a few that may work well:

https://github.com/themadcreator/luqr https://www.npmjs.com/package/m4th https://github.com/larspars/linalg.js/

Skylion007 commented 8 years ago

So apparently, CLMR tracker relies on numeric.js as a dependency. Numeric.js is a very useful Javascript library that includes a highly optimized linear algebra library. Commit e8a9de61214cdf335941580bbef1ac60b3d05c01 begins the process by replacing the unoptimized mat.js function with a call to Numeric.js. Note: Numeric.js does not currently support QRDecomposition, we will need to keep that implemented, but it does support matrix multiplication so we should replace matrix multiplication calls with it. On a side note, we may want to submit our QR Decomposition algorithm as a pull request to numeric.js

Skylion007 commented 8 years ago

Now that we have an optimized version of LUDecomposition, we may be able to still further improve performance if we use the GPU to parallel these operations. Finally found a Javascript library that may be suited to the task of implementing complex functions such as LUDecomposition within the GPU. If anyone can these operations implemented, let me know. They also have a nice little kernel playground here

Update: I have actually found several libraries that could be useful. https://github.com/stormcolor/webclgl

Additionally, if anyone has any experience with some of these GPU libraries feel free to give a shot and implementing and bench marking the results.

Skylion007 commented 4 years ago

Faster matrix libraries now exists that can operate on typed Javascript arrays on both GPU and CPU. When #118 lands we should consider rewriting our matrix libraries to do as much compute on the GPU as possible and also use the various utilities such as image resizing.

Skylion007 commented 3 years ago

WASM SIMD has landed into Firefox and Chrome so we can use that now for even further performance. We should also update TF.js to use the WASM backend when available.