search

sangkeon / java-opa-wasm

Apache License 2.0
15 stars 2 forks source link

Poor performance of wasm binary execution over remote REST api call of OPA server. #10

Open rai-prashanna opened 1 year ago

rai-prashanna commented 1 year ago

I notice that when experimenting with wasm binary, evaluation of input against wasm binary was very poor in comparison to the remote call of REST API of OPA server. I experimented with wasm binary, JARL library, OPA rest API server. I created bundle with optimization flag=1 i.e o=1 and run as server.

Both plan.json and wasm files are located locally Despite OPA being run as a REST API server in a separate container with an overhead of sending and processing requests/responses. poor-performance The green line is used for OPA rest, The violent line is used for WASM, The purple line is used fro JARL.

My policy can be found in this link. https://play.openpolicyagent.org/p/W7dh9TIqMi https://play.openpolicyagent.org/p/LYgbgZj0Qg

My link to the repo of testing https://github.com/rai-prashanna/performanceTesting

The performance of the wasm doesn't match with the details mentioned in this document. https://www.openpolicyagent.org/docs/latest/integration/ image

sangkeon commented 1 year ago

Using languages doesn't support web assembly natively(such as Java, Python and etc), many overhead can be introduced.

If you want to measure pure performance of wasm compiled policy, I recommend using npm-opa-wasm(https://github.com/open-policy-agent/npm-opa-wasm) developed by the OPA team.

rai-prashanna commented 1 year ago

So, it is wasm runtime like wasmtime that is degrading performance. I thought that regardless of what kind of wasm runtime is used, wasm binary execution will at least have almost similar timing compared to OPA-based REST API operations. did I understand correctly?

sangkeon commented 1 year ago

It's not related pure performance of the wasmtime. I don't know about performance of the wasmtime.

Java program cannot interact the wasmtime directly, and there's an additional layer(wasmtime-java) required. I'm just saying about condition control(removing side-effect as possible) and fairness for measuring performance.

I don't know whether the wasm-compiled policy is faster than REST served policy and I haven't tested the performance at all.