search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.08k stars 390 forks source link

[VL] Simplify JniLibLoader #6148

Open zhztheplayer opened 3 weeks ago

zhztheplayer commented 3 weeks ago

Description

Overtime we find that JniLibLoader might be over-designed. Given that when a shared library is unable to be loaded, Gluten usually just refuses to do anything (e.g., just crashes). Thus ideally we load all the libraries needed, or we just raise fatal error.

JniLoader implemented a transactional loading mechanism that supports loading all the specified libraries together and with a rarely used "roll back" API. These stuffs may be the first priority to remove in the topic to simplify code.

ArnavBalyan commented 2 weeks ago

Hi @zhztheplayer I would like to work on this. Can you give some more clarity, we want to get rid of the transactional loading?

zhztheplayer commented 2 weeks ago

we want to get rid of the transactional loading?

I think it doesn't make any actual difference with loading the libs one by one. Probably we can test and prove. The issue is somewhat benign as nevertheless we don't have much code in JniLibLoader.java. Anyway, feel free to take if you want. Thanks.