search

eto-ai / rikai

Parquet-based ML data format optimized for working with unstructured data
https://rikai.readthedocs.io/en/latest/
Apache License 2.0
136 stars 19 forks source link

Assembly dependencies cleaning #588

Closed da-liii closed 2 years ago

da-liii commented 2 years ago

Remove cats and shapeless, we will reduce half of the assembly jar size

See https://github.com/eto-ai/rikai/pull/419

Remove Scala Standard Library

image

Remove MLflow later (contribute the fix upstream) @Renkai

Remove antlr4 (use the Spark built-in one)

image

Remove enableIf (compile time deps)

Renkai commented 2 years ago

Interesting, why cats and shapeless are in rikai now?

da-liii commented 2 years ago

I think reducing the dependencies is an effective way to protect our natural environment as a human being on earth.

Renkai commented 2 years ago

@darcy-shen What tools have you used to get this pic? I want to try it locally. image

da-liii commented 2 years ago

ncdu