search

web3infra-foundation / mega

Mega is an unofficial open source implementation of Google Piper.
https://gitmega.dev
Apache License 2.0
170 stars 31 forks source link

Improve git-pack encode performance. #600

Open Hou-Xiaoxuan opened 2 months ago

Hou-Xiaoxuan commented 2 months ago

The current performance of Git pack encoding is poor and takes too long, causing operations like cloning to be excessively slow, as seen in issue #571 . The main time-consuming factor is the zlib compression process.

Possible solutions:

  1. Try alternative compression algorithms/libraries.
  2. The current encoding is sequential, consider changing it to parallel processing.
genedna commented 1 month ago

@Hou-Xiaoxuan ,

Maby try miniz_oxide, a pure rust replacement for the miniz deflate/zlib encoder/decoder using no unsafe code. It builds in no_std mode, though it requires alloc and collections crates.