rotemdan / lzutf8.js

A high-performance Javascript string compression library
MIT License
322 stars 26 forks source link

Browser addon (web extension API) storage #42

Open garywill opened 2 years ago

garywill commented 2 years ago

Hi @rotemdan ! Thank you for this useful library! It works awesome !

You mentioned different browsers and localStorage, but not browser addons made with web extension API. I'm using LZUTF8 in my browser addon storage.sync, which limits 128k bytes.

I tried different output and found {outputEncoding: "StorageBinaryString"} is the best for browser addon storage.sync

Then I face a problem:

   Y1 = LZUTF8.compress(input-data, {outputEncoding: "StorageBinaryString"});

Storing the compressed data Y (string) into Firefox (78) and Chromium (90), the space it occupies in storage.sync different:

(The space used can be seen by storage.sync.getBytesInUse(). Both Firefox and Chromium have limitation of same quata: 128k bytes)

(Also asked here)

Could you find a explaination?

rotemdan commented 2 years ago

It is possible that the byte count reported depends on the serialization method used internally by the browser.

If JSON is used, for example, then in order to encode the string to a UTF-8 string with the kind of character escaping required to be compatible with JSON, then for every binary string character the number of stored bytes may be greater.

I'm not familiar the implementation details on Chrome so I can't know for certain if that's the case.