pip install plyvel not working on Windows platform

[josephernest]

Are there no binaries for Python 2.7 - Windows? Currently

pip install plyvel 

produces installation error on Python 2.7 - Windows if you don't have the right compiler installed.

Thanks in advance.

[synodriver]

I hava no idea whether snappy is linked against leveldb during the compilation. It just able to compile.

Your windows build is useless if it's not linked to snappy, because it most likely breaks every time there's a leveldb that uses compression (such as the leveldb that browsers create)

Would update the CI later to build against snappy. ~There are so many dependencies to compile against.~

Please do regardless, I am waiting since Dec 21 for a working windows build 😭

Would snappy-python works?

[Avnsx]

Would snappy-python works?

i have no clue 🤯

maybe @lifegpc knows?

[synodriver]

Trying to build with vcpkg. Hope it work.

[synodriver]

Would snappy-python works?

i have no clue 🤯

maybe @lifegpc knows?

Ok, now I have the same error. But I'm sure that snappy is installed.

[synodriver]

What a strange error.

db = plyvel.DB(r"C:\Users\LAB\AppData\Local\Microsoft\Edge\User Data\Default\Local Storage\leveldb")
for k, v in db:
    print(k,v)

Traceback (most recent call last):
  File "F:\conda\lib\code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "plyvel\_plyvel.pyx", line 841, in plyvel._plyvel.Iterator.__next__
  File "plyvel\_plyvel.pyx", line 886, in plyvel._plyvel.Iterator.real_next
  File "plyvel\_plyvel.pyx", line 91, in plyvel._plyvel.raise_for_status
plyvel._plyvel.CorruptionError: b'Corruption: corrupted compressed block contents'

[Avnsx]

What a strange error.

db = plyvel.DB(r"C:\Users\LAB\AppData\Local\Microsoft\Edge\User Data\Default\Local Storage\leveldb")
for k, v in db:
    print(k,v)

Traceback (most recent call last):
  File "F:\conda\lib\code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "plyvel\_plyvel.pyx", line 841, in plyvel._plyvel.Iterator.__next__
  File "plyvel\_plyvel.pyx", line 886, in plyvel._plyvel.Iterator.real_next
  File "plyvel\_plyvel.pyx", line 91, in plyvel._plyvel.raise_for_status
plyvel._plyvel.CorruptionError: b'Corruption: corrupted compressed block contents'

@wbolster please help 🥶

[wbolster]

sorry, can't help with specific windows builds, but the root cause is likely this: https://github.com/wbolster/plyvel/issues/60#issuecomment-566512186

[rkr87]

Has anyone managed to get plyvel working on Windows with Snappy support? If so, how? I feel like I've tried every version of pre-compiled wheel available in the various threads relating to this issue here on Github, always with the same result;

plyvel._plyvel.CorruptionError: b'Corruption: corrupted compressed block contents'

I'm currently using plyvel_wheels-1.3.0-cp310-cp310-win_amd64.whl and it works fine to a point, however as soon as any other app applies any kind of compression to the db, I start getting the above error in plyvel.

[Avnsx]

Has anyone managed to get plyvel working on Windows with Snappy support?

Nope and repo maintainer don't give a fuck about windows, so this is pretty much a helpless case plyvel will never be compatible with windows

[lifegpc]

Has anyone managed to get plyvel working on Windows with Snappy support? If so, how? I feel like I've tried every version of pre-compiled wheel available in the various threads relating to this issue here on Github, always with the same result;

plyvel._plyvel.CorruptionError: b'Corruption: corrupted compressed block contents'

I'm currently using plyvel_wheels-1.3.0-cp310-cp310-win_amd64.whl and it works fine to a point, however as soon as any other app applies any kind of compression to the db, I start getting the above error in plyvel.

You should link leveldb with snappy.

[Avnsx]

You should link leveldb with snappy.

@lifegpc If you are using google chrome and are on windows, you can just open Run (by pressing Windows + R) and paste %LOCALAPPDATA%\Google\Chrome\User Data\Default\Local Storage\leveldb to find a not plyvel decompressable leveldb database. I also had written a entire issue ticket explaining this issue a while ago https://github.com/wbolster/plyvel/issues/137

[lifegpc]

You should link leveldb with snappy.

@lifegpc If you are using google chrome and are on windows, you can just open Run (by pressing Windows + R) and paste %LOCALAPPDATA%\Google\Chrome\User Data\Default\Local Storage\leveldb to find a not plyvel decompressable leveldb database. I also had written a entire issue ticket explaining this issue a while ago #137

set "INCLUDE=%INCLUDE%;Path to snappy directory"
set "LIB=%LIB%;Path to the snappy library"

You should set these variables before build leveldb.

[rkr87]

You should link leveldb with snappy.

@lifegpc If you are using google chrome and are on windows, you can just open Run (by pressing Windows + R) and paste %LOCALAPPDATA%\Google\Chrome\User Data\Default\Local Storage\leveldb to find a not plyvel decompressable leveldb database. I also had written a entire issue ticket explaining this issue a while ago #137

set "INCLUDE=%INCLUDE%;Path to snappy directory"
set "LIB=%LIB%;Path to the snappy library"

You should set these variables before build leveldb.

Are the pre-compiled wheels packaged with plyvel-wheels not linked with snappy? I'd assumed they had to be otherwise what's the point of providing them?

[synodriver]

Update: I successfully linked plyvel with leveldb and snappy (using conda install -c conda-forge leveldb) but when I test it with Edge's data with db = plyvel.DB(r"C:\Users\LAB\AppData\Local\Microsoft\Edge\User Data\Default\Local Storage\leveldb"), just got a segfault (Process finished with exit code -1073741819 (0xC0000005)). So something must be wrong with leveldb or snappy on windows, I think.

[synodriver]

Envs:

(py310) C:\Windows\system32>conda  list|findstr leveldb
leveldb                   1.23                 h699a171_1    conda-forge
(py310) C:\Windows\system32>conda  list|findstr snappy
snappy                    1.1.10               hfb803bf_0    conda-forge
(py310) C:\Windows\system32>python -V
Python 3.10.12

My fork at https://github.com/synodriver/plyvel, maybe someone else can have a try with conda environment to see what's wrong, at least it can link with snappy.

[yangfan069]

不支持中文路径的解析吗？