unable to run witness demo -- ERR: Non zeroed prepad bits = 100000 in b'r'.

[dhh1128]

I've got a fresh clone of the development branch of keripy. I've setup venv, activated it, and done pip install -r requirements.txt. I've also started vLEI-Server without error, and I've run rm -rf /usr/local/var/keri/*; and cd scripts/demo; source demo-scripts.sh. All of this works fine.

Now I'd like to start witnesses. I run: kli witness demo. I immediately get this: ERR: Non zeroed prepad bits = 100000 in b'r'.

I am unable to proceed.

I thought this might be caused by a late-breaking change in the development branch, so I coordinated with Arsh, another Provenant developer who is not having the problem. He was running from commit 85e6ae421c, so I did a git reset --hard to that commit and tried again. This did not change the outcome at all.

I'm happy to debug further, but since I'm such a KERI noob, I'm a little bit unsure which sorts of debug tools I ought to use. For example, is there a way to turn on logging when running the demo scripts? Is there an environment variable that enables "verbose mode" or something?

@pfeairheller

[dhh1128]

The only thing I can think of that might explain my difference is that I'm on a windows machine, running Windows Subsystem for Linux. The shell and environment are essentially Ubuntu 22.04 -- bash, etc. The installed python is 3.10.4 (both system and in virtual env). Normally the WSL solution is excellent and produces behavior that's more or less indistinguishable from running native on Ubuntu. However, there are very subtle differences. One is that the mime type subsystem for web servers has very slight config differences. This causes a default web server launched on my box to report that a .json file is of type text/json instead of application/json, for example. That breaks one unit test in keripy, but I investigated the breakage and decided that it was trivial and unimportant. Perhaps I'll submit a fix. I think it's a red herring on this issue, though.

[dhh1128]

I did a check to make sure the OS (the Ubuntu in the WSL, not the windows container) was fully patched. A few things got patched, but no difference in outcome.

[dhh1128]

Completely reset my local clone and started over from scratch. Created a new orphan branch in git, and merged upstream/development (from WebOfTrust). This should guarantee that my local repo is bit-for-bit identical to what's checked in to the development branch of the WebOfTrust repo.

I got exactly the same error.

[pfeairheller]

The quickest way to debug this is to change a few lines (comment/uncomment) in kli.py to get the full stack trace. (Once upon a time I tried to add a --debug flag to accomplish this but was unable to apply a single flag to all commands).

If you swap the commenting on lines 25, 26 and 27 (comment 25, 26 and uncomment 27) then run the command again and post the stack trace here.

        print(f"ERR: {ex}")
        return -1
        # raise ex

to

        # print(f"ERR: {ex}")
        # return -1
        raise ex

[dhh1128]

Thanks. That makes the info a lot more useful. I am feeling a bit embarrassed that I didn't try that myself. I should have dived in a little deeper before asking such a basic question.

Here's the stack trace.

  File "/home/daniel/code/keripy/src/keri/app/cli/commands/witness/demo.py", line 34, in demo
    wanHby = habbing.Habery(name="wan", salt=Salter(raw=b'wann-the-witness').qb64, temp=False, cf=wancf)
  File "/home/daniel/code/keripy/src/keri/app/habbing.py", line 284, in __init__
    self.db = db if db is not None else basing.Baser(name=self.name,
  File "/home/daniel/code/keripy/src/keri/db/basing.py", line 619, in __init__
    super(Baser, self).__init__(headDirPath=headDirPath, reopen=reopen, **kwa)
  File "/home/daniel/code/keripy/src/keri/db/dbing.py", line 349, in __init__
    super(LMDBer, self).__init__(**kwa)
  File "/home/daniel/code/keripy/venv/lib/python3.10/site-packages/hio/base/filing.py", line 150, in __init__
    self.reopen(clear=clear, reuse=reuse, clean=clean, **kwa)
  File "/home/daniel/code/keripy/src/keri/db/basing.py", line 835, in reopen
    self.reload()
  File "/home/daniel/code/keripy/src/keri/db/basing.py", line 846, in reload
    if (state := self.states.get(keys=data.prefix)) is not None:
  File "/home/daniel/code/keripy/src/keri/db/subing.py", line 1110, in get
    return coring.Serder(raw=bytes(val)) if val is not None else None
  File "/home/daniel/code/keripy/src/keri/core/coring.py", line 4956, in __init__
    super(Serder, self).__init__(raw=raw, ked=ked, kind=kind, sad=sad, code=code)
  File "/home/daniel/code/keripy/src/keri/core/coring.py", line 4707, in __init__
    self.raw = raw  # raw property setter does the deserialization
  File "/home/daniel/code/keripy/src/keri/core/coring.py", line 4801, in raw
    self._saider = Saider(qb64=ked["d"], code=self._code)
  File "/home/daniel/code/keripy/src/keri/core/coring.py", line 3355, in __init__
    super(Saider, self).__init__(raw=raw, code=code, **kwa)
  File "/home/daniel/code/keripy/src/keri/core/coring.py", line 870, in __init__
    self._exfil(qb64)
  File "/home/daniel/code/keripy/src/keri/core/coring.py", line 1157, in _exfil
    raise ValueError(f"Non zeroed prepad bits = "
ValueError: Non zeroed prepad bits = 100000 in b'r'.

[pfeairheller]

Alright, this stack trace indicates that the database the process is using is not empty. KERIpy will use one of three locations for its database:

/usr/local/var/keri if it is available and writable by the current user ~/.keri if the previous is not available /tmp/keri_****** if temp is passed as a flag

The /tmp directory is only used for tests. My guess is that your user can not access /usr/local/var/keri so it is writing to your home directory which you haven't deleted yet (and probably has an old database prior to the neo branch).

The directions for the scripts needs to spell out the directory usage more clearly. I'll open an issue to add documentation to the scripts README to make this clear. Sorry for the confusion.

[dhh1128]

Yep, that fixed it. Thank you!