lnbits / withdraw

LNbits Withdraw Extension
MIT License
6 stars 9 forks source link

Add paginated fetching to withdraw link table #40

Closed LoadJulz closed 1 month ago

LoadJulz commented 2 months ago

We had a severe bug in the Withdraw Extension that prevented us from using the UI. In our app, we handle many different users, and each user will eventually receive an LNURLw link. With thousands of links in the database, it was not very efficient to load all the links simultaneously. Many times we received a "502 Bad Gateway" error because the response took too long. This caused a significant chain reaction because our LNBits instance was completely down afterward, requiring us to reset with a backup to get things running again.

By implementing a paginated loading of the table, this issue would not occur anymore, and the Withdraw plugin would load much faster initially as well.

dni commented 1 month ago

hey, can you rebase and solve conflicts, there is now linting, ci and tests in the repo you can pull latest main and run

poetry install
npm install

and then run

make

for formatting and


poetry run pytest
``` for tests
LoadJulz commented 1 month ago

Issue is poetry is not working in my environment. I have already noted this in another issue: https://github.com/lnbits/lnbits/issues/2568

For the current project I get the following error:

Installing dependencies from lock file

Package operations: 69 installs, 0 updates, 0 removals

  - Installing coincurve (18.0.0): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel

  Traceback (most recent call last):
    File "/usr/local/Cellar/poetry/1.8.3_2/libexec/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
      main()
    File "/usr/local/Cellar/poetry/1.8.3_2/libexec/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
      json_out["return_val"] = hook(**hook_input["kwargs"])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/Cellar/poetry/1.8.3_2/libexec/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 271, in build_wheel
      return _build_backend().build_wheel(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 415, in build_wheel
      return self._build_with_temp_dir(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
      self.run_setup()
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 497, in run_setup
      super().run_setup(setup_script=setup_script)
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
      exec(code, locals())
    File "<string>", line 266, in <module>
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 171, in setup
      ok = dist.parse_command_line()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 478, in parse_command_line
      args = self._parse_command_opts(parser, args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 876, in _parse_command_opts
      nargs = _Distribution._parse_command_opts(self, parser, args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/lz/t1dpdjjj5718xrcnhm_nnyv00000gn/T/tmp_hlk3_gx/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 543, in _parse_command_opts
      if not issubclass(cmd_class, Command):
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  TypeError: issubclass() arg 1 must be a class

  at /usr/local/Cellar/poetry/1.8.3_2/libexec/lib/python3.12/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with coincurve (18.0.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "coincurve (==18.0.0)"'.

Any other idea how I could execute the linter and check if everything is formatted?

LoadJulz commented 1 month ago

Have moved this to a new PR (https://github.com/lnbits/withdraw/pull/41) in order to get the lint checks in the PR. Still cannot build with poetry but with the lints from the PR I was able to fix the issue.