Error after updating to 0.18

[chripede]

Describe the bug Error during startup

    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/app/spoolman/spoolman/main.py", line 16, in <module>
    from spoolman import env, externaldb
  File "/home/app/spoolman/spoolman/externaldb.py", line 20, in <module>
    cache_storage = hishel.AsyncFileStorage()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/hishel/_async/_storages.py", line 103, in __init__
    self._base_path.mkdir(parents=True)
  File "/usr/local/lib/python3.11/pathlib.py", line 1120, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "/usr/local/lib/python3.11/pathlib.py", line 1116, in mkdir
    os.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '.cache'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/pathlib.py", line 1116, in mkdir
    os.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '.cache/hishel'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/app/spoolman/.venv/bin/uvicorn", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main
    run(
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run
    server.run()
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run
    return asyncio.run(self.serve(sockets=sockets))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve
    config.load()
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load
    self.loaded_app = import_from_string(self.app)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/app/spoolman/spoolman/main.py", line 16, in <module>
    from spoolman import env, externaldb
  File "/home/app/spoolman/spoolman/externaldb.py", line 20, in <module>
    cache_storage = hishel.AsyncFileStorage()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/app/spoolman/.venv/lib/python3.11/site-packages/hishel/_async/_storages.py", line 103, in __init__
    self._base_path.mkdir(parents=True)
  File "/usr/local/lib/python3.11/pathlib.py", line 1120, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "/usr/local/lib/python3.11/pathlib.py", line 1116, in mkdir
    os.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '.cache'
usermod: no changes
User UID: 99
User GID: 100
Starting uvicorn...

Spoolman Host (please complete the following information):

• Installation type: Docker
• Installed version: 0.18 (I believe. Newest from docker hub)
• Database type: postgres

[Donkie]

Interesting. I can't really reproduce it, but I have pushed a fix for this. Please try the edge tagged image once it has finished building in ~15 minutes.

That is, if you use the docker-compose from the README, replace image: ghcr.io/donkie/spoolman:latest with image: ghcr.io/donkie/spoolman:edge

[chripede]

Will test new image

I am running it on unraid. Running with docker and the same params on my desktop I cannot reproduce it either.

[chripede]

FYI it seems the action failed to publish a new image, so I will not be testing until tomorrow :)

[Donkie]

Now it succeeded :)

[chripede]

This is the warning that edge outputs

Failed to setup disk-based cache due to permission error. Ensure the path /home/app/spoolman/.cache/hishel is writable. Using in-memory cache instead as fallback.

[chripede]

Also this probably belongs in its own bug report, but it's late.

Importing filaments from the external database is a mess of

DETAIL:  Key (id)=(3) already exists.
[SQL: INSERT INTO vendor (registered, name, empty_spool_weight, comment, external_id) VALUES ($1::TIMESTAMP WITHOUT TIME ZONE, $2::VARCHAR, $3::FLOAT, $4::VARCHAR, $5::VARCHAR) RETURNING vendor.id]
[parameters: (datetime.datetime(2024, 5, 20, 22, 54, 3), 'JAYO', None, None, 'JAYO')]

from both vendor keys and filament keys

[drpetersen]

I ran into this problem when I had PUID and PGID set to something other than 1000 and 1001, respectively, to match existing IDs on my system. This used to work with previous versions.

Restored PUID=1000 and PGID=1001 in my container's environment, and chown'ed the external volumes mounted into the container (chown 1000:1001 -R PATH/TO/SPOOLMAN/DATA), and then the error was gone.

[Donkie]

This is the warning that edge outputs

Failed to setup disk-based cache due to permission error. Ensure the path /home/app/spoolman/.cache/hishel is writable. Using in-memory cache instead as fallback.

Try latest image from edge tag now, the warning should no longer appear.

I have yet to look into the conflicting ID issue. I cant seem to reproduce it. Could you try some more and see if you can make it import for some cases but not others, etc?

[m-klecka]

You're probably using my unraid template for spoolman. You can mount the cache path as a workaround. See screenshot. Don't know why spoolman always has some problems with permissions.. EDIT: the permission issue is probably with unraid using PUID and PGID 99:100 (and container is set up with variables to use this) but the .cache dir is probably created with something else (probably 1000:100?)

[srvoets]

Just wanted to say I woke up this AM to this issue, switching to the edge release fixed it immediately.

[Donkie]

You're probably using my unraid template for spoolman. You can mount the cache path as a workaround. See screenshot.

Don't know why spoolman always has some problems with permissions..

EDIT: the permission issue is probably with unraid using PUID and PGID 99:100 (and container is set up with variables to use this) but the .cache dir is probably created with something else (probably 1000:100?)

I changed the cache dir in the latest commit so now its located inside the spoolman data dir like everything else, there shouldnt be any permission issues anymore.

[Donkie]

I will push an update when the other issue he mentioned is addressed

[challgren]

Having the same exact issue with latest. Trying edge now. Edge fixed the issue

docker-compose.yml is

services:
  spoolman:
    hostname: spoolman
    container_name: spoolman
    image: ghcr.io/donkie/spoolman:edge
    restart: unless-stopped
    volumes:
      - /home/spoolman/data:/home/app/.local/share/spoolman
      - /etc/letsencrypt/live/hallgren.net:/home/spoolman/app/certs/live/hallgren.net:ro
      - /etc/letsencrypt/archive/hallgren.net:/home/spoolman/app/certs/archive/hallgren.net:ro
    ports:
      - 7912:8000
    command: --host 0.0.0.0 --port 8000 --ssl-keyfile /home/spoolman/app/certs/live/hallgren.net/privkey.pem --ssl-certfile /home/spoolman/app/certs/live/hallgren.net/fullchain.pem
    environment:
      - TZ=America/Chicago # Optional, defaults to UTC
      - PUID=1001
      - PGID=1001

networks:
  default:
    external:
      name: br0

[Donkie]

does anyone else have the issue that @chripede had above with "DETAIL: Key (id)=(3) already exists." errors?

[challgren]

does anyone else have the issue that @chripede had above with "DETAIL: Key (id)=(3) already exists." errors?

My issue was mainly the cache no issues with duplicate keys but sounds like he's importing data so probably a separate issue.

[chripede]

The original problem has been solved for me now, thanks!

As for the unique key problem, I have created a new bug report with more info #384

[Donkie]

I have now pushed 0.18.1 that fixes this issue in a stable release. Everyone who switched to edge tag, please switch back to latest, edge should never be used in a production environment since it can sometimes be broken.