Error "unable to open database file"

[Rohlik]

Hello, first of all, thank you for this great tool 👍🏽 . I'm having issue with any command. Basically I'm receiving this error every time:

2022-06-30 16:23:47,323 [grafana_wtf.core      ] INFO   : Setting up response cache to expire after 300 seconds
Traceback (most recent call last):
  File "/home/tom/.local/bin/grafana-wtf", line 8, in <module>
    sys.exit(run())
  File "/home/tom/.local/lib/python3.9/site-packages/grafana_wtf/commands.py", line 174, in run
    engine.enable_cache(expire_after=cache_ttl, drop_cache=options["drop-cache"])
  File "/home/tom/.local/lib/python3.9/site-packages/grafana_wtf/core.py", line 55, in enable_cache
    requests_cache.install_cache(expire_after=expire_after)
  File "/home/tom/.local/lib/python3.9/site-packages/requests_cache/patcher.py", line 48, in install_cache
    backend = init_backend(cache_name, backend, **kwargs)
  File "/home/tom/.local/lib/python3.9/site-packages/requests_cache/backends/__init__.py", line 97, in init_backend
    return BACKEND_CLASSES[backend](cache_name, **kwargs)
  File "/home/tom/.local/lib/python3.9/site-packages/requests_cache/backends/sqlite.py", line 100, in __init__
    self.responses: SQLiteDict = SQLitePickleDict(db_path, table_name='responses', **kwargs)
  File "/home/tom/.local/lib/python3.9/site-packages/requests_cache/backends/sqlite.py", line 155, in __init__
    self.init_db()
  File "/home/tom/.local/lib/python3.9/site-packages/requests_cache/backends/sqlite.py", line 160, in init_db
    with self._lock, self.connection() as con:
  File "/usr/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/home/tom/.local/lib/python3.9/site-packages/requests_cache/backends/sqlite.py", line 168, in connection
    self._local_context.con = sqlite3.connect(self.db_path, **self.connection_kwargs)
sqlite3.OperationalError: unable to open database file

Output above is from grafana-wtf --grafana-url=https://redacted.org/grafana/?verify=no --grafana-token=redacted info --format=yaml Do you have some clue what can be wrong?

[amotl]

Dear Tomas,

thank you for writing in and for reporting this flaw. grafana-wtf makes heavy use of caching. In this case, it looks like the requests_cache package can't access its database file for writing. Maybe it is a permission problem? Can you share some details about your system environment?

With kind regards, Andreas.

[amotl]

Hi again,

I've just followed the implementation in requests_cache/backends/sqlite.py and wanted to report back about my findings. Because grafana-wtf does not configure the cache backend of the requests_cache package in any way, it will use the current working directory for storing an SQLite database file.

In previous versions, e.g. on my machine, I am seeing a cache.sqlite file there. On newer versions of requests_cache, they may have renamed it to http_cache.sqlite.

So, those are my conclusions and recommendations:

a) I am still curious to hear if there is something special about your environment we should know about, in order to provide more reasonable defaults to the requests_cache configuration, unless we improve it to make it configurable by users of grafana-wtf, see c).

b) In order to resolve the problem on your end in the meanwhile, make sure to change the current working directory to a writeable one before launching the program.

c) When there is demand for it, we may well think about adding a command line option or an environment variable to configure the cache path or further caching parameters through, for example, grafana-wtf --cache-path=/tmp/lalala.

Let me know if you can resolve your problem with option b) and also what you think about the other options.

With kind regards, Andreas.

[Rohlik]

That was fast, thank you for your reply 💙. You are right. I run that command as regular user (without sudo) and I was at / so of course I don't have write access there.

Maybe it would be nice to implement some background logic which will try create empty file and then delete it to check if the user have required rights if --cache-path is omitted. This check could provides user-friendly error message.

[amotl]

Hi Tomas,

good that it works for you now. I agree: Having a better error message here would be sweet.

Also, it would be nice to improve the default behavior, that the cache file is not addressed in the working directory, but within the user's home directory instead.

The requests_cache module has two new options here, use_cache_dir and use_temp ^1. I think it would be good to use one of them as a more reasonable default.

I will reopen this issue to have a corresponding reminder for the next development iteration to work on that.

With kind regards, Andreas.

[amotl]

Dear Tomas,

9bf1f65677 will improve the default configuration of the caching subsystem, by using the use_cache_dir=True option to requests_cache. On my macOS machine, this means:

Response cache database location is /Users/amo/Library/Caches/http_cache.sqlite

I hope that this will also improve the situation for you.

With kind regards, Andreas.

[amotl]

I've just released grafana-wtf 0.14.0, which includes the corresponding improvement. Let me know if that works for you.

[amotl]

Hi again,

I think it will be safe to close this issue. Please do not hesitate to re-open if you are still observing issues on this topic. Thank you very much again for your report.

With kind regards, Andreas.

[Rohlik]

I do not have time to try it again, but thank you so much for your fix.