Having problems with SQLite3 connection

[juanpuerto23]

Hello there! I have been trying to use the module for a personal project, but I haven't been able to get it to work, I think this is probably a misconfiguration from my environment from what I have been able to see online, but I still want to ask here so maybe I can get some help, as I haven't been able to find anything that can solve this. Of course, I have tried to do the obvious things like reinstalling everything back again, but still nothing changes. I am able to import the package all fine, but whenever I try to use it, it crashes showing the following message:

Traceback (most recent call last):
  File "C:\Users\Juan\Desktop\analisis-meteorologico\src\main.py", line 19, in <module>
    reg = owm.city_id_registry()
  File "C:\Users\Juan\AppData\Local\Programs\Python\Python310\lib\site-packages\pyowm\owm.py", line 96, in city_id_registry
    return cityidregistry.CityIDRegistry.get_instance()
  File "C:\Users\Juan\AppData\Local\Programs\Python\Python310\lib\site-packages\pyowm\commons\cityidregistry.py", line 29, in get_instance
    return CityIDRegistry(CITY_ID_DB_PATH)
  File "C:\Users\Juan\AppData\Local\Programs\Python\Python310\lib\site-packages\pyowm\commons\cityidregistry.py", line 21, in __init__
    self.connection = self.__decompress_db_to_memory(sqlite_db_path)
  File "C:\Users\Juan\AppData\Local\Programs\Python\Python310\lib\site-packages\pyowm\commons\cityidregistry.py", line 52, in __decompress_db_to_memory
    src_conn = sqlite3.connect(tmpf_name)
sqlite3.OperationalError: unable to open database file

Thanks!

[csparpa]

Hi @juanpuerto23 can you please tell me more about what OS you're running on and what Python version are you using? Also please paste here the relevan piece of code giving the error. I guess it's when you call reg = owm.city_id_registry() ?

[geofbaum]

@csparpa Just a note I recently updated to Python 3.10, on a Windows 10 machine and got a similar issue. I tested the newest version of pyowm on 3.8 and also got the same issue. For both I was able to solve it by decompressing the db file and removing the portion of the code in cityidregistry.py that dealt with decompressing it from bz2 and just calls the path of the cities.db file. I didn't have a chance to play around with it since then as I was working on something else with that part of the code or I would have raised an issue myself.

[csparpa]

I've tried to reproduce the issue on my Ubuntu 20.04 box either with Python 3.8.10 and 3.10 and it works fine for both. I suspect the problem might be related to Windows setups only ? @geofbaum SQLite DB compression is a must, otherwise the library size more than doubles!

[m0n1ker]

I ran into this same issue while trying to look at #404. This is an issue with how NamedTemporaryFile works on Windows. Quoted from the tempfile docs:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows)

In short, you can't call sqlite3.connect(FILE) inside of the with NamedTemporaryFile as FILE block. The easiest solution I can think of here is to pass delete=False into NamedTemporaryFile, do the connect/backup commands outside of the with, and remove the tempfile when finished. To ensure it won't leave temp files around if something goes wrong I would wrap this in a try/finally and put os.remove in the finally block. I can create a PR for this.

[CarlosS7]

@m0n1ker I would like to try your solution. Where you able to create the PR?

[vincentxavier]

In short, you can't call sqlite3.connect(FILE) inside of the with NamedTemporaryFile as FILE block. The easiest solution I can think of here is to pass delete=False into NamedTemporaryFile, do the connect/backup commands outside of the with, and remove the tempfile when finished. To ensure it won't leave temp files around if something goes wrong I would wrap this in a try/finally and put os.remove in the finally block. I can create a PR for this.

If you had already create an PR, could you provide the link ?

[bachittle]

since the PR has never been made and the issue still exists, I made my own PR that fixes the issue as mentioned.