search

Daveiano / weewx-wdc

Weather Data Center skin for WeeWX
https://www.weewx-hbt.de/
GNU General Public License v3.0
48 stars 14 forks source link

[BUG] statistics.html build failing - database is locked #278

Closed mymaestro closed 3 months ago

mymaestro commented 4 months ago

Describe the bug A clear and concise description of what the bug is. Which version of the skin are you using?

To Reproduce Steps to reproduce the behavior:

  1. Install and run weewx-wdc 3.5.1
  2. Wait for report refresh
  3. Observe the syslog (or journal)

Expected behavior Cheetahgenerator should cleanly produce the statistics page without crashing the reportengine.

Additional context Output:

Jul 10 04:30:16 cyclone weewxd[1066]: INFO weewx.manager: Added record 2024-07-10 04:30:00 CDT (1720603800) to daily summary in 'weewx.sdb'
Jul 10 04:30:21 cyclone weewxd[1066]: INFO weewx.engine: Main loop exiting. Shutting engine down.
Jul 10 04:30:21 cyclone weewxd[1066]: INFO weewx.engine: Shutting down StdReport thread
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: Evaluation of template /etc/weewx/skins/weewx-wdc/statistics.html.tmpl failed with exception '<class 'weedb.OperationalError'>'
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/weewx-wdc/statistics.html.tmpl
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: **** Reason: database is locked
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      return fn(*args, **kwargs)
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****             ^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      return sqlite3.Cursor.execute(self, *args, **kwargs)
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****  sqlite3.OperationalError: database is locked
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****  During handling of the above exception, another exception occurred:
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 334, in generate
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_weewx_wdc_statistics_html_tmpl.py", line 258, in respond
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/lib/python3/dist-packages/Cheetah/Template.py", line 1708, in _handleCheetahInclude
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      self._CHEETAH__cheetahIncludes[_includeID].respond(trans)
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_weewx_wdc_includes_body_alternative_inc.py", line 234, in respond
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/lib/python3/dist-packages/Cheetah/Template.py", line 1708, in _handleCheetahInclude
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      self._CHEETAH__cheetahIncludes[_includeID].respond(trans)
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_weewx_wdc_includes_stat_tile_inc.py", line 912, in respond
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_weewx_wdc_includes_stat_tile_inc.py", line 344, in __errorCatcher40
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "<string>", line 1, in <module>
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/tags.py", line 508, in __getattr__
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      vh = self._do_query()
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****           ^^^^^^^^^^^^^^^^
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/tags.py", line 494, in _do_query
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      result = weewx.xtypes.get_aggregate(self.obs_type, self.timespan,
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/xtypes.py", line 136, in get_aggregate
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      return xtype.get_aggregate(obs_type, timespan, aggregate_type, db_manager,
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/xtypes.py", line 930, in get_aggregate
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      for record in db_manager.genBatchRecords(*timespan):
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/manager.py", line 538, in genBatchRecords
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      for row in self.genBatchRows(startstamp, stopstamp):
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/manager.py", line 515, in genBatchRows
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      gen = cursor.execute("SELECT * FROM %s "
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weedb/sqlite.py", line 52, in guarded_fn
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****      raise weedb.OperationalError(e)
Jul 10 04:30:34 cyclone weewxd[1066]: ERROR weewx.cheetahgenerator: ****  weedb.OperationalError: database is locked
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine: Caught unrecoverable exception in generator 'weewx.cheetahgenerator.CheetahGenerator'
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****  database is locked
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      return fn(*args, **kwargs)
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****             ^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      return sqlite3.Cursor.execute(self, *args, **kwargs)
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****  sqlite3.OperationalError: database is locked
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****  During handling of the above exception, another exception occurred:
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****  Traceback (most recent call last):
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 248, in run
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      obj.start()
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/reportengine.py", line 465, in start
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      self.run()
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 166, in run
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      ngen = self.generate(gen_dict[section_name], section_name, self.gen_ts)
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 226, in generate
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      ngen += self.generate(section[subsection], subsection, gen_ts)
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 226, in generate
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      ngen += self.generate(section[subsection], subsection, gen_ts)
Jul 10 04:30:39 cyclone weewxd[1066]: Traceback (most recent call last):
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn
Jul 10 04:30:39 cyclone weewxd[1066]:     return fn(*args, **kwargs)
Jul 10 04:30:39 cyclone weewxd[1066]:            ^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute
Jul 10 04:30:39 cyclone weewxd[1066]:     return sqlite3.Cursor.execute(self, *args, **kwargs)
Jul 10 04:30:39 cyclone weewxd[1066]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]: sqlite3.OperationalError: database is locked
Jul 10 04:30:39 cyclone weewxd[1066]: During handling of the above exception, another exception occurred:
Jul 10 04:30:39 cyclone weewxd[1066]: Traceback (most recent call last):
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 244, in generate
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/reportengine.py", line 248, in run
Jul 10 04:30:39 cyclone weewxd[1066]:     obj.start()
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/reportengine.py", line 465, in start
Jul 10 04:30:39 cyclone weewxd[1066]:     self.run()
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 166, in run
Jul 10 04:30:39 cyclone weewxd[1066]:     ngen = self.generate(gen_dict[section_name], section_name, self.gen_ts)
Jul 10 04:30:39 cyclone weewxd[1066]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 226, in generate
Jul 10 04:30:39 cyclone weewxd[1066]:     ngen += self.generate(section[subsection], subsection, gen_ts)
Jul 10 04:30:39 cyclone weewxd[1066]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 226, in generate
Jul 10 04:30:39 cyclone weewxd[1066]:     ngen += self.generate(section[subsection], subsection, gen_ts)
Jul 10 04:30:39 cyclone weewxd[1066]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/cheetahgenerator.py", line 244, in generate
Jul 10 04:30:39 cyclone weewxd[1066]:     start_ts = default_archive.firstGoodStamp()
Jul 10 04:30:39 cyclone weewxd[1066]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/manager.py", line 330, in firstGoodStamp
Jul 10 04:30:39 cyclone weewxd[1066]:     _row = self.getSql("SELECT MIN(dateTime) FROM %s" % self.table_name)
Jul 10 04:30:39 cyclone weewxd[1066]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weewx/manager.py", line 599, in getSql
Jul 10 04:30:39 cyclone weewxd[1066]:     _cursor.execute(sql, sqlargs)
Jul 10 04:30:39 cyclone weewxd[1066]:   File "/usr/share/weewx/weedb/sqlite.py", line 52, in guarded_fn
Jul 10 04:30:39 cyclone weewxd[1066]:     raise weedb.OperationalError(e)
Jul 10 04:30:39 cyclone weewxd[1066]: weedb.OperationalError: database is locked
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      start_ts = default_archive.firstGoodStamp()
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/manager.py", line 330, in firstGoodStamp
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      _row = self.getSql("SELECT MIN(dateTime) FROM %s" % self.table_name)
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weewx/manager.py", line 599, in getSql
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      _cursor.execute(sql, sqlargs)
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****    File "/usr/share/weewx/weedb/sqlite.py", line 52, in guarded_fn
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****      raise weedb.OperationalError(e)
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****  weedb.OperationalError: database is locked
Jul 10 04:30:39 cyclone weewxd[1066]: ERROR weewx.reportengine:         ****  Generator terminated
Jul 10 04:30:39 cyclone weewxd[1066]: INFO weewx.reportengine: Copied 19 files to /var/www/html/weewx
Jul 10 04:30:39 cyclone weewxd[1066]: CRITICAL __main__: Database OperationalError exception: database is locked
Jul 10 04:30:39 cyclone weewxd[1066]: CRITICAL __main__:     ****  Waiting 2 minutes then retrying...
Jul 10 04:32:39 cyclone weewxd[1066]: INFO __main__: retrying...
Jul 10 04:32:39 cyclone weewxd[1066]: INFO weewx.engine: Loading station type Vantage (weewx.drivers.vantage)
Jul 10 04:32:39 cyclone weewxd[1066]: INFO weewx.engine: StdConvert target unit is 0x1
Daveiano commented 3 months ago

Hello there and thank you for reaching out and posting relevant log data!

I will close this issue in favour of #267 - let's continue the discussion there, thank you.