search

cpbotha / nvpy

Simplenote syncing note-taking application, inspired by Notational Velocity and ResophNotes, but uglier and cross-platformerer.
Other
849 stars 114 forks source link

Crash while house keeping #195

Closed yuuki0xff closed 3 years ago

yuuki0xff commented 4 years ago 
2019-11-07 02:29:28,361 - CRITICAL -
ERROR: An unexpected error occurred.
Thread(ident=140562932799296, name=MainThread)
Traceback (most recent call last):
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/debug.py", line 31, in wrapper
    return fn(*args, **kwargs)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/tk.py", line 30, in wrapper
    return fn(*args, **kwargs)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/nvpy.py", line 645, in observer_view_keep_house
    nsaved = self.notes_db.save_threaded()
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/notes_db.py", line 631, in save_threaded
    for k, n in self.notes.items():
RuntimeError: dictionary changed size during iteration

Other threads:
Thread(ident=140562932799296, name=MainThread)
  File "/opt/nvpy/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/nvpy/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/__main__.py", line 2, in <module>
    nvpy.main()
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/nvpy.py", line 881, in main
    controller.main_loop()
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/nvpy.py", line 429, in main_loop
    self.view.main_loop()
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/view.py", line 1894, in main_loop
    self.root.mainloop()
  File "/opt/nvpy/lib/python3.7/tkinter/__init__.py", line 1283, in mainloop
    self.tk.mainloop(n)
  File "/opt/nvpy/lib/python3.7/tkinter/__init__.py", line 1705, in __call__
    return self.func(*args)
  File "/opt/nvpy/lib/python3.7/tkinter/__init__.py", line 749, in callit
    func(*args)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/view.py", line 2096, in fn
    callback()
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/view.py", line 1597, in handler_housekeeper
    self.notify_observers('keep:house', None)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/utils.py", line 205, in notify_observers
    self.__invoke_observer(o, evt_type, evt)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/utils.py", line 229, in __invoke_observer
    observer(self, event_type, event)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/debug.py", line 45, in wrapper
    all_threads=format_all_tracebacks(),
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/debug.py", line 17, in format_all_tracebacks
    stack = traceback.format_stack(frame)
Thread(ident=140562880931584, name=Thread-1)
  File "/opt/nvpy/lib/python3.7/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/opt/nvpy/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/opt/nvpy/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/debug.py", line 31, in wrapper
    return fn(*args, **kwargs)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/notes_db.py", line 997, in worker_save
    o = self.q_save.get()
  File "/opt/nvpy/lib/python3.7/queue.py", line 170, in get
    self.not_empty.wait()
  File "/opt/nvpy/lib/python3.7/threading.py", line 296, in wait
    waiter.acquire()
Thread(ident=140562803980032, name=Thread-2)
  File "/opt/nvpy/lib/python3.7/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/opt/nvpy/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/opt/nvpy/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/debug.py", line 31, in wrapper
    return fn(*args, **kwargs)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/notes_db.py", line 1024, in worker_sync
    task = self.q_sync.get()
  File "/opt/nvpy/lib/python3.7/queue.py", line 170, in get
    self.not_empty.wait()
  File "/opt/nvpy/lib/python3.7/threading.py", line 296, in wait
    waiter.acquire()
Thread(ident=140562783102720, name=Thread-5)
  File "/opt/nvpy/lib/python3.7/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/opt/nvpy/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/opt/nvpy/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/notes_db.py", line 892, in sync_full_unthreaded
    n, err = self.simplenote.get_note(k)
  File "/home/yuuki/repo/github.com/cpbotha/nvpy/nvpy/notes_db.py", line 43, in get_note
    return super().get_note(*args, **kwargs)
  File "/home/yuuki/repo/github.com/mrtazz/simplenote.py/simplenote/simplenote.py", line 135, in get_note
    response = urllib2.urlopen(request)
  File "/opt/nvpy/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/nvpy/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/opt/nvpy/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/opt/nvpy/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/opt/nvpy/lib/python3.7/urllib/request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/opt/nvpy/lib/python3.7/urllib/request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/opt/nvpy/lib/python3.7/http/client.py", line 1244, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/nvpy/lib/python3.7/http/client.py", line 1290, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/nvpy/lib/python3.7/http/client.py", line 1239, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/nvpy/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/opt/nvpy/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/opt/nvpy/lib/python3.7/http/client.py", line 1406, in connect
    super().connect()
  File "/opt/nvpy/lib/python3.7/http/client.py", line 938, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/opt/nvpy/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
rakslice commented 3 years ago

I'm running into this while doing the first sync in a long time with a sync of a lot of notes in progress... presumably you need to add some kind of locks to prevent the sync in progress from adding a note while save_threaded is doing its pass through the notes

yuuki0xff commented 3 years ago

I added mitigations for this issue in 844e8e5ad67cbae91603e95703b5fc34e78675ab. Reopen it if this issue recurs.