search

StochSS / stochss-compute

A compute delegation server for the StochSS family of stochastic simulators.
https://stochss.github.io/stochss-compute/
GNU General Public License v3.0
5 stars 0 forks source link

JSONDecodeError on get() in t he StartHere notebook #58

Closed briandrawert closed 2 years ago

briandrawert commented 2 years ago

In the [3] cell:

http://localhost:1234/api/v1/gillespy2/model/run
http://localhost:1234/api/v1/job/bfb440ab3ba796a0f324193fc6f69194-run_1/status
The job is complete.
4
http://localhost:1234/api/v1/memory/bfb440ab3ba796a0f324193fc6f69194-run_1/get
---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
~/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.BaseModel.parse_raw()

~/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/pydantic/parse.cpython-38-x86_64-linux-gnu.so in pydantic.parse.load_str_bytes()

/usr/lib/python3.8/json/__init__.py in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    356             parse_constant is None and object_pairs_hook is None and not kw):
--> 357         return _default_decoder.decode(s)
    358     if cls is None:

/usr/lib/python3.8/json/decoder.py in decode(self, s, _w)
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()

/usr/lib/python3.8/json/decoder.py in raw_decode(self, s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

ValidationError                           Traceback (most recent call last)
/tmp/ipykernel_1885851/2043053169.py in <module>
      5 results = RemoteSimulation.on(myServer).with_model(myModel).run()
      6 
----> 7 results = results.resolve()
      8 
      9 results.plot()

~/Desktop/research/stochss-compute/stochss_compute/remote_results.py in resolve(self)
    145 
    146         if not results_response.ok:
--> 147             raise Exception(ErrorResponse.parse_raw(results_response.text).msg)
    148 
    149         print(f"Results size: {sys.getsizeof(results_response.content)}")

~/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.BaseModel.parse_raw()

ValidationError: 1 validation error for ErrorResponse
__root__
  Expecting value: line 1 column 1 (char 0) (type=value_error.jsondecode; msg=Expecting value; doc=<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
  "http://www.w3.org/TR/html4/loose.dtd">
<html>
  <head>
    <title>KeyError: &quot;Dataset &#x27;bfb440ab3ba796a0f324193fc6f69194-run_1&#x27; not found&quot; // Werkzeug Debugger</title>
    <link rel="stylesheet" href="?__debugger__=yes&amp;cmd=resource&amp;f=style.css"
        type="text/css">
    <!-- We need to make sure this has a favicon so that the debugger does
         not accidentally trigger a request to /favicon.ico which might
         change the application's state. -->
    <link rel="shortcut icon"
        href="?__debugger__=yes&amp;cmd=resource&amp;f=console.png">
    <script src="?__debugger__=yes&amp;cmd=resource&amp;f=debugger.js"></script>
    <script type="text/javascript">
      var TRACEBACK = 139980924182192,
          CONSOLE_MODE = false,
          EVALEX = true,
          EVALEX_TRUSTED = false,
          SECRET = "Mx7UrHLS2wV9RiF5erav";
    </script>
  </head>
  <body style="background-color: #fff">
    <div class="debugger">
<h1>KeyError</h1>
<div class="detail">
  <p class="errormsg">KeyError: &quot;Dataset &#x27;bfb440ab3ba796a0f324193fc6f69194-run_1&#x27; not found&quot;</p>
</div>
<h2 class="traceback">Traceback <em>(most recent call last)</em></h2>
<div class="traceback">
  <h3></h3>
  <ul><li><div class="frame" id="frame-139980924066736">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py"</cite>,
      line <em class="line">2091</em>,
      in <code class="function">__call__</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">    </span>def __call__(self, environ: dict, start_response: t.Callable) -&gt; t.Any:</pre>
<pre class="line before"><span class="ws">        </span>&quot;&quot;&quot;The WSGI server calls the Flask application object as the</pre>
<pre class="line before"><span class="ws">        </span>WSGI application. This calls :meth:`wsgi_app`, which can be</pre>
<pre class="line before"><span class="ws">        </span>wrapped to apply middleware.</pre>
<pre class="line before"><span class="ws">        </span>&quot;&quot;&quot;</pre>
<pre class="line current"><span class="ws">        </span>return self.wsgi_app(environ, start_response)</pre></div>
</div>

<li><div class="frame" id="frame-139980924065248">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py"</cite>,
      line <em class="line">2076</em>,
      in <code class="function">wsgi_app</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">            </span>try:</pre>
<pre class="line before"><span class="ws">                </span>ctx.push()</pre>
<pre class="line before"><span class="ws">                </span>response = self.full_dispatch_request()</pre>
<pre class="line before"><span class="ws">            </span>except Exception as e:</pre>
<pre class="line before"><span class="ws">                </span>error = e</pre>
<pre class="line current"><span class="ws">                </span>response = self.handle_exception(e)</pre>
<pre class="line after"><span class="ws">            </span>except:  # noqa: B001</pre>
<pre class="line after"><span class="ws">                </span>error = sys.exc_info()[1]</pre>
<pre class="line after"><span class="ws">                </span>raise</pre>
<pre class="line after"><span class="ws">            </span>return response(environ, start_response)</pre>
<pre class="line after"><span class="ws">        </span>finally:</pre></div>
</div>

<li><div class="frame" id="frame-139980924066880">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py"</cite>,
      line <em class="line">2073</em>,
      in <code class="function">wsgi_app</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">        </span>ctx = self.request_context(environ)</pre>
<pre class="line before"><span class="ws">        </span>error: t.Optional[BaseException] = None</pre>
<pre class="line before"><span class="ws">        </span>try:</pre>
<pre class="line before"><span class="ws">            </span>try:</pre>
<pre class="line before"><span class="ws">                </span>ctx.push()</pre>
<pre class="line current"><span class="ws">                </span>response = self.full_dispatch_request()</pre>
<pre class="line after"><span class="ws">            </span>except Exception as e:</pre>
<pre class="line after"><span class="ws">                </span>error = e</pre>
<pre class="line after"><span class="ws">                </span>response = self.handle_exception(e)</pre>
<pre class="line after"><span class="ws">            </span>except:  # noqa: B001</pre>
<pre class="line after"><span class="ws">                </span>error = sys.exc_info()[1]</pre></div>
</div>

<li><div class="frame" id="frame-139980924067168">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py"</cite>,
      line <em class="line">1518</em>,
      in <code class="function">full_dispatch_request</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">            </span>request_started.send(self)</pre>
<pre class="line before"><span class="ws">            </span>rv = self.preprocess_request()</pre>
<pre class="line before"><span class="ws">            </span>if rv is None:</pre>
<pre class="line before"><span class="ws">                </span>rv = self.dispatch_request()</pre>
<pre class="line before"><span class="ws">        </span>except Exception as e:</pre>
<pre class="line current"><span class="ws">            </span>rv = self.handle_user_exception(e)</pre>
<pre class="line after"><span class="ws">        </span>return self.finalize_request(rv)</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">    </span>def finalize_request(</pre>
<pre class="line after"><span class="ws">        </span>self,</pre>
<pre class="line after"><span class="ws">        </span>rv: t.Union[ResponseReturnValue, HTTPException],</pre></div>
</div>

<li><div class="frame" id="frame-139980924066640">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py"</cite>,
      line <em class="line">1516</em>,
      in <code class="function">full_dispatch_request</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">        </span>self.try_trigger_before_first_request_functions()</pre>
<pre class="line before"><span class="ws">        </span>try:</pre>
<pre class="line before"><span class="ws">            </span>request_started.send(self)</pre>
<pre class="line before"><span class="ws">            </span>rv = self.preprocess_request()</pre>
<pre class="line before"><span class="ws">            </span>if rv is None:</pre>
<pre class="line current"><span class="ws">                </span>rv = self.dispatch_request()</pre>
<pre class="line after"><span class="ws">        </span>except Exception as e:</pre>
<pre class="line after"><span class="ws">            </span>rv = self.handle_user_exception(e)</pre>
<pre class="line after"><span class="ws">        </span>return self.finalize_request(rv)</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">    </span>def finalize_request(</pre></div>
</div>

<li><div class="frame" id="frame-139980924067120">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py"</cite>,
      line <em class="line">1502</em>,
      in <code class="function">dispatch_request</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">            </span>getattr(rule, &quot;provide_automatic_options&quot;, False)</pre>
<pre class="line before"><span class="ws">            </span>and req.method == &quot;OPTIONS&quot;</pre>
<pre class="line before"><span class="ws">        </span>):</pre>
<pre class="line before"><span class="ws">            </span>return self.make_default_options_response()</pre>
<pre class="line before"><span class="ws">        </span># otherwise dispatch to the handler for that endpoint</pre>
<pre class="line current"><span class="ws">        </span>return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">    </span>def full_dispatch_request(self) -&gt; Response:</pre>
<pre class="line after"><span class="ws">        </span>&quot;&quot;&quot;Dispatches the request and on top of that performs request</pre>
<pre class="line after"><span class="ws">        </span>pre and postprocessing as well as HTTP exception catching and</pre>
<pre class="line after"><span class="ws">        </span>error handling.</pre></div>
</div>

<li><div class="frame" id="frame-139980924260016">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/stochss_compute/api/v1/memory.py"</cite>,
      line <em class="line">26</em>,
      in <code class="function">get_memory</code></h4>
  <div class="source "><pre class="line before"><span class="ws"></span>@v1_memory.route(&quot;/&lt;string:memory_id&gt;/get&quot;, methods=[&quot;GET&quot;])</pre>
<pre class="line before"><span class="ws"></span>def get_memory(memory_id: str):</pre>
<pre class="line before"><span class="ws">    </span>if not delegate.job_complete(memory_id):</pre>
<pre class="line before"><span class="ws">        </span>return ErrorResponse(msg=&quot;A memory with this ID does not yet exist.&quot;).json(), 404</pre>
<pre class="line before"><span class="ws"></span> </pre>
<pre class="line current"><span class="ws">    </span>memory_json = delegate.job_results(memory_id)</pre>
<pre class="line after"><span class="ws">    </span>compressed_memory = bz2.compress(memory_json.encode())</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">    </span>response = make_response(compressed_memory)</pre>
<pre class="line after"><span class="ws">    </span>response.headers[&quot;Content-Encoding&quot;] = &quot;bzip2&quot;</pre>
<pre class="line after"><span class="ws"></span> </pre></div>
</div>

<li><div class="frame" id="frame-139980924258624">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/stochss_compute/api/delegate/dask_delegate.py"</cite>,
      line <em class="line">186</em>,
      in <code class="function">job_results</code></h4>
  <div class="source "><pre class="line before"><span class="ws"></span> </pre>
<pre class="line before"><span class="ws">        </span>return status</pre>
<pre class="line before"><span class="ws"></span> </pre>
<pre class="line before"><span class="ws">    </span>def job_results(self, job_id: str):</pre>
<pre class="line before"><span class="ws">        </span># The results of this job may exist on the client dataset.</pre>
<pre class="line current"><span class="ws">        </span>result_dataset = self.client.get_dataset(name=job_id).result()</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">        </span>if result_dataset is not None:</pre>
<pre class="line after"><span class="ws">            </span>print(&quot;[DEBUG] Getting results from dataset.&quot;)</pre>
<pre class="line after"><span class="ws">            </span>return result_dataset</pre>
<pre class="line after"><span class="ws"></span> </pre></div>
</div>

<li><div class="frame" id="frame-139980924258048">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/client.py"</cite>,
      line <em class="line">2343</em>,
      in <code class="function">get_dataset</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">        </span>See Also</pre>
<pre class="line before"><span class="ws">        </span>--------</pre>
<pre class="line before"><span class="ws">        </span>Client.publish_dataset</pre>
<pre class="line before"><span class="ws">        </span>Client.list_datasets</pre>
<pre class="line before"><span class="ws">        </span>&quot;&quot;&quot;</pre>
<pre class="line current"><span class="ws">        </span>return self.sync(self._get_dataset, name, default=default, **kwargs)</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">    </span>async def _run_on_scheduler(self, function, *args, wait=True, **kwargs):</pre>
<pre class="line after"><span class="ws">        </span>response = await self.scheduler.run_function(</pre>
<pre class="line after"><span class="ws">            </span>function=dumps(function, protocol=4),</pre>
<pre class="line after"><span class="ws">            </span>args=dumps(args, protocol=4),</pre></div>
</div>

<li><div class="frame" id="frame-139980924258240">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py"</cite>,
      line <em class="line">310</em>,
      in <code class="function">sync</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">            </span>future = func(*args, **kwargs)</pre>
<pre class="line before"><span class="ws">            </span>if callback_timeout is not None:</pre>
<pre class="line before"><span class="ws">                </span>future = asyncio.wait_for(future, callback_timeout)</pre>
<pre class="line before"><span class="ws">            </span>return future</pre>
<pre class="line before"><span class="ws">        </span>else:</pre>
<pre class="line current"><span class="ws">            </span>return sync(</pre>
<pre class="line after"><span class="ws">                </span>self.loop, func, *args, callback_timeout=callback_timeout, **kwargs</pre>
<pre class="line after"><span class="ws">            </span>)</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws"></span>def in_async_call(loop, default=False):</pre></div>
</div>

<li><div class="frame" id="frame-139980924258288">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py"</cite>,
      line <em class="line">364</em>,
      in <code class="function">sync</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">    </span>else:</pre>
<pre class="line before"><span class="ws">        </span>while not e.is_set():</pre>
<pre class="line before"><span class="ws">            </span>e.wait(10)</pre>
<pre class="line before"><span class="ws">    </span>if error[0]:</pre>
<pre class="line before"><span class="ws">        </span>typ, exc, tb = error[0]</pre>
<pre class="line current"><span class="ws">        </span>raise exc.with_traceback(tb)</pre>
<pre class="line after"><span class="ws">    </span>else:</pre>
<pre class="line after"><span class="ws">        </span>return result[0]</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws"></span>class LoopRunner:</pre></div>
</div>

<li><div class="frame" id="frame-139980924259968">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py"</cite>,
      line <em class="line">349</em>,
      in <code class="function">f</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">                </span>raise RuntimeError(&quot;sync() called from thread of running loop&quot;)</pre>
<pre class="line before"><span class="ws">            </span>yield gen.moment</pre>
<pre class="line before"><span class="ws">            </span>future = func(*args, **kwargs)</pre>
<pre class="line before"><span class="ws">            </span>if callback_timeout is not None:</pre>
<pre class="line before"><span class="ws">                </span>future = asyncio.wait_for(future, callback_timeout)</pre>
<pre class="line current"><span class="ws">            </span>result[0] = yield future</pre>
<pre class="line after"><span class="ws">        </span>except Exception:</pre>
<pre class="line after"><span class="ws">            </span>error[0] = sys.exc_info()</pre>
<pre class="line after"><span class="ws">        </span>finally:</pre>
<pre class="line after"><span class="ws">            </span>e.set()</pre>
<pre class="line after"><span class="ws"></span> </pre></div>
</div>

<li><div class="frame" id="frame-139980924258192">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/tornado/gen.py"</cite>,
      line <em class="line">762</em>,
      in <code class="function">run</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">                </span>self.future = None</pre>
<pre class="line before"><span class="ws">                </span>try:</pre>
<pre class="line before"><span class="ws">                    </span>exc_info = None</pre>
<pre class="line before"><span class="ws"></span> </pre>
<pre class="line before"><span class="ws">                    </span>try:</pre>
<pre class="line current"><span class="ws">                        </span>value = future.result()</pre>
<pre class="line after"><span class="ws">                    </span>except Exception:</pre>
<pre class="line after"><span class="ws">                        </span>exc_info = sys.exc_info()</pre>
<pre class="line after"><span class="ws">                    </span>future = None</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">                    </span>if exc_info is not None:</pre></div>
</div>

<li><div class="frame" id="frame-139980924257952">
  <h4>File <cite class="filename">"/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/client.py"</cite>,
      line <em class="line">2320</em>,
      in <code class="function">_get_dataset</code></h4>
  <div class="source library"><pre class="line before"><span class="ws">        </span>with self.as_current():</pre>
<pre class="line before"><span class="ws">            </span>out = await self.scheduler.publish_get(name=name, client=self.id)</pre>
<pre class="line before"><span class="ws"></span> </pre>
<pre class="line before"><span class="ws">        </span>if out is None:</pre>
<pre class="line before"><span class="ws">            </span>if default is NO_DEFAULT_PLACEHOLDER:</pre>
<pre class="line current"><span class="ws">                </span>raise KeyError(f&quot;Dataset &#x27;{name}&#x27; not found&quot;)</pre>
<pre class="line after"><span class="ws">            </span>else:</pre>
<pre class="line after"><span class="ws">                </span>return default</pre>
<pre class="line after"><span class="ws">        </span>return out[&quot;data&quot;]</pre>
<pre class="line after"><span class="ws"></span> </pre>
<pre class="line after"><span class="ws">    </span>def get_dataset(self, name, default=NO_DEFAULT_PLACEHOLDER, **kwargs):</pre></div>
</div>
</ul>
  <blockquote>KeyError: &quot;Dataset &#x27;bfb440ab3ba796a0f324193fc6f69194-run_1&#x27; not found&quot;</blockquote>
</div>

<div class="plain">
    <p>
      This is the Copy/Paste friendly version of the traceback.
    </p>
    <textarea cols="50" rows="10" name="code" readonly>Traceback (most recent call last):
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py&quot;, line 2091, in __call__
    return self.wsgi_app(environ, start_response)
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py&quot;, line 2076, in wsgi_app
    response = self.handle_exception(e)
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py&quot;, line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py&quot;, line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py&quot;, line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py&quot;, line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File &quot;/home/brian/Desktop/research/stochss-compute/stochss_compute/api/v1/memory.py&quot;, line 26, in get_memory
    memory_json = delegate.job_results(memory_id)
  File &quot;/home/brian/Desktop/research/stochss-compute/stochss_compute/api/delegate/dask_delegate.py&quot;, line 186, in job_results
    result_dataset = self.client.get_dataset(name=job_id).result()
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/client.py&quot;, line 2343, in get_dataset
    return self.sync(self._get_dataset, name, default=default, **kwargs)
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py&quot;, line 310, in sync
    return sync(
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py&quot;, line 364, in sync
    raise exc.with_traceback(tb)
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py&quot;, line 349, in f
    result[0] = yield future
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/tornado/gen.py&quot;, line 762, in run
    value = future.result()
  File &quot;/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/client.py&quot;, line 2320, in _get_dataset
    raise KeyError(f&quot;Dataset &#x27;{name}&#x27; not found&quot;)
KeyError: &quot;Dataset &#x27;bfb440ab3ba796a0f324193fc6f69194-run_1&#x27; not found&quot;</textarea>
</div>
<div class="explanation">
  The debugger caught an exception in your WSGI application.  You can now
  look at the traceback which led to the error.  <span class="nojavascript">
  If you enable JavaScript you can also use additional features such as code
  execution (if the evalex feature is enabled), automatic pasting of the
  exceptions and much more.</span>
</div>
      <div class="footer">
        Brought to you by <strong class="arthur">DON'T PANIC</strong>, your
        friendly Werkzeug powered traceback interpreter.
      </div>
    </div>

    <div class="pin-prompt">
      <div class="inner">
        <h3>Console Locked</h3>
        <p>
          The console is locked and needs to be unlocked by entering the PIN.
          You can find the PIN printed out on the standard output of your
          shell that runs the server.
        <form>
          <p>PIN:
            <input type=text name=pin size=14>
            <input type=submit name=btn value="Confirm Pin">
        </form>
      </div>
    </div>
  </body>
</html>

<!--

Traceback (most recent call last):
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py", line 2091, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py", line 2076, in wsgi_app
    response = self.handle_exception(e)
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/brian/Desktop/research/stochss-compute/stochss_compute/api/v1/memory.py", line 26, in get_memory
    memory_json = delegate.job_results(memory_id)
  File "/home/brian/Desktop/research/stochss-compute/stochss_compute/api/delegate/dask_delegate.py", line 186, in job_results
    result_dataset = self.client.get_dataset(name=job_id).result()
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/client.py", line 2343, in get_dataset
    return self.sync(self._get_dataset, name, default=default, **kwargs)
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py", line 310, in sync
    return sync(
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py", line 364, in sync
    raise exc.with_traceback(tb)
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/utils.py", line 349, in f
    result[0] = yield future
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/tornado/gen.py", line 762, in run
    value = future.result()
  File "/home/brian/Desktop/research/stochss-compute/venv/lib/python3.8/site-packages/distributed/client.py", line 2320, in _get_dataset
    raise KeyError(f"Dataset '{name}' not found")
KeyError: "Dataset 'bfb440ab3ba796a0f324193fc6f69194-run_1' not found"

-->
; pos=0; lineno=1; colno=1)
mdip226 commented 2 years ago

I'm not 100% sure what's causing this error, but I got the same thing when I started a new instance without clearing the cache first (./sd-cache).

ethangreen-dev commented 2 years ago

This issue occurs when results are queried which exist in the cache but not as a dataset within the memory of the cluster. See issue #62 for more details.