CI ui-tests are not actually passing

[trevorcampbell]

The current commit af47e5e looks like it is passing all ui-tests, but there are quite a few tracebacks / errors in the ui-tests logs that aren't being caught as failures. I'm not sure if these are actual problems that weren't noticed because they're failing silently, or they are known and being ignored on purpose, but I figured I'd point it out here in case!

For example, see the log for the CI run on af47e5e :

https://github.com/jupyter/nbgrader/actions/runs/5925665890/job/16065530603

Under the "Run Playwright Tests" section, you will see quite a few errors. The first is related to the exchange being unwriteable:

[WebServer] [E 2023-08-21 11:34:21.644 ServerApp] Traceback (most recent call last):
      File "/home/runner/work/nbgrader/nbgrader/nbgrader/server_extensions/assignment_list/handlers.py", line 85, in list_released_assignments
        assignments = lister.start()
      File "/home/runner/work/nbgrader/nbgrader/nbgrader/exchange/abc/list.py", line 23, in start
        super(ExchangeList, self).start()
      File "/home/runner/work/nbgrader/nbgrader/nbgrader/exchange/default/exchange.py", line 133, in start
        self.ensure_root()
      File "/home/runner/work/nbgrader/nbgrader/nbgrader/exchange/default/exchange.py", line 59, in ensure_root
        self.fail("Unwritable directory, please contact your instructor: {}".format(self.root))
      File "/home/runner/work/nbgrader/nbgrader/nbgrader/exchange/abc/exchange.py", line 58, in fail
        raise ExchangeError(msg)
    nbgrader.exchange.abc.exchange.ExchangeError: Unwritable directory, please contact your instructor: /usr/local/share/nbgrader/exchange

And then there follows quite a few 404s / file not found errors:

[WebServer] [W 2023-08-21 11:34:22.071 ServerApp] wrote error: 'No such file or directory: nbgrader_config.py'
    Traceback (most recent call last):
      File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/tornado/web.py", line 1786, in _execute
        result = await result
      File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 266, in put
        await self._save(model, path)
      File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 191, in _save
        model = await ensure_async(self.contents_manager.save(model, path))
      File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 182, in ensure_async
        result = await obj
      File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/jupyter_server/services/contents/largefilemanager.py", line 133, in save
        return await super().save(model, path)
      File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 934, in save
        model = await self.get(path, content=False)
      File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 861, in get
        raise web.HTTPError(404, "No such file or directory: %s" % path)
    tornado.web.HTTPError: HTTP 404: Not Found (No such file or directory: nbgrader_config.py)

[brichet]

These errors have always been there, as you mention in https://github.com/jupyter/nbgrader/discussions/1749#discussioncomment-6794240. I think this is related to the fact that some files don't exist at startup, or are deleted from disk while open in the browser.

We can leave this issue open, it would be great to clean it up.

[brichet]

@trevorcampbell I opened https://github.com/jupyter/nbgrader/pull/1871 to remove some confusing error messages (at least on Linux).