nteract / bookstore

📚 Notebook storage and publishing workflows for the masses
https://bookstore.readthedocs.io
BSD 3-Clause "New" or "Revised" License
202 stars 23 forks source link

Add tornado requirement to address event loop issues #28

Closed mpacer closed 5 years ago

mpacer commented 5 years ago

I was attempting to run the integration tests on master locally(more on that in a separate PR) and was running into issues with how Futures were being processed by the event loop.

[I 17:45:05.385 NotebookApp] Processing storage write of ci-local-writeout2.ipynb [E 17:45:05.387 NotebookApp] Error while archiving file: ci-local-writeout2.ipynb object Future can't be used in 'await' expression Traceback (most recent call last): File "/Users/mpacer/jupyter/bookstore/bookstore/archive.py", line 70, in archive await self.write_to_s3(full_path, record.content) TypeError: object Future can't be used in 'await' expression

After investigating https://stackoverflow.com/questions/34376814/await-future-from-executor-future-cant-be-used-in-await-expression and looking at https://github.com/nteract/bookstore/blob/15162ab5f855ab4de124d931e9bef6195dc41e0f/bookstore/archive.py#L70 https://github.com/nteract/bookstore/blob/15162ab5f855ab4de124d931e9bef6195dc41e0f/bookstore/archive.py#L80-L81 https://github.com/nteract/bookstore/blob/15162ab5f855ab4de124d931e9bef6195dc41e0f/bookstore/archive.py#L44

I figured tornado may have changed how they handled their event loop between the two versions.

Once I updated the integration tests did not run successfully, but they did not run into errors related to await issues.

mpacer commented 5 years ago

I have since discovered the issues that were persisting after updating tornado. It was related to the amount of time minio needed to wait before it received another request to store a document.

https://github.com/nteract/bookstore/pull/30