search

LMBooth / pybci

Create real-time BCI's with the LSL, PyTorch, SKLearn and TensorFlow packages.
https://pybci.readthedocs.io/en/latest/
MIT License
22 stars 4 forks source link

Testing notes #24

Closed jsheunis closed 1 year ago

jsheunis commented 1 year ago

I ran pytest, but turns out this package is not installed.

I think you should add a requirements file (e.g. requirements-devel.txt) to your repo root directory containing any requirements that developers might need for local use, including pytest. In addtion, I suggest adding a note on the readme and in the contributing docs to instruct people what to do in order to run tests locally.

jsheunis commented 1 year ago

Ping: https://github.com/openjournals/joss-reviews/issues/5706

jsheunis commented 1 year ago

After installing pytest and running it locally, pytest collected 10 items and started running. The second one failed (Tests/test_Pytorch.py) and the test batch is still running after 15min.

Do you know how long one should expect the tests to run? Especially if it's a long time (>10min) it's good to notify developers what to expect.

LMBooth commented 1 year ago

The last time i ran any tests on mac or linux operating systems myself was before i integrated the pseudo device (about two months ago) which wraps lsl marker and data creation in threaded or multi-processed operations, since fighting with the appveyor ubuntu process there seems to be an issue there which i haven't been able to solve yet, and am currently trying to flesh out the same process for the mac appveyor build. From doing some reading through other lsl repository issues it may be a problem starting LSL outlets out of the main thread.

For reference when i had tested on the linux and mac previously i had pseudo data generated in an example script meaning the pseudo device python instance was different to the one running pybci. Windows seems to be fine with non main thread lsl data creation and population, but until i can verify all tests work via appveyor this may be an issue. If these builds fail i'll need to rewrite the current method of data generation on mac and linux machines to make sure data generation is handled in the main thread.

Again, thank you for recommending integrating appveyor as this gave insight and a method to verify that this is a current issue.

LMBooth commented 1 year ago

Hi @jsheunis, for an update

I ran pytest, but turns out this package is not installed.

I think you should add a requirements file (e.g. requirements-devel.txt) to your repo root directory containing any requirements that developers might need for local use, including pytest. In addtion, I suggest adding a note on the readme and in the contributing docs to instruct people what to do in order to run tests locally.

Added a requirement-devel.txt and put notes in the root readme.md and updated readthedocs with the same information.

After installing pytest and running it locally, pytest collected 10 items and started running. The second one failed (Tests/test_Pytorch.py) and the test batch is still running after 15min.

Do you know how long one should expect the tests to run? Especially if it's a long time (>10min) it's good to notify developers what to expect.

I've updated the Pseudo Device and ran tests on Mac, Windows and Linux and got everything to work. The Appveyor process verifies Mac and Windows, but still has an issue with finding both pseudo lsl outlets, it works on my home Ubuntu 22.04 with Python 3.10. Feel free to re-open if any issues persist.