search

YttriLab / A-SOID

An active learning platform for expert-guided, data efficient discovery of behavior.
Other
55 stars 7 forks source link

feature extraction #13

Closed colinyong closed 1 year ago

colinyong commented 1 year ago

Hello, when I use a-soid to enter the second stage--(feature extraction), streamlit always shows that it is connecting and can't move to the next stage. Is there some way to solve this problem? Thank you.

JensBlack commented 1 year ago

Hi Colin,

Not sure what is the issue right there. Could you add a more detailed report to this including the version of asoid that you are using? Thanks!

sean-afshar commented 1 year ago

Hi, I am experiencing similar issues when I try to extract features. The streamlit page freezes and the progress bar of the feature extraction stage is stuck at 0%. When I check the error trace in the terminal I get the following

(asoid-test) λ asoid app

  You can now view your Streamlit app in your browser.

  Local URL: http://localhost:8501
  Network URL: http://10.1.37.148:8501

New project Feb-18-2023 created.
Config updated.
New project Feb-18-2023 created.
Config updated.
Config updated.
Config updated.
Config updated.
Config updated.
C:\Miniconda3\envs\asoid-test\lib\site-packages\numba\core\typed_passes.py:329: NumbaPerformanceWarning:

The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see https://numba.readthedocs.io/en/stable/user/parallel.html#diagnostics for help.

File "d:\a-soid\a-soid\asoid\utils\extract_features.py", line 154:
@jit(nopython=True, parallel=True)
def fast_feature_extraction(data, framerate, index, smooth):
^

C:\Miniconda3\envs\asoid-test\lib\site-packages\numba\core\ir_utils.py:2147: NumbaPendingDeprecationWarning:

Encountered the use of a type that is scheduled for deprecation: type 'reflected list' found for argument 'data' of function 'fast_feature_extraction'.

For more information visit https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-reflection-for-list-and-set-types

File "d:\a-soid\a-soid\asoid\utils\extract_features.py", line 154:
@jit(nopython=True, parallel=True)
def fast_feature_extraction(data, framerate, index, smooth):
^
RafaelParker34 commented 1 year ago

Hi, I am now experiencing the same issue with the same console output as Sean.

I have been running ASOID 0.2.0 for a while and it has been working fine for the initial feature extraction. I updated it to 0.2.2 and began having this feature extraction issue. I have tried reverting to 0.2.1 and 0.2.0 but neither has been successful.

In addition to the console output that has already been posted, the anaconda console loses connection with the app entirely and reverts to a prompt. In Streamlit it changes from running to connecting in the top right corner and the browser devtools tells me that "The connection to ws://localhost:8501/stream was interrupted while the page was loading."

JensBlack commented 1 year ago

Thank you all for reporting, we are looking into it right now. My first guess is that this might be an issue with streamlit/python timing out when a computation runs to long. Can you give me some info on the amount of data that you are trying to extract features from?

sean-afshar commented 1 year ago

Thanks for writing back @JensBlack ! I was running it on a ~430,000 kb video of around 18,000 frames. I had labeled around 100 frames in a boris file of 9 kb. Let me know if you need anything else.

JensBlack commented 1 year ago

@sean-afshar Thanks for sending me test data. I was able to recreate your error, which I believe is different from the others. It would be great if you could post the solution and the issue to a seperate issue.

@colinyong @RafaelParker34 This is most likely a streamlit issue. Can you give me additional info about the amount of data that you are trying to use?

sean-afshar commented 1 year ago

Hi @JensBlack I fixed my BORIS file exporting error but still run into this issue.

RafaelParker34 commented 1 year ago

I was able to get it working again by rebuilding my anaconda environment from the original .yaml file rather than the newer .yml. Maybe then it was an update in a dependency causing it to hang at feature extraction?

JensBlack commented 1 year ago

Interesting that you say that. We are aware of the issue and are looking into it. With the new version on the brink of release, I am trying to find a solution ASAP. However, I can not replicate the issue so far. @RafaelParker34 It would be great if you can ellaborate on your fix and I'll recover the old yaml if that is the problem.