search

openworm / open-worm-analysis-toolbox

A testing pipeline that allows us to run a behavioural phenotyping of our virtual worm running the same test statistics the Schafer Lab used on their worm data.
Other
49 stars 27 forks source link

First-time installation issues: can't run video_to_features.py #209

Open mfshao opened 5 years ago

mfshao commented 5 years ago

Hello,

I have installed OWAT on my computer and configured by following the instruction. I have also downloaded the example data from Google Drive. The tools/test_setup.py can run without any issues and I can also get some of the tests to run. However, when I try to run video_to_features.py, it output the following error messages:

C:\Programs\Python\Python37-32\lib\site-packages\scipy\signal_arraytools.py:45: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result. b = a[a_slice] C:\Programs\Python\Python37-32\lib\site-packages\numpy\core\fromnumeric.py:83: RuntimeWarning: invalid value encountered in reduce return ufunc.reduce(obj, axis, dtype, out, *passkwargs) D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py:253: RuntimeWarning: invalid value encountered in ceil max_boundary = np.ceil(max_data / bin_width) bin_width Traceback (most recent call last): File "D:/PycharmProjects/open-worm-analysis-toolbox/examples/video_to_features.py", line 142, in main() File "D:/PycharmProjects/open-worm-analysis-toolbox/examples/video_to_features.py", line 98, in main experiment_histograms = mv.HistogramManager(experiment_files) File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram_manager.py", line 104, in init new_histogram_set = self.init_histograms(worm_features) File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram_manager.py", line 203, in init_histograms return np.array([Histogram.create_histogram(f) for f in worm_features]) File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram_manager.py", line 203, in return np.array([Histogram.create_histogram(f) for f in worm_features]) File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py", line 149, in create_histogram return cls(feature) File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py", line 113, in init self.compute_covering_bins() File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py", line 278, in compute_covering_bins step=bin_width) ValueError: arange: cannot compute length

I am running Python 3.7.0 on a Windows 7 machine and using PyCharm 2018.2.1, with the following packages installed: atlas 0.27.0 nose 1.3.7 pandas 0.23.2 statsmodels 0.9.0 h5py 2.8.0 seaborn 0.9.0

I'm not sure what is wrong here. Any help would be appreciated.

Thank you!

JimHokanson commented 5 years ago

@tshao87

I'll try and replicate your result. Thanks.

JimHokanson commented 5 years ago

@tshao87

I fixed a couple of bugs that were due to changes in numpy behavior. I still need to fix a few things with getting proper setup but hopefully things will be ready in the next couple of days.

JimHokanson commented 5 years ago

@tshao87

Just checking in ...

I'm ending up needing to rewrite a lot of the post-processing statistics code. I'm almost done but there is still a bit more work to do. I'm not sure why these bugs weren't found earlier in testing ... I'll let you know as soon as it is ready to go.

mfshao commented 5 years ago

@JimHokanson

No problem. Thanks for keeping me posted.

JimHokanson commented 5 years ago

@tshao87

The examples should work now, including the referenced example. I've marked a couple as not working as well at the top of the files. If it works for you please close this issue.

For reference to everyone on changes made:

  1. The previous code had poor nan support. It makes me wonder if the numpy interface changed ...
  2. The referenced example was working with un-extended features, which was causing some problems. I've tried to make fewer assumptions regarding what types of features are being processed statistically, as well as fewer assumptions about the consistency in features processed across each video. I've also changed the histograms into a clearly defined matrix where each row consists of the same feature. This was not well handled before.
  3. Finally, I was getting an error in the false discovery adjustment code (p-values to q-values). I ended up rewriting that code based on the Matlab version and everything seems to be working now.