search

Social-Evolution-and-Behavior / anTraX

anTraX: high throughput tracking of color-tagged insects
https://antrax.readthedocs.io/
GNU General Public License v3.0
17 stars 3 forks source link

'antrax configure' crashes on Debian 10 #4

Closed janamach closed 3 years ago

janamach commented 3 years ago

Hi there,

I am very excited to try out your software. I was following your instructions for Linux and I think I got everything right. I do not have a full MATLAB installation at the moment, so I went with the MATLAB Runtime option instead.

Problem: when running

$ antrax configure anTraX-data/JS16/

I see a window appear (see screenshot) and then it disappears with no warning. There is no error message in the terminal too. Screenshot from 2021-01-19 23-47-36

Any thoughts why this may be happening? Is there a traceback/verbose mode available?

asafgal commented 3 years ago

Hey Jana, thanks for trying out anTraX!

I'm a bit baffled by your issue. Usually, a crash without an error message happens when MCR is not installed properly, but as the GUI did appear for you, this is not the case. Otherwise, usually an informative error stack is printed on terminal. I actually never tried anTraX on Debian, but I don't see a reason why it will behave unexpectedly.

Some things that might help pinpoint the issue:

janamach commented 3 years ago

Hi Asaf,

Thank you for your reply!

I installed it on another machine that is running Ubuntu 20.04, but the problem persists.

Are there other messages in the terminal between running the command and the crash? Can you paste it here? It will help me understand at what point the program crashed.

There are no error messages whatsoever, the output looks like this:

(antrax) jana@str09:~/src$ antrax configure anTraX-data/JS16/
(antrax) jana@str09:~/src$

Try deleting the session directory ('antrax_demo') under JS16, and try again. Do you see the prompt for creating a new session?

(antrax) jana@str09:~/src/anTraX-data/JS16$ rm -rf antrax_demo/
(antrax) jana@str09:~/src/anTraX-data/JS16$ ls
videos
(antrax) jana@str09:~/src/anTraX-data/JS16$ antrax configure .

A GUI opened: Screenshot from 2021-01-20 15-03-29 I selected the second option and entered a new name: Screenshot from 2021-01-20 15-06-00

Clicked "OK" and then it crashed. It did create a folder, but it was empty:

(antrax) jana@str09:~/src/anTraX-data/JS16$ ls
ant_test  videos
(antrax) jana@str09:~/src/anTraX-data/JS16$ tree ant_test/
ant_test/
├── graphs
├── images
├── parameters
└── tracklets

4 directories, 0 files

Try running 'antrax configure' without the path to the experiment. If it loads ok, choose 'open' from the 'Experiment' menu.

This opened up a GUI: Screenshot from 2021-01-20 14-49-40 I selected the JS16 folder that still contained the antrax_demo/ folder and the thing crashed in the exact same way as above (i.e., no error traceback). After removing the antrax_demo/ folder, selecting the JS16 folder in the GUI did nothing. Clicking Help > About also did not do anything -- I am wondering if the GUI that I do get is kind of "broken"?

I used the same MCR installation file on both Ubuntu and Debian machines (MATLAB_Runtime_R2019a_Update_9_glnxa64.zip). Could this be the problem? What Linux distribution(s) did you try? Does Python version play a role? I used Python 3.6.3 and 3.6.8, because tensorflow 1.15.0 doesn't exist for Python 3.8 (my default choice).

asafgal commented 3 years ago

ok, I think I understand why it doesn't display anything to the stdout. I compiled a new version for debugging, it is in the branch 'debug-jana'. Can you give it a try?

Before running, also set debug mode:

export ANTRAX_DEBUG_MODE=True

Hopefully the printouts will help us debug the issue now.

My environment is Ubuntu 18.04/python3.6.8. Your GUI seems fine, the 'About' dialog got broken sometime ago and I didn't get to fixing it...

janamach commented 3 years ago

Now it gives an output:

(antrax-debug) jana@corvus:~/src$ antrax configure anTraX-data/JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

20/01/21 20:48:22 -D- antrax cli entry point
20/01/21 20:48:22 -D- running matlab mcr 
20/01/21 20:48:22 -D- command is: /home/jana/src/anTraX/bin/antrax_glnxa64_mcr_interface antrax anTraX-data/JS16/
20:48:34 -D- starting antrax configure app
20:48:34 -D- set_experiment function
20:48:36 -D- initializing expreader object
20:48:36 -I- Reading video information from file
20:48:48 -I- opening video file anTraX-data/JS16/videos//1_6/JS16_1.mp4
20:48:48 -W- Get info failrd, retrying
20:48:48 -W- Get info failrd, retrying
20:48:48 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20:48:49 -W- Get info failrd, retrying
20/01/21 20:48:56 -D- matlab app exited with code 249

I tried looking up the error, but did not find anything.

asafgal commented 3 years ago

It seems the problem is with ffmpeg. Can you try out these two commands in the terminal? 

ffprobe -v error -show_entries format=format_name,size -of default=noprint_wrappers=1:nokey=1 <path-to-video>
ffprobe -v error -show_entries stream=start_time,duration,pix_fmt,width,height,avg_frame_rate,nb_frames -of default=noprint_wrappers=1:nokey=1 <path-to-video>
janamach commented 3 years ago

Hmmm, I use ffmpeg quite often, I never have trouble with it. Here's the output (I used one of your sample videos as an input):

jana@corvus:~/src$ ffprobe -v error -show_entries format=format_name,size -of default=noprint_wrappers=1:nokey=1 anTraX-data/JS16/videos/1_6/JS16_1.mp4 
mov,mp4,m4a,3gp,3g2,mj2
10402940

and

jana@corvus:~/src$ ffprobe -v error -show_entries stream=start_time,duration,pix_fmt,width,height,avg_frame_rate,nb_frames -of default=noprint_wrappers=1:nokey=1 anTraX-data/JS16/videos/1_6/JS16_1.mp4
960
720
yuvj420p
10/1
0.000000
300.000000
3000

This is my ffmpeg version (in case that's relevant):

jana@corvus:~/src$ apt list -a ffmpeg
Listing... Done
ffmpeg/stable,stable,now 7:4.1.6-1~deb10u1 amd64 [installed]
asafgal commented 3 years ago

Can you try pulling and running again? Should print more relevant info now... Thanks for your patience!

janamach commented 3 years ago

Thanks for your patience!

Thank you for your support! I am used to this type of troubleshooting, so it's not much trouble for me. I also hope this will be useful for the future users.

On Ubuntu 20.04 machine:

(antrax-debug-jan21) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

13:51:48 -I- Reading video information from file
13:52:02 -I- opening video file JS16/videos//1_6/JS16_1.mp4
13:52:02 -W- Get info failed, retrying
13:52:02 -W- Get info failed, retrying
13:52:02 -W- Get info failed, retrying
13:52:02 -W- Get info failed, retrying
13:52:03 -W- Get info failed, retrying
13:52:03 -W- Get info failed, retrying
13:52:03 -W- Get info failed, retrying
13:52:03 -W- Get info failed, retrying
13:52:03 -W- Get info failed, retrying
13:52:03 -W- Get info failed, retrying
13:52:03 -W- Get info failed, retrying
13:52:03 -E- ffinfo failed 10 times
Error using ffreader/collectInfo (line 175)
Index exceeds the number of array elements (1).
Error in ffreader (line 49)

Error in expreader/open_vid (line 393)

Error in expreader/fetch_frame (line 338)

Error in expreader/read_frame (line 263)

Error in trhandles/read_frame (line 415)

Error in antrax/set_Trck (line 410)

Error in antrax/set_experiment (line 344)

Error in antrax/startupFcn (line 1184)

Error in appdesigner.internal.service.AppManagementService/tryCallback (line 336)

Error in matlab.apps.AppBase/runStartupFcn (line 41)

Error in antrax (line 3563)

Error in antrax_mcr_interface (line 8)
MATLAB:badsubscript
(antrax-debug-jan21) jana@str09:~/src/anTraX-data$

In the meantime I got a MATLAB R2020b license key, I will install it on my work computer (Ubuntu 20.04) and report back.

asafgal commented 3 years ago

Definitely helpful! I think we're getting there :) Can you please pull&run again? This time delete first the file JS16/antrax_demo/parameters/movies_info.txt?

Also - you will need matlab 2019a to run anTraX (they annoyingly made some incompatible change in recent releases that I haven't accounted for yet). It's the same license - so you can install 2019a in parallel to 2020b. Just make sure to install the 2019a python engine into the anTraX environment.

janamach commented 3 years ago

Here is the output with MCR:

(antrax-debug-jan21) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

16:16:32 -I- Reading video information from file
16:16:47 -I- opening video file JS16/videos//1_6/JS16_1.mp4
Index exceeds the number of array elements (1).
Error in ffinfo (line 30)

Error in ffreader/collectInfo (line 166)

Error in ffreader (line 49)

Error in expreader/open_vid (line 393)

Error in expreader/fetch_frame (line 338)

Error in expreader/read_frame (line 263)

Error in trhandles/read_frame (line 415)

Error in antrax/set_Trck (line 410)

Error in antrax/set_experiment (line 344)

Error in antrax/startupFcn (line 1184)

Error in appdesigner.internal.service.AppManagementService/tryCallback (line 336)

Error in matlab.apps.AppBase/runStartupFcn (line 41)

Error in antrax (line 3563)

Error in antrax_mcr_interface (line 8)
MATLAB:badsubscript

I tried MATLAB 2020b before I saw your message and got an error. I installed R2019a and the activation key worked as you said -- thanks, I didn't know! But with the full installation of R2019a I also get an error. I might be doing something wrong, so I will include the relevant output:

To keep things separate, I created a new python environment with a fresh installation of anTraX from the debug branch and installed the Matlab 2019a python engine:

(matlab2019a) jana@str09:/usr/local/MATLAB/R2019a/extern/engines/python$ python setup.py install
running install
running build
running build_py
running install_lib
creating /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab
copying build/lib/matlab/__init__.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab
copying build/lib/matlab/mlarray.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab
copying build/lib/matlab/mlexceptions.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab
creating /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/matlabengine.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/enginehelper.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/__init__.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/fevalfuture.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/futureresult.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/enginesession.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/engineerror.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/matlabfuture.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/basefuture.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
copying build/lib/matlab/engine/_arch.txt -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine
creating /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/_internal
copying build/lib/matlab/_internal/__init__.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/_internal
copying build/lib/matlab/_internal/mlarray_utils.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/_internal
copying build/lib/matlab/_internal/mlarray_sequence.py -> /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/_internal
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/__init__.py to __init__.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/mlarray.py to mlarray.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/mlexceptions.py to mlexceptions.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/matlabengine.py to matlabengine.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/enginehelper.py to enginehelper.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/__init__.py to __init__.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/fevalfuture.py to fevalfuture.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/futureresult.py to futureresult.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/enginesession.py to enginesession.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/engineerror.py to engineerror.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/matlabfuture.py to matlabfuture.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/basefuture.py to basefuture.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/_internal/__init__.py to __init__.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/_internal/mlarray_utils.py to mlarray_utils.cpython-36.pyc
byte-compiling /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/_internal/mlarray_sequence.py to mlarray_sequence.cpython-36.pyc
running install_egg_info
Writing /home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlabengineforpython-R2018a-py3.6.egg-info

And then I tried to run anTraX:

(matlab2019a) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

Undefined function 'strel' for input arguments of type 'char'.

Error in trhandles/load_params (line 20)
Trck.prmtrs.segmentation_ImClosingStrel = strel('disk',Trck.prmtrs.segmentation_ImClosingSize);

Error in trhandles.load (line 1021)
            Trck.load_params;

Error in antrax/set_experiment (line 291)
                    app.Trck = trhandles.load(app.expdir);

Error in antrax/startupFcn (line 1184)
            set_experiment(app, Trck, p.Results.session)

Error in antrax (line 3563)
            runStartupFcn(app, @(app)startupFcn(app, varargin{:}))

Traceback (most recent call last):
  File "/home/jana/anaconda3/envs/matlab2019a/bin/antrax", line 8, in <module>
    sys.exit(main())
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/cli.py", line 651, in main
    """)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/sigtools/modifiers.py", line 158, in __call__
    return self.func(*args, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 363, in run
    ret = cli(*args)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 262, in _cli
    return func('{0} {1}'.format(name, command), *args)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/cli.py", line 104, in configure
    launch_matlab_app('antrax', args, mcr=mcr)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/matlab.py", line 198, in launch_matlab_app
    app = eval('eng.' + appname + '(' + ','.join([str(a) for a in args]) + ')')
  File "<string>", line 1, in <module>
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/matlabengine.py", line 71, in __call__
    _stderr, feval=True).result()
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/futureresult.py", line 67, in result
    return self.__future.result(timeout)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/fevalfuture.py", line 82, in result
    self._result = pythonengine.getFEvalResult(self._future,self._nargout, None, out=self._out, err=self._err)
matlab.engine.MatlabExecutionError: 
  File /home/jana/src/anTraX/matlab/@trhandles/load_params.m, line 20, in load_params

  File /home/jana/src/anTraX/matlab/@trhandles/trhandles.m, line 1021, in trhandles.load

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 291, in antrax.set_experiment

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 1184, in antrax.startupFcn

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 3563, in antrax.antrax
Undefined function 'strel' for input arguments of type 'char'.

The error I received when I first tried to run it with MATLAB R2020b looked very similar. I think this is irrelevant to the first problem, but in case you can clearly see what I did wrong -- can you let me know?

asafgal commented 3 years ago

It seems you turned off the debug mode, can you set it to True and rerun?

Regarding the matlab - 'strel' is a function in the image processing toolbox, do you have it installed? Can you try just running this function in the matlab command window?

janamach commented 3 years ago

Ah, sorry! Debug more on:

(antrax-debug-jan21) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

21/01/21 17:28:59 -D- antrax cli entry point
21/01/21 17:28:59 -D- running matlab mcr 
21/01/21 17:28:59 -D- command is: /home/jana/src/anTraX/bin/antrax_glnxa64_mcr_interface antrax JS16/
17:29:12 -D- starting antrax configure app
17:29:12 -D- set_experiment function
17:29:14 -D- initializing expreader object
17:29:14 -I- Reading video information from file
17:29:30 -I- opening video file JS16/videos//1_6/JS16_1.mp4
17:29:30 -D- Running ffprobe #1 for file JS16/videos//1_6/JS16_1.mp4

Index exceeds the number of array elements (1).
Error in ffinfo (line 30)

Error in ffreader/collectInfo (line 166)

Error in ffreader (line 49)

Error in expreader/open_vid (line 393)

Error in expreader/fetch_frame (line 338)

Error in expreader/read_frame (line 263)

Error in trhandles/read_frame (line 415)

Error in antrax/set_Trck (line 410)

Error in antrax/set_experiment (line 344)

Error in antrax/startupFcn (line 1184)

Error in appdesigner.internal.service.AppManagementService/tryCallback (line 336)

Error in matlab.apps.AppBase/runStartupFcn (line 41)

Error in antrax (line 3563)

Error in antrax_mcr_interface (line 8)
MATLAB:badsubscript
21/01/21 17:29:36 -D- matlab app exited with code 249
(antrax-debug-jan21) jana@str09:~/src/anTraX-data$
asafgal commented 3 years ago

So it seems that the ffprobe command does not produce any output, when executed through the matlab program. I'm not sure what the reason is, as it produced the correct output through the terminal.

Since you have matlab up and running, try this in the matlab command line, and see if you get the expected output:

[~,ffprobe] = system('which ffprobe');
ffprobe = ffprobe(1:end-1);
system([ffprobe ' -v error -show_entries format=format_name,size -of default=noprint_wrappers=1:nokey=1 ''',<path-to-video>,'''',' 2> /dev/null']);

If that works, than try running the anTraX function:

ffinfo(<path-vid>)
janamach commented 3 years ago

Regarding the matlab - 'strel' is a function in the image processing toolbox, do you have it installed?

I did not have it installed, thank you for the tip :)

Running antrax configure JS16/ with matlab installation gives a different looking error to the one with MCR, but both complain about ffmpeg:

(matlab2019a) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

21/01/21 18:05:47 -D- antrax cli entry point
18:06:03 -D- starting antrax configure app
18:06:04 -D- set_experiment function
18:06:06 -D- initializing expreader object
18:06:06 -I- Reading video information from file
18:06:21 -I- opening video file JS16/videos//1_6/JS16_1.mp4
18:06:21 -D- Running ffprobe #1 for file JS16/videos//1_6/JS16_1.mp4

Index exceeds the number of array elements (1).

Error in ffinfo (line 30)
info.filesize = str2double(out{2});

Error in ffreader/collectInfo (line 166)
            self.info = ffinfo(self.file);

Error in ffreader (line 49)
            obj.collectInfo;

Error in expreader/open_vid (line 393)
            er.vr = er.VideoReader(er.cur_mov_name);

Error in expreader/fetch_frame (line 338)
                er.open_vid(m);

Error in expreader/read_frame (line 263)
            er.fetch_frame(t);

Error in trhandles/read_frame (line 415)
            [Trck.currfrm.CData,Trck.currfrm.dat] = Trck.er.read_frame(f);

Error in antrax/set_Trck (line 410)
            app.himage = imshow(app.Trck.read_frame(1),'Parent',app.ax);

Error in antrax/set_experiment (line 344)
            app.set_Trck();

Error in antrax/startupFcn (line 1184)
            set_experiment(app, Trck, p.Results.session)

Error in antrax (line 3563)
            runStartupFcn(app, @(app)startupFcn(app, varargin{:}))

Traceback (most recent call last):
  File "/home/jana/anaconda3/envs/matlab2019a/bin/antrax", line 8, in <module>
    sys.exit(main())
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/cli.py", line 651, in main
    """)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/sigtools/modifiers.py", line 158, in __call__
    return self.func(*args, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 363, in run
    ret = cli(*args)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 262, in _cli
    return func('{0} {1}'.format(name, command), *args)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/cli.py", line 104, in configure
    launch_matlab_app('antrax', args, mcr=mcr)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/matlab.py", line 198, in launch_matlab_app
    app = eval('eng.' + appname + '(' + ','.join([str(a) for a in args]) + ')')
  File "<string>", line 1, in <module>
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/matlabengine.py", line 71, in __call__
    _stderr, feval=True).result()
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/futureresult.py", line 67, in result
    return self.__future.result(timeout)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/fevalfuture.py", line 82, in result
    self._result = pythonengine.getFEvalResult(self._future,self._nargout, None, out=self._out, err=self._err)
matlab.engine.MatlabExecutionError: 
  File /home/jana/src/anTraX/matlab/misc/ffinfo.m, line 30, in ffinfo

  File /home/jana/src/anTraX/matlab/@ffreader/ffreader.m, line 166, in ffreader.collectInfo

  File /home/jana/src/anTraX/matlab/@ffreader/ffreader.m, line 49, in ffreader.ffreader

  File /home/jana/src/anTraX/matlab/@expreader/expreader.m, line 393, in expreader.open_vid

  File /home/jana/src/anTraX/matlab/@expreader/expreader.m, line 338, in expreader.fetch_frame

  File /home/jana/src/anTraX/matlab/@expreader/expreader.m, line 263, in expreader.read_frame

  File /home/jana/src/anTraX/matlab/@trhandles/trhandles.m, line 415, in trhandles.read_frame

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 410, in antrax.set_Trck

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 344, in antrax.set_experiment

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 1184, in antrax.startupFcn

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 3563, in antrax.antrax
Index exceeds the number of array elements (1).

Since you have matlab up and running, try this in the matlab command line, and see if you get the expected output

It didn't give any output and 'ffinfo' was undefined:

Screenshot from 2021-01-21 18-49-23

asafgal commented 3 years ago

'ffinfo' is an anTraX function, so you need to add the anTraX source dir to the matlab search path for it to be recognized in an interactive session:

addpath(genpath([<path-to-anTraX>,'/matlab']));

In the matlab prompt, try to see if ffprobe is indeed recognized:

[~,ffprobe] = system('which ffprobe');
ffprobe = ffprobe(1:end-1);
disp(ffprobe)
isfile(ffprobe)

And try again the command without the error redirect:

system([ffprobe ' -v error -show_entries format=format_name,size -of default=noprint_wrappers=1:nokey=1 ', <path-to-vid>]);
janamach commented 3 years ago

Ah, sorry -- matlab is still a foreign language to me :-) I will try that tomorrow when I am back at my work computer.

janamach commented 3 years ago

But why wait if I can do it now over ssh?

ffinfo:

>> addpath(genpath(['~/src/anTraX/','/matlab']));
>> ffinfo('JS16_1.mp4')
Index exceeds the number of array elements (1).

Error in ffinfo (line 30)
info.filesize = str2double(out{2});

>>

In the matlab prompt, try to see if ffprobe is indeed recognized:

>> [~,ffprobe] = system('which ffprobe');
>> ffprobe = ffprobe(1:end-1);
>> disp(ffprobe)
/usr/bin/ffprobe

>> isfile(ffprobe)

ans =
  logical

   1 

>>

And try again the command without the error redirect:

>> system([ffprobe ' -v error -show_entries format=format_name,size -of default=noprint_wrappers=1:nokey=1 ', 'JS16_1.mp4']); 
/usr/bin/ffprobe: /usr/local/MATLAB/R2019a/sys/os/glnxa64/libstdc++.so.6: version `GLIBCXX_3.4.25' not found (required by /lib/x86_64-linux-gnu/libopenmpt.so.0)
>>
asafgal commented 3 years ago

ok, we seem to be getting there.

anTraX loads the libstdc++ library from matlab (either the MCR or the full, depending on the mode). However, ffmpeg on your machine was compiled with a newer version. So when anTraX calls ffmpeg, it throws an error.

There are a few solutions for this that we can try (and I will need to find a permanent one at some point):

  1. Preload the system library before calling anTraX:
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6
antrax <antrax command>
  1. The 'brute force' solution is to delete the library from the matlab directory, so the system one will be loaded (change the paths if your have matlab installed elsewhere):
sudo mv /usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64/libstdc++.so.6 /usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64/libstdc++.so.6.orig
sudo mv /usr/local/MATLAB/R2019a/sys/os/glnxa64/libstdc++.so.6 /usr/local/MATLAB/R2019a/sys/os/glnxa64/libstdc++.so.6.orig
janamach commented 3 years ago

Oh cool! We're indeed getting there, it loads well with MCR with both solutions. On Ubuntu 20.04 the GUI loads and closes nicely:

(antrax-debug-jan21) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

22/01/21 09:39:55 -D- antrax cli entry point
22/01/21 09:39:55 -D- running matlab mcr 
22/01/21 09:39:55 -D- command is: /home/jana/src/anTraX/bin/antrax_glnxa64_mcr_interface antrax JS16/
09:40:07 -D- starting antrax configure app
09:40:08 -D- set_experiment function
09:40:09 -D- initializing expreader object
09:40:09 -I- Reading video information from file
09:40:23 -I- opening video file JS16/videos//1_6/JS16_1.mp4
09:40:23 -D- Running ffprobe #1 for file JS16/videos//1_6/JS16_1.mp4

mov,mp4,m4a,3gp,3g2,mj2
10402940

09:40:23 -D- Running ffprobe #2 for file JS16/videos//1_6/JS16_1.mp4

960
720
yuvj420p
10/1
0.000000
300.000000
3000

09:40:24 -D- antrax configure app ready!
app closing..
22/01/21 09:41:37 -D- matlab app exited with code 0

I will also test this tonight on Debian 10 at home.

But it's a different story with the MATLAB 2019a full installation, neither of the two solutions were successful. It loads further than before and crashes after successfully running ffprobe:

(matlab2019a) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

22/01/21 09:41:42 -D- antrax cli entry point
09:41:56 -D- starting antrax configure app
09:41:57 -D- set_experiment function
09:41:58 -D- initializing expreader object
09:41:58 -I- Reading video information from file
09:42:12 -I- opening video file JS16/videos//1_6/JS16_1.mp4
09:42:12 -D- Running ffprobe #1 for file JS16/videos//1_6/JS16_1.mp4

mov,mp4,m4a,3gp,3g2,mj2
10402940

09:42:12 -D- Running ffprobe #2 for file JS16/videos//1_6/JS16_1.mp4

960
720
yuvj420p
10/1
0.000000
300.000000
3000

09:42:12 -W- VideoReader error, recreating object..
09:42:12 -D- Running ffprobe #1 for file JS16/videos//1_6/JS16_1.mp4

mov,mp4,m4a,3gp,3g2,mj2
10402940

09:42:12 -D- Running ffprobe #2 for file JS16/videos//1_6/JS16_1.mp4

960
720
yuvj420p
10/1
0.000000
300.000000
3000

Undefined function 'popenr' for input arguments of type 'char'.

Error in ffreader/openPipe (line 152)
                    self.p = popenr([ffmpeg,' -loglevel quiet',' -ss ',num2str(ss),' -i "',self.file,'" -pix_fmt rgb24 -f image2pipe -vcodec rawvideo pipe:1']);

Error in ffreader/read (line 77)
                self.openPipe(f);

Error in expreader/fetch_frame (line 348)
                er.last_frame0 = er.vr.read(mf);

Error in expreader/read_frame (line 263)
            er.fetch_frame(t);

Error in trhandles/read_frame (line 415)
            [Trck.currfrm.CData,Trck.currfrm.dat] = Trck.er.read_frame(f);

Error in antrax/set_Trck (line 410)
            app.himage = imshow(app.Trck.read_frame(1),'Parent',app.ax);

Error in antrax/set_experiment (line 344)
            app.set_Trck();

Error in antrax/startupFcn (line 1184)
            set_experiment(app, Trck, p.Results.session)

Error in antrax (line 3563)
            runStartupFcn(app, @(app)startupFcn(app, varargin{:}))

Traceback (most recent call last):
  File "/home/jana/anaconda3/envs/matlab2019a/bin/antrax", line 8, in <module>
    sys.exit(main())
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/cli.py", line 651, in main
    """)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/sigtools/modifiers.py", line 158, in __call__
    return self.func(*args, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 363, in run
    ret = cli(*args)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 262, in _cli
    return func('{0} {1}'.format(name, command), *args)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/cli.py", line 104, in configure
    launch_matlab_app('antrax', args, mcr=mcr)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/antrax/matlab.py", line 198, in launch_matlab_app
    app = eval('eng.' + appname + '(' + ','.join([str(a) for a in args]) + ')')
  File "<string>", line 1, in <module>
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/matlabengine.py", line 71, in __call__
    _stderr, feval=True).result()
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/futureresult.py", line 67, in result
    return self.__future.result(timeout)
  File "/home/jana/anaconda3/envs/matlab2019a/lib/python3.6/site-packages/matlab/engine/fevalfuture.py", line 82, in result
    self._result = pythonengine.getFEvalResult(self._future,self._nargout, None, out=self._out, err=self._err)
matlab.engine.MatlabExecutionError: 
  File /home/jana/src/anTraX/matlab/@ffreader/ffreader.m, line 152, in ffreader.openPipe

  File /home/jana/src/anTraX/matlab/@ffreader/ffreader.m, line 77, in ffreader.read

  File /home/jana/src/anTraX/matlab/@expreader/expreader.m, line 348, in expreader.fetch_frame

  File /home/jana/src/anTraX/matlab/@expreader/expreader.m, line 263, in expreader.read_frame

  File /home/jana/src/anTraX/matlab/@trhandles/trhandles.m, line 415, in trhandles.read_frame

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 410, in antrax.set_Trck

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 344, in antrax.set_experiment

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 1184, in antrax.startupFcn

  File /home/jana/src/anTraX/matlab/apps/antrax.mlapp, line 3563, in antrax.antrax
Undefined function 'popenr' for input arguments of type 'char'.
asafgal commented 3 years ago

Great!

The new issue with the full licensed matlab is because you need to compile the popnr file first:

In the matlab prompt:

cd <path-to-anTraX>/matlab/external/popenmatlab
mex -setup
mex popenr.c
janamach commented 3 years ago

Ah, thank you!

It is still behaving weird though. When I run the command, it starts somewhat normally:

(matlab2019a) jana@str09:~/src/anTraX-data$ antrax configure JS16/

==================================================================================

Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

22/01/21 10:27:41 -D- antrax cli entry point
10:27:56 -D- starting antrax configure app
10:27:58 -D- set_experiment function
10:27:58 -D- initializing expreader object
10:27:58 -I- Reading video information from file
10:28:12 -I- opening video file JS16/videos//1_6/JS16_1.mp4
10:28:12 -D- Running ffprobe #1 for file JS16/videos//1_6/JS16_1.mp4

mov,mp4,m4a,3gp,3g2,mj2
10402940

10:28:12 -D- Running ffprobe #2 for file JS16/videos//1_6/JS16_1.mp4

960
720
yuvj420p
10/1
0.000000
300.000000
3000

10:28:13 -D- antrax configure app ready!

Then, a very unhappy GUI appears:

Screenshot from 2021-01-22 11-21-11

And it stays like this until I quit it. Closing the GUI by clicking on x doesn't give the -D- matlab app exited with code 0 message. Instead, it doesn't say anything:

10:28:13 -D- antrax configure app ready!
(matlab2019a) jana@str09:~/src/anTraX-data$

I received a warning when compiling the popenr file -- could this be the problem?

(matlab2019a) jana@str09:~/src/anTraX/matlab/external/popenmatlab$ mex -setup
MEX configured to use 'gcc' for C language compilation.
Warning: The MATLAB C and Fortran API has changed to support MATLAB
     variables with more than 2^32-1 elements. You will be required
     to update your code to utilize the new API.
     You can find more information about this at:
     https://www.mathworks.com/help/matlab/matlab_external/upgrading-mex-files-to-use-64-bit-api.html.

To choose a different language, execute one from the following:
 mex -setup C++ 
 mex -setup FORTRAN
(matlab2019a) jana@str09:~/src/anTraX/matlab/external/popenmatlab$ mex popenr.c
Building with 'gcc'.
Warning: You are using gcc version '9.3.0'. The version of gcc is not supported. The version currently supported with MEX is '6.3.x'. For a list of currently supported compilers see: https://www.mathworks.com/support/compilers/current_release.
MEX completed successfully.
asafgal commented 3 years ago

The mex compilation warnings are ok.

Regarding the frozen GUI: Can you try launching the app from the matlab prompt? Use this syntax (after setting the path as before):

Trck = trhandles('fullpath to JS16');
antrax(Trck);
janamach commented 3 years ago

That opened up a GUI that failed to load the session, the prompt was complaining:

>> addpath(genpath(['~/src/anTraX/','/matlab']));
>> Trck = trhandles('~/src/anTraX-data/JS16');
15:21:10 -D- initializing Trck object
15:21:10 -D- finding sessions
15:21:10 -I- Loading tracking session from expdir
15:21:12 -D- initializing expreader object
15:21:12 -I- Reading video information from file
>> antrax(Trck);
15:21:31 -D- starting antrax configure app
Warning: MATLAB has disabled some advanced graphics rendering features by switching to software OpenGL. For more information, click here. 
15:21:40 -D- set_experiment function
15:21:41 -I- opening video file ~/src/anTraX-data/JS16/videos//1_6/JS16_1.mp4
15:21:41 -D- Running ffprobe #1 for file ~/src/anTraX-data/JS16/videos//1_6/JS16_1.mp4

Index exceeds the number of array elements (1).

Error in ffinfo (line 30)
info.filesize = str2double(out{2});

Error in ffreader/collectInfo (line 166)
            self.info = ffinfo(self.file);

Error in ffreader (line 49)
            obj.collectInfo;

Error in expreader/open_vid (line 393)
            er.vr = er.VideoReader(er.cur_mov_name);

Error in expreader/fetch_frame (line 338)
                er.open_vid(m);

Error in expreader/read_frame (line 263)
            er.fetch_frame(t);

Error in trhandles/read_frame (line 415)
            [Trck.currfrm.CData,Trck.currfrm.dat] = Trck.er.read_frame(f);

Error in antrax/set_Trck (line 410)
            app.himage = imshow(app.Trck.read_frame(1),'Parent',app.ax);

Error in antrax/set_experiment (line 344)
            app.set_Trck();

Error in antrax/startupFcn (line 1184)
            set_experiment(app, Trck, p.Results.session)

Error in antrax (line 3563)
            runStartupFcn(app, @(app)startupFcn(app, varargin{:}))

app closing..
>>

The click here in

Warning: MATLAB has disabled some advanced graphics rendering features by switching to software OpenGL. For more information, click here.

opens the MATLAB Documentation

asafgal commented 3 years ago

This seems to be related to the libstdc++ issue, and not the new frozen GUI issue. You need to set the LD_PRELOAD before running matlab. You can add this alias to your bash profile, so it will do it automatically every time:

alias matlab="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 matlab"

Don't forget to source...

janamach commented 3 years ago

I was about to say that I used the 'brute force' approach earlier:

$ ls /usr/local/MATLAB/R2019a/sys/os/glnxa64/libstdc++.so.6*
/usr/local/MATLAB/R2019a/sys/os/glnxa64/libstdc++.so.6.0.22  /usr/local/MATLAB/R2019a/sys/os/glnxa64/libstdc++.so.6.orig

and that setting LD_PRELOAD didn't make a difference, but then I ssh'ed to that computer with X forwarding and the program started successfully without me doing anything additionally. This was unexpected, I will check that again next time I am physically there.

With MCR it starts successfully now on both Ubuntu 20.04 and Debian 10. I am really happy about that since I am more likely to use MCR in the long run.

Thank you so much for all your help, it was incredibly useful and I learned from it. Let me know if I can help you debug or test something else.

Looking forward to using anTraX :-)

asafgal commented 3 years ago

Great!

I'm closing this issue then, feel free to open a new one with any question or issue down the road. Also, I will be happy to hear (either here or by email) any comment you might have, and whether you found it useful (or not) for your data.

Good luck!