"ValueError: invalid literal for int() with base 10: 'images' " during classification step

[Giacomoalciatore]

Hello, I recently ran into this error while launching the classification step and I'm not sure where it is coming from. What I have tried: 1) Run the classification on the same cluster profile and with the same conda environment on another dataset, it worked 2) Repeat the tracking, in case the images files were corrupted. Same error pops up 3) the error is the same for all videos in the camera

I am at a loss because it seems like it's an incompatibility between this specific experimental folder and a code which works well with other folders.

Here is the error message:

WARNING:tensorflow:From /cluster/home/yulrich/miniconda3/envs/antrax2/lib/python3.6/site-packages/tensorflow_core/python/ops/init_ops.py:97: calling GlorotUniform.__init__ (from tensorflow.python
.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /cluster/home/yulrich/miniconda3/envs/antrax2/lib/python3.6/site-packages/tensorflow_core/python/ops/init_ops.py:97: calling Zeros.__init__ (from tensorflow.python.ops.ini
t_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /cluster/home/yulrich/miniconda3/envs/antrax2/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (
from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /cluster/home/yulrich/miniconda3/envs/antrax2/lib/python3.6/site-packages/tensorflow_core/python/ops/init_ops.py:97: calling Ones.__init__ (from tensorflow.python.ops.init
_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
2021-10-04 14:00:46.890561: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-10-04 14:00:47.113362: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2021-10-04 14:00:47.113438: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (eu-g1-021-2): /proc/driver/nvidia/version does not exist
2021-10-04 14:00:47.114343: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-10-04 14:00:47.126270: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2249985000 Hz
2021-10-04 14:00:47.126717: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x558b01973730 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-10-04 14:00:47.126733: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Welcome to anTraX - a software for tracking color tagged ants (and other insects)

==================================================================================

Traceback (most recent call last):
  File "/cluster/home/user/miniconda3/envs/antrax2/bin/antrax", line 8, in <module>
    sys.exit(main())
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/antrax/cli.py", line 659, in main
    """)
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/sigtools/modifiers.py", line 158, in __call__
    return self.func(*args, **kwargs)
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/clize/runner.py", line 363, in run
    ret = cli(*args)
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/clize/runner.py", line 262, in _cli
    return func('{0} {1}'.format(name, command), *args)
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/antrax/cli.py", line 482, in classify
    c.predict_experiment(e, movlist=movlist, report=True)
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/antrax/classifier.py", line 369, in predict_experiment
    movieindex = [int(x.rstrip('.mat').split('_')[1]) for x in self.imagefiles]
  File "/cluster/home/user/miniconda3/envs/antrax2/lib/python3.6/site-packages/antrax/classifier.py", line 369, in <listcomp>
    movieindex = [int(x.rstrip('.mat').split('_')[1]) for x in self.imagefiles]
ValueError: invalid literal for int() with base 10: 'images'
Sender: LSF System <lsfadmin@eu-g1-042-4>
Subject: Job 186880930[38]: <Class_3a_v[1-56]> in cluster <euler> Exited

Job <Class_3a_v[1-56]> was submitted from host <eu-login-38> by user <user> in cluster <euler> at Mon Oct  4 14:00:09 2021
Job was executed on host(s) <4*eu-g1-042-4>, in queue <normal.24h>, as user <user> in cluster <euler> at Mon Oct  4 14:00:38 2021
</cluster/home/user> was used as the home directory.
</cluster/scratch/user/Age> was used as the working directory.
Started at Mon Oct  4 14:00:38 2021
Terminated at Mon Oct  4 14:00:49 2021
Results reported at Mon Oct  4 14:00:49 2021

[asafgal]

It seems there is a problem parsing the image filenames. Can you paste here the names of the files in the session/images folder?

[Giacomoalciatore]

Here is this folder:

(base) ibz-eve-160:~ giacomo$ ls /Volumes/Age_3_Mac/Experiment_Age_cam_3a/antrax/images/
frame_passed_1.mat  frame_passed_20.mat frame_passed_31.mat frame_passed_42.mat frame_passed_53.mat frame_score_13.mat  frame_score_24.mat  frame_score_35.mat  frame_score_46.mat  frame_score_6.mat   images_17.mat       images_28.mat       images_39.mat       images_5.mat
frame_passed_10.mat frame_passed_21.mat frame_passed_32.mat frame_passed_43.mat frame_passed_54.mat frame_score_14.mat  frame_score_25.mat  frame_score_36.mat  frame_score_47.mat  frame_score_7.mat   images_18.mat       images_29.mat       images_4.mat        images_50.mat
frame_passed_11.mat frame_passed_22.mat frame_passed_33.mat frame_passed_44.mat frame_passed_55.mat frame_score_15.mat  frame_score_26.mat  frame_score_37.mat  frame_score_48.mat  frame_score_8.mat   images_19.mat       images_3.mat        images_40.mat       images_51.mat
frame_passed_12.mat frame_passed_23.mat frame_passed_34.mat frame_passed_45.mat frame_passed_56.mat frame_score_16.mat  frame_score_27.mat  frame_score_38.mat  frame_score_49.mat  frame_score_9.mat   images_2.mat        images_30.mat       images_41.mat       images_52.mat
frame_passed_13.mat frame_passed_24.mat frame_passed_35.mat frame_passed_46.mat frame_passed_6.mat  frame_score_17.mat  frame_score_28.mat  frame_score_39.mat  frame_score_5.mat   images_1.mat        images_20.mat       images_31.mat       images_42.mat       images_53.mat
frame_passed_14.mat frame_passed_25.mat frame_passed_36.mat frame_passed_47.mat frame_passed_7.mat  frame_score_18.mat  frame_score_29.mat  frame_score_4.mat   frame_score_50.mat  images_10.mat       images_21.mat       images_32.mat       images_43.mat       images_54.mat
frame_passed_15.mat frame_passed_26.mat frame_passed_37.mat frame_passed_48.mat frame_passed_8.mat  frame_score_19.mat  frame_score_3.mat   frame_score_40.mat  frame_score_51.mat  images_11.mat       images_22.mat       images_33.mat       images_44.mat       images_55.mat
frame_passed_16.mat frame_passed_27.mat frame_passed_38.mat frame_passed_49.mat frame_passed_9.mat  frame_score_2.mat   frame_score_30.mat  frame_score_41.mat  frame_score_52.mat  images_12.mat       images_23.mat       images_34.mat       images_45.mat       images_56.mat
frame_passed_17.mat frame_passed_28.mat frame_passed_39.mat frame_passed_5.mat  frame_score_1.mat   frame_score_20.mat  frame_score_31.mat  frame_score_42.mat  frame_score_53.mat  images_13.mat       images_24.mat       images_35.mat       images_46.mat       images_6.mat
frame_passed_18.mat frame_passed_29.mat frame_passed_4.mat  frame_passed_50.mat frame_score_10.mat  frame_score_21.mat  frame_score_32.mat  frame_score_43.mat  frame_score_54.mat  images_14.mat       images_25.mat       images_36.mat       images_47.mat       images_7.mat
frame_passed_19.mat frame_passed_3.mat  frame_passed_40.mat frame_passed_51.mat frame_score_11.mat  frame_score_22.mat  frame_score_33.mat  frame_score_44.mat  frame_score_55.mat  images_15.mat       images_26.mat       images_37.mat       images_48.mat       images_8.mat
frame_passed_2.mat  frame_passed_30.mat frame_passed_41.mat frame_passed_52.mat frame_score_12.mat  frame_score_23.mat  frame_score_34.mat  frame_score_45.mat  frame_score_56.mat  images_16.mat       images_27.mat       images_38.mat       images_49.mat       images_9.mat

While this one run smoothly:

(base) ibz-eve-160:~ giacomo$  ls /Volumes/Age_3_Mac/Experiment_Age_cam_3b/antrax/images/
frame_passed_1.mat  frame_passed_124.mat    frame_passed_34.mat frame_passed_6.mat  frame_passed_85.mat frame_score_11.mat  frame_score_2.mat   frame_score_45.mat  frame_score_70.mat  frame_score_96.mat  images_120.mat      images_30.mat       images_56.mat       images_81.mat
frame_passed_10.mat frame_passed_125.mat    frame_passed_35.mat frame_passed_60.mat frame_passed_86.mat frame_score_110.mat frame_score_20.mat  frame_score_46.mat  frame_score_71.mat  frame_score_97.mat  images_121.mat      images_31.mat       images_57.mat       images_82.mat
frame_passed_100.mat    frame_passed_126.mat    frame_passed_36.mat frame_passed_61.mat frame_passed_87.mat frame_score_111.mat frame_score_21.mat  frame_score_47.mat  frame_score_72.mat  frame_score_98.mat  images_122.mat      images_32.mat       images_58.mat       images_83.mat
frame_passed_101.mat    frame_passed_127.mat    frame_passed_37.mat frame_passed_62.mat frame_passed_88.mat frame_score_112.mat frame_score_22.mat  frame_score_48.mat  frame_score_73.mat  frame_score_99.mat  images_123.mat      images_33.mat       images_59.mat       images_84.mat
frame_passed_102.mat    frame_passed_128.mat    frame_passed_38.mat frame_passed_63.mat frame_passed_89.mat frame_score_113.mat frame_score_23.mat  frame_score_49.mat  frame_score_74.mat  images_1.mat        images_124.mat      images_34.mat       images_6.mat        images_85.mat
frame_passed_103.mat    frame_passed_13.mat frame_passed_39.mat frame_passed_64.mat frame_passed_9.mat  frame_score_114.mat frame_score_24.mat  frame_score_5.mat   frame_score_75.mat  images_10.mat       images_125.mat      images_35.mat       images_60.mat       images_86.mat
frame_passed_104.mat    frame_passed_14.mat frame_passed_4.mat  frame_passed_65.mat frame_passed_90.mat frame_score_115.mat frame_score_25.mat  frame_score_50.mat  frame_score_76.mat  images_100.mat      images_126.mat      images_36.mat       images_61.mat       images_87.mat
frame_passed_105.mat    frame_passed_15.mat frame_passed_40.mat frame_passed_66.mat frame_passed_91.mat frame_score_116.mat frame_score_26.mat  frame_score_51.mat  frame_score_77.mat  images_101.mat      images_127.mat      images_37.mat       images_62.mat       images_88.mat
frame_passed_106.mat    frame_passed_16.mat frame_passed_41.mat frame_passed_67.mat frame_passed_92.mat frame_score_117.mat frame_score_27.mat  frame_score_52.mat  frame_score_78.mat  images_102.mat      images_128.mat      images_38.mat       images_63.mat       images_89.mat
frame_passed_107.mat    frame_passed_17.mat frame_passed_42.mat frame_passed_68.mat frame_passed_93.mat frame_score_118.mat frame_score_28.mat  frame_score_53.mat  frame_score_79.mat  images_103.mat      images_13.mat       images_39.mat       images_64.mat       images_9.mat
frame_passed_108.mat    frame_passed_18.mat frame_passed_43.mat frame_passed_69.mat frame_passed_94.mat frame_score_119.mat frame_score_29.mat  frame_score_54.mat  frame_score_8.mat   images_104.mat      images_14.mat       images_4.mat        images_65.mat       images_90.mat
frame_passed_109.mat    frame_passed_19.mat frame_passed_44.mat frame_passed_7.mat  frame_passed_95.mat frame_score_12.mat  frame_score_3.mat   frame_score_55.mat  frame_score_80.mat  images_105.mat      images_15.mat       images_40.mat       images_66.mat       images_91.mat
frame_passed_11.mat frame_passed_2.mat  frame_passed_45.mat frame_passed_70.mat frame_passed_96.mat frame_score_120.mat frame_score_30.mat  frame_score_56.mat  frame_score_81.mat  images_106.mat      images_16.mat       images_41.mat       images_67.mat       images_92.mat
frame_passed_110.mat    frame_passed_20.mat frame_passed_46.mat frame_passed_71.mat frame_passed_97.mat frame_score_121.mat frame_score_31.mat  frame_score_57.mat  frame_score_82.mat  images_107.mat      images_17.mat       images_42.mat       images_68.mat       images_93.mat
frame_passed_111.mat    frame_passed_21.mat frame_passed_47.mat frame_passed_72.mat frame_passed_98.mat frame_score_122.mat frame_score_32.mat  frame_score_58.mat  frame_score_83.mat  images_108.mat      images_18.mat       images_43.mat       images_69.mat       images_94.mat
frame_passed_112.mat    frame_passed_22.mat frame_passed_48.mat frame_passed_73.mat frame_passed_99.mat frame_score_123.mat frame_score_33.mat  frame_score_59.mat  frame_score_84.mat  images_109.mat      images_19.mat       images_44.mat       images_7.mat        images_95.mat
frame_passed_113.mat    frame_passed_23.mat frame_passed_49.mat frame_passed_74.mat frame_score_1.mat   frame_score_124.mat frame_score_34.mat  frame_score_6.mat   frame_score_85.mat  images_11.mat       images_2.mat        images_45.mat       images_70.mat       images_96.mat
frame_passed_114.mat    frame_passed_24.mat frame_passed_5.mat  frame_passed_75.mat frame_score_10.mat  frame_score_125.mat frame_score_35.mat  frame_score_60.mat  frame_score_86.mat  images_110.mat      images_20.mat       images_46.mat       images_71.mat       images_97.mat
frame_passed_115.mat    frame_passed_25.mat frame_passed_50.mat frame_passed_76.mat frame_score_100.mat frame_score_126.mat frame_score_36.mat  frame_score_61.mat  frame_score_87.mat  images_111.mat      images_21.mat       images_47.mat       images_72.mat       images_98.mat
frame_passed_116.mat    frame_passed_26.mat frame_passed_51.mat frame_passed_77.mat frame_score_101.mat frame_score_127.mat frame_score_37.mat  frame_score_62.mat  frame_score_88.mat  images_112.mat      images_22.mat       images_48.mat       images_73.mat       images_99.mat
frame_passed_117.mat    frame_passed_27.mat frame_passed_52.mat frame_passed_78.mat frame_score_102.mat frame_score_128.mat frame_score_38.mat  frame_score_63.mat  frame_score_89.mat  images_113.mat      images_23.mat       images_49.mat       images_74.mat
frame_passed_118.mat    frame_passed_28.mat frame_passed_53.mat frame_passed_79.mat frame_score_103.mat frame_score_13.mat  frame_score_39.mat  frame_score_64.mat  frame_score_9.mat   images_114.mat      images_24.mat       images_5.mat        images_75.mat
frame_passed_119.mat    frame_passed_29.mat frame_passed_54.mat frame_passed_8.mat  frame_score_104.mat frame_score_14.mat  frame_score_4.mat   frame_score_65.mat  frame_score_90.mat  images_115.mat      images_25.mat       images_50.mat       images_76.mat
frame_passed_12.mat frame_passed_3.mat  frame_passed_55.mat frame_passed_80.mat frame_score_105.mat frame_score_15.mat  frame_score_40.mat  frame_score_66.mat  frame_score_91.mat  images_116.mat      images_26.mat       images_51.mat       images_77.mat
frame_passed_120.mat    frame_passed_30.mat frame_passed_56.mat frame_passed_81.mat frame_score_106.mat frame_score_16.mat  frame_score_41.mat  frame_score_67.mat  frame_score_92.mat  images_117.mat      images_27.mat       images_52.mat       images_78.mat
frame_passed_121.mat    frame_passed_31.mat frame_passed_57.mat frame_passed_82.mat frame_score_107.mat frame_score_17.mat  frame_score_42.mat  frame_score_68.mat  frame_score_93.mat  images_118.mat      images_28.mat       images_53.mat       images_79.mat
frame_passed_122.mat    frame_passed_32.mat frame_passed_58.mat frame_passed_83.mat frame_score_108.mat frame_score_18.mat  frame_score_43.mat  frame_score_69.mat  frame_score_94.mat  images_119.mat      images_29.mat       images_54.mat       images_8.mat
frame_passed_123.mat    frame_passed_33.mat frame_passed_59.mat frame_passed_84.mat frame_score_109.mat frame_score_19.mat  frame_score_44.mat  frame_score_7.mat   frame_score_95.mat  images_12.mat       images_3.mat        images_55.mat       images_80.mat

[asafgal]

Try to run this code in a python console:

import os

imageddir = <path to session/images>

imagefiles = [f for f in os.listdir(imagedir) if isfile(os.path.join(imagedir, f))]
imagefiles = [f for f in imagefiles if 'images' in f]
for x in imagefiles:
    try:
        m = int(x.rstrip('.mat').split('_')[1])
    except:
        print('failed on file ' + x)

[Giacomoalciatore]

I noticed that I could run the classification locally on my computer, so I rsynced again the session folder antrax fro my computer to the cluster and after that it worked. So I'm not sure what was the issue, but at least is solved! thanks