YttriLab / A-SOID

An active learning platform for expert-guided, data efficient discovery of behavior.
Other
55 stars 7 forks source link

error running active learning classify on CaIMS21 dataset: "setting array element with a sequence" #47

Closed vickerse1 closed 1 year ago

vickerse1 commented 1 year ago

Hi Jens and Viola,

Everything on this step (active learning, classify) appears to finish (i.e. it goes through a bunch of training iterations successfully), then it fails to save or something like this. This is with the Jul10 2023 github package running on 64 bit Windows 10, both conda and pip install . steps run. Here is the command line error:


2023-07-10 09:59:47.792 Uncaught app exception Traceback (most recent call last): File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\legacy_caching\caching.py", line 539, in get_or_create_cached_value return_value = _read_from_cache( File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\legacy_caching\caching.py", line 338, in _read_from_cache raise e File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\legacy_caching\caching.py", line 323, in _read_from_cache return _read_from_mem_cache( File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\legacy_caching\caching.py", line 241, in _read_from_mem_cache raise CacheKeyNotFoundError("Key not found in mem cache") streamlit.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\apps\C_auto_active_learning.py", line 98, in main load_all_train(working_dir, prefix) File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\legacy_caching\caching.py", line 573, in wrapped_func return get_or_create_cached_value() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\legacy_caching\caching.py", line 557, in get_or_create_cached_value return_value = func(*args, **kwargs) File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\load_workspace.py", line 84, in load_all_train data = _load_sav(path, name, 'all_train.sav') File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\load_workspace.py", line 9, in _load_sav with open(os.path.join(path, name, filename), 'rb') as fr: FileNotFoundError: [Errno 2] No such file or directory: 'J:\ASOID_M2_2\asoid_output\Jul-10-2023\all_train.sav'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\scriptrunner\script_runner.py", line 557, in _run_script exec(code, module.dict) File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\app.py", line 332, in main() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\app.py", line 328, in main application_function() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\apps\C_auto_active_learning.py", line 128, in main rf_classifier.main() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 550, in main self.self_learn() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 445, in self_learn self.show_training_performance(it + 1) File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 466, in show_training_performance np.hstack([100 round(np.mean(np.mean(self.iterX_f1_scores_list[j], axis=0), File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 466, in np.hstack([100 round(np.mean(np.mean(self.iterX_f1_scores_list[j], axis=0), File "<__array_function__ internals>", line 200, in mean File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\numpy\core\fromnumeric.py", line 3464, in mean return _methods._mean(a, axis=axis, dtype=dtype, File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\numpy\core_methods.py", line 165, in _mean arr = asanyarray(a) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (10,) + inhomogeneous part.

And, here is the error from the ASOID streamlit GUI running in Chrome browser:


ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (10,) + inhomogeneous part. Traceback: File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\streamlit\scriptrunner\script_runner.py", line 557, in _run_script exec(code, module.dict) File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\app.py", line 332, in main() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\app.py", line 328, in main application_function() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\apps\C_auto_active_learning.py", line 128, in main rf_classifier.main() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 550, in main self.self_learn() File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 445, in self_learn self.show_training_performance(it + 1) File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 466, in show_training_performance np.hstack([100 round(np.mean(np.mean(self.iterX_f1_scores_list[j], axis=0), File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\auto_active_learning.py", line 466, in np.hstack([100 round(np.mean(np.mean(self.iterX_f1_scores_list[j], axis=0), File "<__array_function__ internals>", line 200, in mean File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\numpy\core\fromnumeric.py", line 3464, in mean return _methods._mean(a, axis=axis, dtype=dtype, File "C:\ProgramData\Anaconda3\envs\asoid-test\lib\site-packages\numpy\core_methods.py", line 165, in _mean arr = asanyarray(a)

Thanks,

Evan

JensBlack commented 1 year ago

Hi Evan,

Viola was a typo in my mail. I meant voilá, as in "here it is", but I am glad that the previous issue has been solved-ish.

Can you elaborate on the parameters you set during active learning?

I've recreated both the paper parameters (n_iter = 20, samples_per_iter = 200) and the current default parameters (n_iter = 100, samples_per_iter = 20). Both work fine, so your info will help me to recreate the error.

The issue (although similar to the other numpy issues you encountered) seems to be a result of something going wrong during the active learning steps. I am looking forward to your answer!

vickerse1 commented 1 year ago

Hi Jens,

Yes, everything works now for me - you can close all of these issues.

Next, on to learning Boris for me.....

Thanks,

Evan


From: Jens Tillmann @.> Sent: Tuesday, July 11, 2023 2:58 AM To: YttriLab/A-SOID @.> Cc: Evan Vickers @.>; Author @.> Subject: Re: [YttriLab/A-SOID] error running active learning classify on CaIMS21 dataset: "setting array element with a sequence" (Issue #47)

Hi Evan,

Viola was a typo in my mail. I meant voilá, as in "here it is", but I am glad that the previous issue has been solved-ish.

Can you elaborate on the parameters you set during active learning?

I've recreated both the paper parameters (n_iter = 20, samples_per_iter = 200) and the current default parameters (n_iter = 100, samples_per_iter = 20). Both work fine, so your info will help me to recreate the error.

The issue (although similar to the other numpy issues you encountered) seems to be a result of something going wrong during the active learning steps. I am looking forward to your answer!

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/YttriLab/A-SOID/issues/47*issuecomment-1630529437__;Iw!!C5qS4YX3!AkjOf7ksgi5AZr6ps6WGKb_CTHQ9M6ZZOEy4cq7XKbdcDpZylu3FAArUuB5n82-vyU1sHqYzd5z5V5SFybxde0g2Wg_b$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AJGCDZTJEENSGJEFOJSSDZLXPUPVJANCNFSM6AAAAAA2E2ZCNA__;!!C5qS4YX3!AkjOf7ksgi5AZr6ps6WGKb_CTHQ9M6ZZOEy4cq7XKbdcDpZylu3FAArUuB5n82-vyU1sHqYzd5z5V5SFybxde9onDkiX$. You are receiving this because you authored the thread.Message ID: @.***>

JensBlack commented 1 year ago

Great to hear! Please feel free to reach out if you are having any other issues. We are, of course, also to hear about your successes!