japalmer29
commented
5 years ago Hi,
I think the problem is your data only has 768 time points, and the likelihood is infinite. Data should have 10s of thousands of time points, particularly when the data is being portioned out over each node in a cluster.
Jason
From: moo-n notifications@github.com Sent: Wednesday, July 31, 2019 12:18 PM To: japalmer29/amica amica@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [japalmer29/amica] AMICA on cluster machine (#14)
Dear Dr. Jason Palmer, Thank you for checking the issues we have with amica. I'm trying to make AMICA work on a cluster machine. When I tried with the data set that comes with EEGLAB, amica seems to iterate just fine but when I try with the real data, I get only 7 iterations and then it stops.
this is the error message I got from the real data:
run runAmica
pop_loadset(): loading file /home/mohamed/bmaold/subma.set ...
Warning: the name of the dataset has changed on disk, updating EEG structure accordingly
Reading float file '/home/mohamed/bmaold/subma.fdt'...
Scaling components to RMS microvolt
Writing data file: /home/mohamed/bmaold/tmpdata81472.fdt
1 processor name = falcon
1 host_num = 1281896239
This is MPI process 1 of 1 ; I am process 1 of 1 on node: falcon
1 : node root process 1 of 1
Processing arguments ...
num_files = 1
FILES:
/home/mohamed/bmaold/tmpdata81472.fdt
num_dir_files = 1
initial matrix block_size = 128
do_opt_block = 0
blk_min = 256
blk_step = 256
blk_max = 1024
number of models = 1
max_thrds = 32
use_min_dll = 1
min dll = 1.00000000000000006E-9
use_grad_norm = 1
min grad norm = 9.99999999999999955E-8
number of density mixture components = 3
pdf type = 0
max_iter = 2000
num_samples = 1
data_dim = 128
field_dim = 768
do_history = 0
histstep = 10
share_comps = 0
share_start = 100
comp_thresh = 0.98999999999999999
share_int = 100
initial lrate = 5.00000000000000028E-2
minimum lrate = 1.00000000000000002E-8
minimum data covariance eigenvalue = 9.9999999999999998E-13
lrate factor = 0.5
initial rholrate = 5.00000000000000028E-2
rho0 = 1.5
min rho = 1.
max rho = 2.
rho lrate factor = 0.5
kurt_start = 3
num kurt = 5
kurt interval = 1
do_newton = 1
newt_start = 50
newt_ramp = 10
initial newton lrate = 1.
do_reject = 0
num reject = 3
reject sigma = 3.
reject start = 2
reject interval = 3
write step = 20
write_nd = 0
write_LLt = 1
dec window = 1
max_decs = 3
fix_init = 0
update_A = 1
update_c = 1
update_gm = 1
update_alpha = 1
update_mu = 1
update_beta = 1
invsigmax = 100.
invsigmin = 0.E+0
do_rho = 1
load_rej = 0
load_c = 0
load_gm = 0
load_alpha = 0
load_mu = 0
load_beta = 0
load_rho = 0
load_comp_list = 0
do_mean = 1
do_sphere = 1
doPCA = 1
pcakeep = 128
pcadb = 30.
byte_size = 4
doscaling = 1
scalestep = 1
mkdir: cannot create directory ‘/home/mohamed/bmaold/amicaouttmp/’: File exists
output directory = /home/mohamed/bmaold/amicaouttmp/
1 : setting num_thrds to 32 ...
1 : using 32 threads.
1 : node_thrds = 32
bytes in real = 4
1 : REAL nbyte = 4
getting segment list ...
blocks in sample = 768
total blocks = 768
node blocks = 768
node 1 start: file 1 sample 1 index 1
node 1 stop : file 1 sample 1 index 768
1 : data = 42.12274169921875 -22.660226821899414
getting the mean ...
mean = 19.213357688044198 -3.6652333481858173 11.912760259234346
subtracting the mean ...
getting the sphering matrix ...
cnt = 768
doing eig nx = 128 lwork = 163840
minimum eigenvalues = 4.76517915723355459E-2 5.44860527342523465E-2 6.17080211329246175E-2
maximum eigenvalues = 3456.1695873535355 1986.8285709131624 527.06571703521831
num eigs kept = 128
sphering the data ...
numeigs = 128
1 Allocating variables ...
1 : Initializing variables ...
1 : block size = 128
1 : entering the main loop ...
iter 1 lrate = 0.0500000000 LL = 0.0000000000 nd = NaN, D = 0.68564E-01 0.68564E-01 ( 0.00 s, 0.0 h)
iter 2 lrate = 0.0500000000 LL = 0.0000000000 nd = NaN, D = NaN NaN ( 0.00 s, 0.0 h)
iter 3 lrate = 0.0500000000 LL = 0.0000000000 nd = NaN, D = NaN NaN ( 0.00 s, 0.0 h)
iter 4 lrate = 0.0500000000 LL = 0.0000000000 nd = NaN, D = NaN NaN ( 0.00 s, 0.0 h)
iter 5 lrate = 0.0500000000 LL = 0.0000000000 nd = NaN, D = NaN NaN ( 0.00 s, 0.0 h)
iter 6 lrate = 0.0500000000 LL = 0.0000000000 nd = NaN, D = NaN NaN ( 0.00 s, 0.0 h)
iter 7 lrate = 0.0500000000 LL = 0.0000000000 nd = NaN, D = NaN NaN ( 0.00 s, 0.0 h)
Exiting because likelihood increasing by less than 1.00000000000000006E-9 for more than 5 iterations ...
... done. Execution time: 0.00 h
output directory = /home/mohamed/bmaold/amicaouttmp/
Something went wrong...
No gm present, setting num_models to 1
No W present, exiting
Reference to non-existent field 'W'.
Error in runamica15 (line 892)
weights = mods.W(:,:,1);Error in runAmica (line 16)
[weights,sphere,mods] = runamica15(EEG.data, 'num_models',num_models, 'outdir',outdir, ...Error in run (line 63)
evalin('caller', [script ';']);
Best Regards,
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/japalmer29/amica/issues/14?email_source=notifications&email_token=ACRBESU3OLFAPLFDZPBRS33QCD76JA5CNFSM4IICCHZ2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HCOT5EA , or mute the thread https://github.com/notifications/unsubscribe-auth/ACRBESULVAZ2WKK6XNXXU4TQCD76JANCNFSM4IICCHZQ . https://github.com/notifications/beacon/ACRBESQXTDMDN2M53N3JZHDQCD76JA5CNFSM4IICCHZ2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HCOT5EA.gif
moo-n
pknytl
Dear Dr. Jason Palmer, Thank you for checking the issues we have with amica. I'm trying to make AMICA work on a cluster machine. When I tried with the data set that comes with EEGLAB, amica seems to iterate just fine but when I try with the real data, I get only 7 iterations and then it stops.
this is the error message I got from the real data:
Best Regards,