ilarinieminen / SOM-Toolbox

GNU General Public License v2.0
119 stars 56 forks source link

How to handling lots of columns of input data for som training #21

Open JimmyGao0204 opened 7 years ago

JimmyGao0204 commented 7 years ago

Recently, I want to construct a som model using somtoolbox in matlab. I have 350 SST (sea surface temperature) images. Each image is 400 pixels by 480 pixels. And each SST image was transformed into a single row vector by concatenating image rows. So that the input matrix consisted of 350 rows (image) and 192000 columns (pixels). But when I run the following code,some errors occured. Could you please tell me how to solve this problem? Thanks for your consideration.

`clc;clear; %prepare your data (satellite remote sensing data) myFiles=dir('D:/MatlabTemp/SSC/.tif'); fileLength=length(myFiles); myMatrix=[]; firstRaster=imread('D:/MatlabTemp/SSC/m20030530.tif'); [m,n]=size(firstRaster); for i=1:(fileLength) filename=myFiles(i).name; path='D:\MatlabTemp\SSC'; imgPath=strcat(path,'\',filename); raster=imread(imgPath); vector=reshape(raster,1,mn); myMatrix=[myMatrix;vector]; end

%TSM data n * m (n samples(image)) m (pixels) myMatrix(isnan(myMatrix)) = -1;

%inialize parameters of som algorithm msize = [3 4]; init = 'lin'; train = 'batch'; neigh = 'ep'; rad = [5 1];

y = myMatrix;

%construct the input data used for som algorithm sD = som_data_struct(y);

if strcmp(init,'rand') sM = som_randinit(sD,'msize',msize); elseif strcmp(init,'lin') sM = som_lininit(sD,'msize',msize,'lattice','rect','shape','sheet'); end %train som algorithm if strcmp(train,'batch') sM = som_batchtrain(sM, sD,'radius',rad,'trainlen',1000,'alpha_type','inv','neigh',neigh); elseif strcmp(train,'seq') sM = som_seqtrain(sM, sD,'radius',rad,'trainlen',1000,'neigh',neigh); elseif strcmp(train,'sompak') sM = som_sompaktrain(sM,sD,'radius',rad,'trainlen',1000,'neigh',neigh,'alpha_type','inv'); end`

Error using zeros

Requested 192000×192000 (274.7GB)array exceeds maximum array preference.Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.

Error in som_liminit(line 222) A = zeros(dim);