Insufficient Memory - re_classify function

[frankross5]

Describe the bug The re_classify function is called several times in the Suitability_Analysis.ipynb, and it works for:

• degradation
• precip
• evapotranspiration
• max_temp
• ph
• carbon
• elev
• slope
• aspect

However, when having to deal with landcover I encounter the error:

{
    "name": "MemoryError",
    "message": "Unable to allocate 48.2 GiB for an array with shape (1, 48871, 132376) and data type float64",
    "stack": "---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
c:\\Users\\Francesco\\Desktop\\progreen_suitability\
otebooks\\Suitability_Analysis.ipynb Cell 19 line 1
      <a href='vscode-notebook-cell:/c%3A/Users/Francesco/Desktop/progreen_suitability/notebooks/Suitability_Analysis.ipynb#X24sZmlsZQ%3D%3D?line=0'>1</a> landcover = {'min_val':None,
      <a href='vscode-notebook-cell:/c%3A/Users/Francesco/Desktop/progreen_suitability/notebooks/Suitability_Analysis.ipynb#X24sZmlsZQ%3D%3D?line=1'>2</a>              'max_val':None,
      <a href='vscode-notebook-cell:/c%3A/Users/Francesco/Desktop/progreen_suitability/notebooks/Suitability_Analysis.ipynb#X24sZmlsZQ%3D%3D?line=2'>3</a>              'map_vals':{10:50,  # Trees
   (...)
     <a href='vscode-notebook-cell:/c%3A/Users/Francesco/Desktop/progreen_suitability/notebooks/Suitability_Analysis.ipynb#X24sZmlsZQ%3D%3D?line=13'>14</a>                         },
     <a href='vscode-notebook-cell:/c%3A/Users/Francesco/Desktop/progreen_suitability/notebooks/Suitability_Analysis.ipynb#X24sZmlsZQ%3D%3D?line=14'>15</a>             'classifier': 'convert'}
---> <a href='vscode-notebook-cell:/c%3A/Users/Francesco/Desktop/progreen_suitability/notebooks/Suitability_Analysis.ipynb#X24sZmlsZQ%3D%3D?line=17'>18</a> lc_d, lc_profile = clip_and_classify(landcover_vrt, inB, landcover)

File c:\\Users\\Francesco\\Desktop\\progreen_suitability\
otebooks\\../src\\pro_green.py:149, in clip_and_classify(global_file, bounds, defs)
    147 inR = rasterio.open(global_file)
    148 local_d, profile = rMisc.clipRaster(inR, bounds, crop=False)
--> 149 local_d = re_classify(local_d, defs, profile['nodata'])
    150 return([local_d, profile])

File c:\\Users\\Francesco\\Desktop\\progreen_suitability\
otebooks\\../src\\pro_green.py:69, in re_classify(array, defs, no_data)
     62 def re_classify(array, defs, no_data):
     63     ''' re-classify array based on categories in defs
     64     
     65     :param array: numpy array of desired data
     66     :param defs: dictionary defining re-classification
     67     :param no_data: value to set no_data
     68     '''
---> 69     mask = np.zeros(array.shape) + 1
     70     if defs['max_val']: max_val = defs['max_val']
     71     else: max_val = array.max()

MemoryError: Unable to allocate 48.2 GiB for an array with shape (1, 48871, 132376) and data type float64"
}

Local RAM I am working on a workstation with 32gb RAM.

Possible Solutions I have tried to make the data type float32, and I get the following suitability map (1):

which is substantially different from the suitability map (2) already within progreen_suitability\docs\images:

Double Check Can you please re-run the code present in git and make sure that you get a suitability map (2)?

[bpstewar]

I think the map you show with much less suitability is correct using all the layers now included in the analysis; the example in the repo was earlier in the project when we had a smaller number of input datasets