Memory Error

[mmcfarland]

Staging has reported a memory error for the following request, however the AoI size is small, indicating that the memory was not consumed by this single request, but that this was the one which pushed over the limit. It occurs at the focal operation. Would be interesting to correlate some resource metrics with an execution.

Stack trace

5. File "/usr/local/lib/python2.7/dist-packages/pollinator-0.1.0-py2.7.egg/pollinator/crop_yield.py" line 114 in calc_abundance [args] [locals] function=focal_op)
6. File "/usr/local/lib/python2.7/dist-packages/scipy/ndimage/filters.py" line 1245 in generic_filter [args] [locals] cval, origins, extra_arguments, extra_keywords)

Model input

{
                "inputs": [
                  {
                    "name": "numberofhives", 
                    "area": "0", 
                    "effectiveUnits": null, 
                    "value": 0.2, 
                    "summary": "", 
                    "shape": null, 
                    "effectiveShape": null, 
                    "units": "acres", 
                    "effectiveArea": null, 
                    "type": ""
                  }
                ], 
                "modification_hash": "7e687e983de7b2ac5588277ee6982901", 
                "aoi_census": null, 
                "modification_censuses": null, 
                "modification_pieces": [
                  {
                    "category": "landcover", 
                    "shape": {
                      "geometry": {
                        "type": "Polygon", 
                        "coordinates": [
                          [
                            [
                              -76.30558490753172, 
                              39.97940153174264
                            ], 
                            [
                              -76.30492508411407, 
                              39.977683270137085
                            ], 
                            [
                              -76.30194783210754, 
                              39.97832865097396
                            ], 
                            [
                              -76.30082130432129, 
                              39.97989480460415
                            ], 
                            [
                              -76.30111634731293, 
                              39.98033874713501
                            ], 
                            [
                              -76.30558490753172, 
                              39.97940153174264
                            ]
                          ]
                        ]
                      }, 
                      "type": "Feature", 
                      "properties": {}
                    }, 
                    "value": 29, 
                    "area": 17.254255134113887
                  }
                ], 
                "area_of_interest": {
                  "type": "MultiPolygon", 
                  "coordinates": [
                    [
                      [
                        [
                          -76.3055956363678, 
                          39.979417974228724
                        ], 
                        [
                          -76.30110561847687, 
                          39.9803675210884
                        ], 
                        [
                          -76.30078375339508, 
                          39.979870141045645
                        ], 
                        [
                          -76.30193173885345, 
                          39.978308097538076
                        ], 
                        [
                          -76.30495190620422, 
                          39.97767093795983
                        ], 
                        [
                          -76.3055956363678, 
                          39.979417974228724
                        ]
                      ]
                    ]
                  ]
                }, 
                "inputmod_hash": "c6a18cd51152cb8fc8fdb0937aecbbe67e687e983de7b2ac5588277ee6982901"
              }

https://rollbar.com/ProjectICP/Bee-Pollinator/items/2/

[mmcfarland]

Free memory graphed for the particular worker instance that reported the error. Not correlated with number of model executions happening, but from the consistent pattern, it's possible that each spike is a single execution - which would be a substantial allocation. Caring about intermediate raster data types may be an important optimization. The good news is that nothing appears to hold onto memory for long and that it does not leak.