Open arjclark opened 1 year ago
As mentioned offline, this seems to be due to the way esmpy handles arrays and I'm not sure how possible it would be to solve this directly so I think this could be solved with a work around. If it is only the source data which is too large, it ought to be possible to create a series of regridders from slices of the large source which are each small enough to fit in memory. once regridding is done, the result can be concatenated back together. If this can be demonstrated as a viable solution, it should be possible to refine this process to be something that iris-esmf-regrid can handle internally.
@SciTools-incubator/esmf-regrid-devs This issue is stale due to a lack of activity in the last 90 days. Remove stale label or comment, otherwise this issue will close automatically in 7 days time.
Still relevant and proof of concept provided by @stephenworsley seems to be working in getting round this issue.
@SciTools-incubator/esmf-regrid-devs This issue is stale due to a lack of activity in the last 180 days. Remove stale label or comment, otherwise this issue will close automatically in 14 days time.
@stephenworsley - any chance this is going to get worked on any time soon?
@arjclark We're hoping to work on this as part of a focus on performance as planned in the roadmap here #278. The next release (0.10) ought to finish the work on regridder unification so that this should be included in work on 0.11 which ought to be starting up around March. If things go well there should be a release by summer, though that depends on how successful attempts at performance improvements are so I can't guarantee when these improvements will be released
@SciTools-incubator/esmf-regrid-devs This issue is stale due to a lack of activity in the last 180 days. Remove stale label or comment, otherwise this issue will close automatically in 14 days time.
@stephenworsley - where are we at with this?
This is lined up to be the next bit of big work we do on iris-esmf-regrid. The timeline on this depends on how big this turns out to be, but I'd hope to start looking into this next month.
🐛 Bug Report
@stephenworsley - When working with large source data and trying to generate a regridder am hitting
TypeError: buffer is too small for requested array
How To Reproduce
Steps to reproduce the behaviour:
Expected behaviour
regridder to be generated
Environment
esmf_regrid
version: 0.5.0Additional context