search

pwolfram / MPAS-Model

Repository for MPAS models and shared framework releases.
Other
1 stars 1 forks source link

pyAMG coarsening improperly applied in 3 dimensions #20

Open bradyrx opened 5 years ago

bradyrx commented 5 years ago

@pwolfram ,

Currently pyAMG coarsens the initial mesh in 3 dimensions, whereas it should probably only coarsen in 2D and then apply this filter to all vertical layers. I.e., we should only use pyAMG to coarsen the horizontal so that the vertical seeding option remains valid in the vertical.

For instance, here's the linear seeding case for an arbitrary grid cell in the 60to30km grid:

And after applying pyAMG:

The odd thing is that it still seeds the vertical column with 20 particles, but they don't at all follow the seeding strategy (e.g. linear or log).

bradyrx commented 5 years ago

As the code currently stands, it downsamples first (at the stage when it creates particles on cell centers), then adds passive floats. Perhaps these steps need to be flipped? However, I don't see why the seeding would get so messed up in the current format. It should be able to coarsen horizontal cell centers, and then apply the vertical weights when seeding.

bradyrx commented 5 years ago

@pwolfram , here's a simple first solution attempt: https://gist.github.com/bradyrx/ddb6c944e425ac5e63b8e8e7361809dd

This accepts a stride argument and simply strides through the lat and lon list removing columns. This doesn't produce a uniform coarsening, but does retain full vertical columns. Perhaps it could be improved by randomly selecting a subset of the lat and lon list, rather than striding through the ordered list. Current runtime is ~1 minute for 60to30.

pwolfram commented 5 years ago

@bradyrx, there is an obvious bug here if the coarsening is in 3D, but as I looked back through the code I don't think the coarsening is in 3D. It should have only been in 2D; but if it is in 3D this explains the extreme slowdown. I'm currently in progress on the horizontal interpolation and am going to focus on that now but agree this needs some revisiting. It isn't obvious why this is the case-- the downsampling looks like it should only be applied in 2D: https://github.com/pwolfram/MPAS-Model/blob/ea058b8c4c64f54079a56f8ea0e6fd0924447775/testing_and_setup/compass/utility_scripts/LIGHTparticles/make_particle_file.py#L315 You are just doing the building step (no remapping), correct?

bradyrx commented 5 years ago

but if it is in 3D this explains the extreme slowdown

Yes, this would be encouraging in terms of the runtime (https://github.com/pwolfram/MPAS-Model/issues/19).

You're right on what it should be doing. I didn't notice that this was being called just from the cell_centers stage.

You are just doing the building step (no remapping), correct?

Correct. And there's no reason why the remapping should be called with the way I've been calling the script.

Example usage for our 30to10 case:

./make_particle_file.py -i init_file.nc -g graph_file -p 9600 -t 'surface','passive' --nvertlevels 15 --vertseedtype 'log' --spatialfilter SouthernOceanXYZ -d 1 -o particles.nc