storm climatology

[Aebestach]

Hello, @alphaparrot I am currently running a model for the Kerbin planet in Kerbal Space Program. I added some information about storm climatology to the configuration. I would like to ask if it will generate cyclones during the simulation. If it can generate cyclones (such as extratropical cyclones, tropical cyclones, etc.), where should I view these data?

import exoplasim as exo
Kerbin = exo.Model(workdir="Kerbin_run",modelname="Kerbin",inityear=0,outputtype=".nc",ncpus=32,precision=8,resolution="T42",layers=10)
Kerbin.configure(startemp=5840.0,flux=1360.0,
                  year=426.0,eccentricity=0.0,obliquity=0.0,lonvernaleq=0.0,fixedorbit=True,
                  rotationperiod=0.25,
                  gravity=9.81,radius=1.0,
                  vegetation=2,vegaccel=1,nforestgrowth=1.0,initgrowth=0.5,initstomcond=1.0,initrough=2.0,initsoilcarbon=0.0,initplantcarbon=0.0,
                  wetsoil=False,seaice=True,oceanzenith="ECHAM-3",
                  landmap="SRA/Kerbin_surf_0172.sra",topomap="SRA/Kerbin_surf_0129.sra",
                  pressure=1.013,drycore=False,ozone=True,
                  timestep=15.0,runsteps=11520,snapshots=1440,
                  stormcapture={'toggle': 1,'NKTRIGGER': 1})
Kerbin.exportcfg()
#Kerbin.run(years=10,crashifbroken=True)
Kerbin.runtobalance(threshold=0.005,baseline=10,maxyears=200,minyears=10,crashifbroken=True)
Kerbin.finalize("Kerbin",allyears=False,keeprestarts=False)
Kerbin.save()

[Aebestach]

I noticed several parameters such as cape, lnb, sdef, absz, umax, vent, vrumax, gpi, but after exporting their tables, I found that they were 0, -1 and 1

[alphaparrot]

Hi, tropical cyclone and hurricane formation is mostly only expected at extremely high resolutions, i.e. maybe T85 but realistically T170 is probably needed. Actually simulating cyclones realistically, with correct physics, requires both even higher resolution than T170 and a different model with better convective cloud microphysics. Be aware that at these resolutions, a very short timestep is needed and it will likely take at least several days to compute even a single year on consumer hardware. There is sadly a reason hurricane forecasting is done by governments, rather than individuals.

To compute and output storm climatology, you need to have stormclim=True. This will roughly double the time it takes to run the model (storm climatology is very expensive to compute). The stormcapture dictionary controls when high-cadence output is triggered—the model will try to only write high-cadence output when there is or is likely to be a storm. This is necessary because the normal output cadence is far too long to see individual storms and weather systems; only their impacts on climate. But if we wrote high-cadence output for the whole model, even at lower resolutions it would consume enormous amounts of disk space. Expect large files even with the current on-trigger high cadence paradigm. More frequent writes to disk also slows down the model further.

It's possible to productively use storm climatology at lower resolutions, but mostly as climate variables, not weather variables. The Komacek, et al paper that forms the basis for some of these features, for example, looked to see if hurricanes might form on tidally-locked planets, and if so, where. The way to do that is time-averaged quantities like the Genesis Potential Index, which can indicate the likelihood of cyclogenesis in a given region at a given time, even without directly simulating a hurricane.

On Tue, Jun 18, 2024, 9:12 AM Aebestach @.***> wrote:

I noticed several parameters such as cape, lnb, sdef, absz, umax, vent, vrumax, gpi, but after exporting their tables, I found that they were 0, -1 and 1

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2176071196, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQERT7VFWVR7NTU3VYDZIAW45AVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZWGA3TCMJZGY . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

OK, thanks, but I saw in Panoply that a structure similar to a cold vortex will appear. Another question is, can ExoPlaSim simulate gaseous planets?

[alphaparrot]

ExoPlaSim can and will form vortices (especially cold vortices on the night side of tidally locked planets), and it will do better than Eulerian models at the same resolution because the spectral method it uses does a better job conserving subscale vorticity, but vortex formation is quite different from a cyclone, i.e. a storm. Vortices are a consequence of fluid dynamics; cyclones are a consequence of moist physics, and are characterized by much higher vorticity and a degree of self-sustenance—you can think of a storm, especially a powerful one like a hurricane, as a heat engine plowing through the atmosphere. It will resist dissipation until it runs out of fuel, and actively change the airflow around it. Vortices do not necessarily or even usually do that, and if you examine the ones you see in the output you'll see some of them are dry and/or not associated with significant cloud and precipitation activity. Modeling a cyclone, as a self-sustaining energetic storm, requires horizontal resolution around 100km or less. I sometimes saw cyclogenesis in earthlike models at T170, but not reliably.

ExoPlaSim won't necessarily crash if you give it a low mean molecular weight, high surface pressure, and high gravity, but it will definitely get the wrong answer. Kristen Menou once built and used a modified version of the underlying PlaSim model to do hot jupiters, but that involved replacing the fluid dynamics with equations suited to gas giants.

ExoPlaSim suffers from two problems when it comes to gaseous planets:

1. If there is an underlying ocean or significant water content in the atmosphere, the fact that the bulk atmosphere is lighter than water means humidity stops convection, rather than starting it. ExoPlaSim does not (yet) represent that in its physics.

2. Atmospheric fluid dynamics is dominated by the propagation of waves, and there are various kinds of waves that can be important. Models like ExoPlaSim use a set of dynamical equations called "shallow water equations", so called because they do a good job in the limit where the fluid being modelled is very shallow relative to the waves that might form. The dominant waves propagate horizontally, not vertically. If the atmosphere becomes tall enough, suddenly waves that couldn't really form or propagate vertically can do so, and even have multiple wavelengths before reflections. Modelling that case correctly requires that the model solve the 3D Navier-Stokes equation. Gas giants, mini-Neptunes, and potentially even H2-dominated super-Earths all fall in the latter category and can't be modelled correctly by ExoPlaSim. ExoPlaSim is really best-suited to temperate terrestrial planets with at least moderately Earth-like atmospheric compositions.

On Tue, Jun 18, 2024, 9:49 AM Aebestach @.***> wrote:

OK, thanks, but I saw in Panoply that a structure similar to a cold vortex will appear. Another question is, can ExoPlaSim simulate gaseous planets?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2176154127, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQAXAK3UDICWZMJP4WLZIA3GVAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZWGE2TIMJSG4 . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

Okay, I see, thank you very much. Are those indicators of storms also indicators of convective activity, such as CAPE? For example, does this mean the possibility of a storm or the possibility of convection?

[alphaparrot]

Yes and sort of, CAPE is convective available potential energy. It's not a dead indicator of storm activity, but convective potential and storms go hand in hand, at least for strong convective systems in the summer. Winter storms are a bit different. If you want to measure convection directly, I recommend looking to variables like 'wa' and 'wap', which measure vertical velocity (in m/s or Pa/s). You don't need storm climatology to access those. Divergence ('d') and vorticity ('zeta') may also be informative (e.g. negative divergence, or convergence, at lower levels at positive divergence aloft would tend to indicate upwelling, and you could determine the upwelling mass flux from the upward wind speed).

On Tue, Jun 18, 2024, 11:40 AM Aebestach @.***> wrote:

Okay, I see, thank you very much. Are those indicators of storms also indicators of convective activity, such as CAPE?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2176412095, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQGR7AJZOPEKDFO5INLZIBIHRAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZWGQYTEMBZGU . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

But the vertical wind simulated by ExoPlaSim seems to be very small, even always 0. Does it really simulate vertical wind?

[alphaparrot]

Convection is slow, on average! Note that the units are meters per second, for 'wa'. It doesn't take much to power convective systems or keep an eagle aloft. Also note that these quantities represent the mean inside the grid cell, and even at T42 these cells are large—at the equator, a grid cell is around 312km wide. In real life, you would likely encounter thermals at the sub-km scale (even sub-100m) that would reach much higher vertical velocities, but you would also find stabler air nearby as well as localized downwelling. This is part of why very high resolution is needed to resolve convective storms and cyclones, which are driven by localized very strong convection.

On Tue, Jun 18, 2024, 12:03 PM Aebestach @.***> wrote:

But the vertical wind simulated by ExoPlaSim seems to be very small, even always 0. Does it really simulate vertical wind?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2176463787, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQFOGZ4JGN6AIQWDRKDZIBK5JAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZWGQ3DGNZYG4 . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

Then if I want more accurate numbers, I still need the T170 resolution to simulate. I would like to ask if you have used ROCKE-3D before, and can it run on a home computer compared to ExoPlaSim?

[alphaparrot]

I have not used ROCKE-3D before, but I know the developers. You can technically run it on a home computer, just like you can technically run ExoPlaSim at much higher than T170 resolution on a home computer. It is much more difficult to use however with a lot more care needed to tune it properly so it doesn't crash, and it runs on a different timescale. Simulations with GCMs in that class take weeks to months, not hours to days, even on advanced hardware. However, in return you get far more accurate physics that can handle more extreme scenarios. ExoPlaSim is benchmarked and validated against models like ROCKE-3D (it was in fact one of the models I used in my validation). ExoPlaSim is close enough for the scenarios it's suitable for, which makes it useful for a wide range of science on its own and as a pathfinder model for large experiments that will use models like ROCKE-3D, LMD, the UK Met model, and so on for the robust, precise scientific predictions.

For example, one might use ExoPlaSim to identify the parts of planetary parameter space where hurricane cyclogenesis is most likely to frequently occur, and then run the bigger, more advanced models only for those planets, to verify that they do form and characterize how they differ from storms on Earth, without spending huge computing resources on models that won't be very interesting.

On Tue, Jun 18, 2024, 12:31 PM Aebestach @.***> wrote:

Then if I want more accurate numbers, I still need the T170 resolution to simulate. I would like to ask if you have used ROCKE-3D before, and can it run on a home computer compared to ExoPlaSim?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2176517141, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQAAZUKNHIT3QLYJ57DZIBOEDAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZWGUYTOMJUGE . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

I'd better give up ROCKE3D. In fact, its installation is very troublesome and time-consuming. In comparison, ExoPlaSim is more friendly and the overall calculation cost is also low. I want to report a strange thing. In ExoPlaSim 3.3.0, I can only run the above code with 2 CPUs. But in 3.2.4, I can use 32, which is very strange. I installed Ubuntu using WSL. Oh, by the way, when pip3 install ExoPlaSim, it is still version 3.3.0. When I specify version 3.4.0, it tells me that there is no such version.

[alphaparrot]

Check which python version pip3 uses. ExoPlaSim 3.4.0 drops support for some older versions of python, because it now relies on NumPy behaviour that doesn't exist in the last available NumPy version for older versions of Python. I recommend Python 3.9 for users who don't want to switch yet to some of the possibly-breaking changes in 3.10+ but need a more modern python installation.

Give 3.4 a try with a more modern python version and see if you still run into problems with the number of CPUs.

On Tue, Jun 18, 2024, 12:49 PM Aebestach @.***> wrote:

I'd better give up ROCKE3D. In fact, its installation is very troublesome and time-consuming. In comparison, ExoPlaSim is more friendly and the overall calculation cost is also low. I want to report a strange thing. In ExoPlaSim 3.3.0, I can only run the above code with 2 CPUs. But in 3.2.4, I can use 32, which is very strange. I installed Ubuntu using WSL. Oh, by the way, when pip3 install ExoPlaSim, it is still version 3.3.0. When I specify version 3.4.0, it tells me that there is no such version.

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2176550220, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQE3PEYPEMU7GTMQGS3ZIBQJLAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZWGU2TAMRSGA . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

That might be the problem, I'm still using Python 3.6, I'll update to 3.9 during the day to give it a try.Thank you again

[Aebestach]

No module named 'exoplasim.pyfft'
No module named 'exoplasim.pyfft'
Error writing output to MOST.00000.nc; log written to postprocess.log
Going to stop here just in case......
mkdir: cannot create directory ‘/home/aebestach/Code/Tekto2_crashed’: File exists
ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/Code/Tekto2_crashed/
mkdir: cannot create directory ‘/home/aebestach/Code/Tekto2_crashed’: File exists
mv: cannot stat '/home/aebestach/Code/Tekto2/*': No such file or directory
Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1374, in postprocess
    pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3690, in postprocess
    data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1394, in dataset
    rawdata = readfile(filename)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 597, in readfile
    import exoplasim.pyfft as pyfft
ModuleNotFoundError: No module named 'exoplasim.pyfft'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1374, in postprocess
    pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3740, in postprocess
    data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1394, in dataset
    rawdata = readfile(filename)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 597, in readfile
    import exoplasim.pyfft as pyfft
ModuleNotFoundError: No module named 'exoplasim.pyfft'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1078, in _run
    timeavg=self.postprocess(dataname,None,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1396, in postprocess
    if self.integritycheck("%s%s"%(inputfile,extension)):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1433, in integritycheck
    ioe = self.postprocess(ncfile[:-3],"example.nl",crashifbroken=False)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1406, in postprocess
    raise RuntimeError("Going to stop here just in case......")
RuntimeError: Going to stop here just in case......

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1093, in _run
    self._crash()
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1989, in _crash
    raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname))
RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/Code/Tekto2_crashed/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/Code/tekto.py", line 9, in <module>
    Tekto.run(years=10,crashifbroken=True,clean=True)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 613, in run
    self._run(**kwargs)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1147, in _run
    self._crash() #Bring in the cleaners
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1989, in _crash
    raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname))
RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/Code/Tekto2_crashed/

I use Ubuntu 22.04 to run the model, but it always gives such an error, the version of python is 3.10

import exoplasim as exo
Tekto = exo.Model(workdir="Tekto2",modelname="Tekto2",inityear=0,outputtype=".nc",ncpus=16,precision=4,resolution="T21",layers=10)
Tekto.configure(startemp=5772.0,flux=100.67,year=720.0,eccentricity=0.016715,obliquity=0,lonvernaleq=102.7,fixedorbit=True,rotationperiod=15.0,
gravity=2.45,radius=0.4667,soilalbedo=0.2,seaice=False,
                  desertplanet=True,
                  pressure=1.2463,gascon=289.6,
                  timestep=45.0,runsteps=11520,snapshots=480)
Tekto.exportcfg()
Tekto.run(years=10,crashifbroken=True,clean=True)
Tekto.finalize("Tekto",allyears=False,keeprestarts=False,clean=True)
Tekto.save()

This is my installation command

sudo apt-get update -y
sudo apt install -y f2c
sudo apt install -y fftw3
sudo apt install -y openmpi-bin libopenmpi-dev
sudo apt install -y python3-pip
sudo apt-get install -y build-essential gfortran
sudo apt-get install -y libscalapack-openmpi-dev
sudo apt-get install -y python3-dev libxml2-dev libxslt1-dev zlib1g-dev libsasl2-dev libldap2-dev build-essential libssl-dev libffi-dev libmysqlclient-dev libjpeg-dev libpq-dev libjpeg8-dev liblcms2-dev libblas-dev libatlas-base-dev
pip3 install exoplasim
pip3 install netCDF4
pip3 install h5py

I tried the solution in the issue, but it didn't work.

[alphaparrot]

Try running exoplasim.sysconfigure() and then exoplasim.compile_pyfft(). If that doesn't fix the problem post the outputs from those commands and we'll see if the problem is your library environment or NumPy somehow.

On Tue, Jun 18, 2024, 11:35 PM Aebestach @.***> wrote:

No module named 'exoplasim.pyfft' No module named 'exoplasim.pyfft' Error writing output to MOST.00000.nc; log written to postprocess.log Going to stop here just in case...... mkdir: cannot create directory ‘/home/aebestach/Code/Tekto2_crashed’: File exists ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/Code/Tekto2_crashed/ mkdir: cannot create directory ‘/home/aebestach/Code/Tekto2_crashed’: File exists mv: cannot stat '/home/aebestach/Code/Tekto2/*': No such file or directory Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1374, in postprocess pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3690, in postprocess data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1394, in dataset rawdata = readfile(filename) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 597, in readfile import exoplasim.pyfft as pyfft ModuleNotFoundError: No module named 'exoplasim.pyfft'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1374, in postprocess pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3740, in postprocess data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1394, in dataset rawdata = readfile(filename) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 597, in readfile import exoplasim.pyfft as pyfft ModuleNotFoundError: No module named 'exoplasim.pyfft'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1078, in _run timeavg=self.postprocess(dataname,None, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1396, in postprocess if self.integritycheck("%s%s"%(inputfile,extension)): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1433, in integritycheck ioe = self.postprocess(ncfile[:-3],"example.nl",crashifbroken=False) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1406, in postprocess raise RuntimeError("Going to stop here just in case......") RuntimeError: Going to stop here just in case......

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1093, in _run self._crash() File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1989, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/Code/Tekto2_crashed/

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/Code/tekto.py", line 9, in Tekto.run(years=10,crashifbroken=True,clean=True) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 613, in run self._run(**kwargs) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1147, in _run self._crash() #Bring in the cleaners File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1989, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/Code/Tekto2_crashed/

I use Ubuntu 22.04 to run the model, but it always gives such an error, the version of python is 3.10

import exoplasim as exoTekto = exo.Model(workdir="Tekto2",modelname="Tekto2",inityear=0,outputtype=".nc",ncpus=16,precision=4,resolution="T21",layers=10)Tekto.configure(startemp=5772.0,flux=100.67,year=720.0,eccentricity=0.016715,obliquity=0,lonvernaleq=102.7,fixedorbit=True,rotationperiod=15.0,gravity=2.45,radius=0.4667,soilalbedo=0.2,seaice=False, desertplanet=True, pressure=1.2463,gascon=289.6, timestep=45.0,runsteps=11520,snapshots=480)Tekto.exportcfg()Tekto.run(years=10,crashifbroken=True,clean=True)Tekto.finalize("Tekto",allyears=False,keeprestarts=False,clean=True)Tekto.save()

This is my installation command

sudo apt-get update -y sudo apt install -y f2c sudo apt install -y fftw3 sudo apt install -y openmpi-bin libopenmpi-dev sudo apt install -y python3-pip sudo apt-get install -y build-essential gfortran sudo apt-get install -y libscalapack-openmpi-dev sudo apt-get install -y python3-dev libxml2-dev libxslt1-dev zlib1g-dev libsasl2-dev libldap2-dev build-essential libssl-dev libffi-dev libmysqlclient-dev libjpeg-dev libpq-dev libjpeg8-dev liblcms2-dev libblas-dev libatlas-base-dev pip3 install exoplasim pip3 install netCDF4 pip3 install h5py

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2177508164, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQHQ6HHJRLAEILW752TZID363AVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZXGUYDQMJWGQ . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import exoplasim
>>> exoplasim.sysconfigure()
./configure -v 3.10
Found FORTRAN-90 compiler     at: /usr/bin/gfortran
gfortran version  11.4
Found C compiler              at: /usr/bin/gcc
Found C++ compiler            at: /usr/bin/c++
Found MPI FORTRAN-90 compiler at: /usr/bin/mpif90
Found MPI C compiler          at: /usr/bin/mpicc
Found MPI C++ compiler        at: /usr/bin/mpicxx
Found GNU assembler           at: /usr/bin/as
Fast Legendre Transformation    : ACTIVE (Linux)
gfortran -o f90check.x f90check.f90
gcc -o cc_check.x cc_check.c

System info for <AebestachLaptop>
Architecture: Linux AebestachLaptop 5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Endian format             : little endian
FORTRAN control word size : 4 bytes
FORTRAN integer size      : 4 bytes
FORTRAN real    size      : 4 bytes
C       int     size      : 4 bytes
C       float   size      : 4 bytes
C       long    size      : 8 bytes
C  long long    size      : 8 bytes
FORTRAN Compiler: gfortran
C       Compiler: gcc
>>> exoplasim.compile_pyfft()
/home/aebestach/.local/lib/python3.10/site-packages/exoplasim
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 123, in compile_pyfft
    raise e
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 84, in compile_pyfft
    failed = numpy.f2py.compile(pyfft_source,modulename='pyfft',
  File "/home/aebestach/.local/lib/python3.10/site-packages/numpy/f2py/__init__.py", line 83, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy.f2py' has no attribute 'compile'

python3$pyversion -m numpy.f2py -c -m pyfft --f90exec=gfortran --f77exec=gfortran --f90flags="-O3" pyfft.f90 && mv pyfft.cpython*.so pyfft.so I solved this problem with this command, but first you need to go to home\username\.local\lib\python3.10\site-packages\exoplasim

[Aebestach]

1. I set ncpus to 64 and it works fine. But I can see only 32 cores by running the command cat /proc/cpuinfo | grep processor | wc -l. How come ncpus=64 works?
2. If the resolution of T170 is used, can ncpus=32 run? If not, what is the minimum required?
3. In Ubuntu 18.04, I tried to adjust the resolution to T170 and changed the other variables accordingly, but it always gives an error. Why is this? The log and code are placed below

   #KerbinT170.py
   import exoplasim as exo
   Kerbin = exo.Model(workdir="Kerbin_run",modelname="Kerbin",inityear=0,outputtype=".nc",ncpus=32,precision=8,resolution="T170",layers=10)
   Kerbin.configure(startemp=5840.0,flux=1360.0,
                 year=426.0,eccentricity=0.0,obliquity=0.0,lonvernaleq=0.0,fixedorbit=True,
                 rotationperiod=0.25,
                 gravity=9.81,radius=1.0,
                 vegetation=2,vegaccel=1,nforestgrowth=1.0,initgrowth=0.5,initstomcond=1.0,initrough=2.0,initsoilcarbon=0.0,initplantcarbon=0.0,
                 wetsoil=False,seaice=True,oceanzenith="ECHAM-3",
                 landmap="SRA/Kerbin_surf_0172.sra",topomap="SRA/Kerbin_surf_0129.sra",
                 pressure=1.013,drycore=False,ozone=True,
                 timestep=5.0,runsteps=11520,snapshots=4320,stormclim=True,
                 stormcapture={'toggle': 1,'NKTRIGGER': 1})
   Kerbin.exportcfg()
   Kerbin.run(years=10,crashifbroken=True)
   Kerbin.finalize("Kerbin",allyears=False,keeprestarts=False)
   Kerbin.save()

   
   Writing to /home/aebestach/code/Kerbin/Kerbin_run/Kerbin.cfg....
   --------------------------------------------------------------------------
   MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
   with errorcode 0.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them.

'152' '152' Error writing output to MOST.00000.nc; log written to postprocess.log Going to stop here just in case...... ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/ mkdir: cannot create directory ‘/home/aebestach/code/Kerbin/Kerbin_crashed’: File exists mv: cannot stat '/home/aebestach/code/Kerbin/Kerbin_run/*': No such file or directory Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1256, in postprocess gravity=self.gravity,gascon=self.gascon,**newkwargs) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/pyburn.py", line 3688, in postprocess physfilter=physfilter,logfile=logfile) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/pyburn.py", line 1413, in dataset gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon, KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1256, in postprocess gravity=self.gravity,gascon=self.gascon,**newkwargs) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/pyburn.py", line 3737, in postprocess zonal=zonal, substellarlon=substellarlon, physfilter=physfilter,logfile=logfile) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/pyburn.py", line 1413, in dataset gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon, KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 960, in _run log="burnout",crashifbroken=crashifbroken) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1276, in postprocess if self.integritycheck("%s%s"%(inputfile,extension)): File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1313, in integritycheck ioe = self.postprocess(ncfile[:-3],"example.nl",crashifbroken=False) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1286, in postprocess raise RuntimeError("Going to stop here just in case......") RuntimeError: Going to stop here just in case......

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 973, in _run self._crash() File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1869, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "Kerbin.py", line 14, in Kerbin.run(years=10,crashifbroken=True) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 495, in run self._run(**kwargs) File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1027, in _run self._crash() #Bring in the cleaners File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/init.py", line 1869, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

4. I have successfully run tekto.py in Ubuntu 22.04 and got normal output, but when I run Kerbin.py, it shows the style in the video. Is this normal?

```python
#tekto.py
import exoplasim as exo
Tekto = exo.Model(workdir="Tekto2",modelname="Tekto2",inityear=0,outputtype=".nc",ncpus=16,precision=4,resolution="T21",layers=10)
Tekto.configure(startemp=5772.0,flux=100.67,year=720.0,eccentricity=0.016715,obliquity=0,lonvernaleq=102.7,fixedorbit=True,rotationperiod=15.0,
gravity=2.45,radius=0.4667,soilalbedo=0.2,seaice=False,
                  desertplanet=True,
                  pressure=1.2463,gascon=289.6,
                  timestep=45.0,runsteps=11520,snapshots=480)
Tekto.exportcfg()
Tekto.run(years=10,crashifbroken=True,clean=True)
Tekto.finalize("Tekto",allyears=False,keeprestarts=False,clean=True)
Tekto.save()

#Kerbin.py
import exoplasim as exo
Kerbin = exo.Model(workdir="Kerbin_run",modelname="Kerbin",inityear=0,outputtype=".nc",ncpus=32,precision=8,resolution="T42",layers=10)
Kerbin.configure(startemp=5840.0,flux=1360.0,
                  year=426.0,eccentricity=0.0,obliquity=0.0,lonvernaleq=0.0,fixedorbit=True,
                  rotationperiod=0.25,
                  gravity=9.81,radius=1.0,
                  vegetation=2,vegaccel=1,nforestgrowth=1.0,initgrowth=0.5,initstomcond=1.0,initrough=2.0,initsoilcarbon=0.0,initplantcarbon=0.0,
                  wetsoil=False,seaice=True,oceanzenith="ECHAM-3",
                  landmap="SRA/Kerbin_surf_0172.sra",topomap="SRA/Kerbin_surf_0129.sra",
                  pressure=1.013,drycore=False,ozone=True,
                  timestep=15.0,runsteps=11520,snapshots=1440,stormclim=True,
                  stormcapture={'toggle': 1,'NKTRIGGER': 1})
Kerbin.exportcfg()
Kerbin.run(years=10,crashifbroken=True)
Kerbin.finalize("Kerbin",allyears=False,keeprestarts=False)
Kerbin.save()

https://github.com/alphaparrot/ExoPlaSim/assets/64588584/0e3f4dfd-b032-4403-8102-703e471be2e5

[alphaparrot]

Something is wrong with your numpy installation. Try reinstalling numpy.

Re: 64 cores, your machine might support hyperthreading, i.e. 2 threads per core. If it doesn't, you can still spin up 64 threads, but the processor will make them take turns, which will slow down computation (possibly slower than if you'd just used 32, but I haven't tested this kind of run strategy).

On Wed, Jun 19, 2024, 1:12 AM Aebestach @.***> wrote:

I have another question. I set ncpus to 64 and it works fine. But I can see only 32 cores by running the command cat /proc/cpuinfo | grep processor | wc -l. How come ncpus=64 works?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2177759411, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQA553TEKCO7N4SZKYDZIEHNJAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZXG42TSNBRGE . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

Just now I can use it normally through the command python3$pyversion -m numpy.f2py -c -m pyfft --f90exec=gfortran --f77exec=gfortran --f90flags="-O3" pyfft.f90 && mv pyfft.cpython*.so pyfft.so

Do you have any suggestions for the second, third, and fourth questions?

[alphaparrot]

Well I don't know why you ran into an MPI problem, but your logs make it very clear that you're still using Python 3.6. I strongly suspect your compilation/library environment is causing you problems.

You can run T170 on a single core; it'll just take forever. More cores means faster runs, up to the number of latitudes in the specified resolution (256 for T170). Note that the model is not validated at extremely high resolutions; there are physical processes relevant on small scales that are not parameterized well in the model, or at least aren't parameterized well.

On Wed, Jun 19, 2024, 6:55 AM Aebestach @.***> wrote:

Just now I can use it normally through the command python3$pyversion -m numpy.f2py -c -m pyfft --f90exec=gfortran --f77exec=gfortran --f90flags="-O3" pyfft.f90 && mv pyfft.cpython*.so pyfft.so

Do you have any suggestions for the second, third, and fourth questions?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2178380240, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQBKEBEALWE4HXPGEJTZIFPTJAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZYGM4DAMRUGA . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

In ExoPlaSim==3.4.0, there is also such a problem

Writing to /home/aebestach/code/Kerbin/Kerbin_run/Kerbin.cfg....
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 0.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
'152'
'152'
Error writing output to MOST.00000.nc; log written to postprocess.log
Going to stop here just in case......
ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/
mkdir: cannot create directory ‘/home/aebestach/code/Kerbin/Kerbin_crashed’: File exists
mv: cannot stat '/home/aebestach/code/Kerbin/Kerbin_run/*': No such file or directory
Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1374, in postprocess
    pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3690, in postprocess
    data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1417, in dataset
    gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon,
KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1374, in postprocess
    pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3740, in postprocess
    data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1417, in dataset
    gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon,
KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1078, in _run
    timeavg=self.postprocess(dataname,None,
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1396, in postprocess
    if self.integritycheck("%s%s"%(inputfile,extension)):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1433, in integritycheck
    ioe = self.postprocess(ncfile[:-3],"example.nl",crashifbroken=False)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1406, in postprocess
    raise RuntimeError("Going to stop here just in case......")
RuntimeError: Going to stop here just in case......

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1093, in _run
    self._crash()
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1989, in _crash
    raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname))
RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aebestach/code/Kerbin/Kerbin.py", line 14, in <module>
    Kerbin.run(years=10,crashifbroken=True)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 613, in run
    self._run(**kwargs)
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1147, in _run
    self._crash() #Bring in the cleaners
  File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/__init__.py", line 1989, in _crash
    raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname))
RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

Even in version 3.4.0, running the model with T42 resolution, its console output frequency is very strange and very fast, which is not the case with T21 resolution.

[Aebestach]

Well, I gave up on the T170, it really feels like there is something wrong with it, I will try the T85. But do you think what is shown in the video is normal?

[alphaparrot]

Your model's crashing for some reason. What the error output is telling you is just that the model output it's trying to postprocess is incomplete; the actual question is why the model itself crashed. GCMs do crash from time to time; debugging why is its own challenge and more often than not is solved by fiddling, not identifying root causes. It could be your topography, it could be your short rotation period, it could be some combination of factors. If you compile with debug=True you might see more informative logs from the MPI runtime, and they might give you a hint as to what you need to change.

On Wed, Jun 19, 2024, 7:14 AM Aebestach @.***> wrote:

In ExoPlaSim==3.4.0, there is also such a problem

Writing to /home/aebestach/code/Kerbin/Kerbin_run/Kerbin.cfg....

MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 0.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them.

'152' '152' Error writing output to MOST.00000.nc; log written to postprocess.log Going to stop here just in case...... ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/ mkdir: cannot create directory ‘/home/aebestach/code/Kerbin/Kerbin_crashed’: File exists mv: cannot stat '/home/aebestach/code/Kerbin/Kerbin_run/*': No such file or directory Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1374, in postprocess pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3690, in postprocess data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1417, in dataset gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon, KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1374, in postprocess pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3740, in postprocess data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1417, in dataset gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon, KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1078, in _run timeavg=self.postprocess(dataname,None, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1396, in postprocess if self.integritycheck("%s%s"%(inputfile,extension)): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1433, in integritycheck ioe = self.postprocess(ncfile[:-3],"example.nl",crashifbroken=False) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1406, in postprocess raise RuntimeError("Going to stop here just in case......") RuntimeError: Going to stop here just in case......

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1093, in _run self._crash() File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1989, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/code/Kerbin/Kerbin.py", line 14, in Kerbin.run(years=10,crashifbroken=True) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 613, in run self._run(**kwargs) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1147, in _run self._crash() #Bring in the cleaners File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1989, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

Even in version 3.4.0, running the model with T42 resolution, its console output frequency is very strange and very fast, which is not the case with T21 resolution.

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2178418415, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQEQJFZ4A6VE6HHHYSTZIFRXRAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZYGQYTQNBRGU . You are receiving this because you were mentioned.Message ID: @.***>

[alphaparrot]

Yes, the video is normal. It does that when it runs.

On Wed, Jun 19, 2024, 7:24 AM Adiv Paradise @.***> wrote:

Your model's crashing for some reason. What the error output is telling you is just that the model output it's trying to postprocess is incomplete; the actual question is why the model itself crashed. GCMs do crash from time to time; debugging why is its own challenge and more often than not is solved by fiddling, not identifying root causes. It could be your topography, it could be your short rotation period, it could be some combination of factors. If you compile with debug=True you might see more informative logs from the MPI runtime, and they might give you a hint as to what you need to change.

On Wed, Jun 19, 2024, 7:14 AM Aebestach @.***> wrote:

In ExoPlaSim==3.4.0, there is also such a problem

Writing to /home/aebestach/code/Kerbin/Kerbin_run/Kerbin.cfg....

MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 0.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them.

'152' '152' Error writing output to MOST.00000.nc; log written to postprocess.log Going to stop here just in case...... ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/ mkdir: cannot create directory ‘/home/aebestach/code/Kerbin/Kerbin_crashed’: File exists mv: cannot stat '/home/aebestach/code/Kerbin/Kerbin_run/*': No such file or directory Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1374, in postprocess pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3690, in postprocess data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1417, in dataset gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon, KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1374, in postprocess pyburn.postprocess(inputfile,inputfile+self.extension,logfile=log,namelist=namelist, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 3740, in postprocess data = dataset(rawfile, variables, mode=mode,radius=radius,gravity=gravity,gascon=gascon, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/pyburn.py", line 1417, in dataset gridlnps,lnpsmeta = _transformvar(lon[:],lat[:],rawdata[str(lnpscode)][:],ilibrary[str(lnpscode)][:],nlat,nlon, KeyError: '152'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1078, in _run timeavg=self.postprocess(dataname,None, File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1396, in postprocess if self.integritycheck("%s%s"%(inputfile,extension)): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1433, in integritycheck ioe = self.postprocess(ncfile[:-3],"example.nl",crashifbroken=False) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1406, in postprocess raise RuntimeError("Going to stop here just in case......") RuntimeError: Going to stop here just in case......

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1093, in _run self._crash() File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1989, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/aebestach/code/Kerbin/Kerbin.py", line 14, in Kerbin.run(years=10,crashifbroken=True) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 613, in run self._run(**kwargs) File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1147, in _run self._crash() #Bring in the cleaners File "/home/aebestach/.local/lib/python3.10/site-packages/exoplasim/init.py", line 1989, in _crash raise RuntimeError("ExoPlaSim has crashed or begun producing garbage. All working files have been moved to %s_crashed/"%(os.getcwd()+"/"+self.modelname)) RuntimeError: ExoPlaSim has crashed or begun producing garbage. All working files have been moved to /home/aebestach/code/Kerbin/Kerbin_crashed/

Even in version 3.4.0, running the model with T42 resolution, its console output frequency is very strange and very fast, which is not the case with T21 resolution.

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2178418415, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQEQJFZ4A6VE6HHHYSTZIFRXRAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZYGQYTQNBRGU . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

I was stupid. The terrain data is still T42 data, not adjusted to T170. It's really stupid. I ignored this.

So actually for cyclones, T85 has the potential to produce cyclones, and T170 is better, even though the cyclone data may not be correct, is that right?

[alphaparrot]

It might happen in T85, but I'd just go straight to the highest resolution you and your machine can tolerate. T127 is an option as well. Supported resolutions beyond T170 are T341 (512 latitudes), T682 (1024 latitudes), and T1365 (2048 latitudes). I haven't pushed past T170 myself due to the computational expense.

On Wed, Jun 19, 2024, 7:34 AM Aebestach @.***> wrote:

I was stupid. The terrain data is still T42 data, not adjusted to T170. It's really stupid. I ignored this.

So actually for cyclones, T85 has the potential to produce cyclones, and T170 is better, even though the cyclone data may not be correct, is that right?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2178471975, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQAX6A3OCXZW4ZGJCEDZIFUCLAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZYGQ3TCOJXGU . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

For T127, what is the recommended time step?

[alphaparrot]

Whatever works! You need to use trial and error. Start with 5 minutes and expect to need to reduce it. Make sure you choose values that divide cleanly into 24 hours (regardless of your rotation period).

On Wed, Jun 19, 2024, 9:19 AM Aebestach @.***> wrote:

For T127, what is the recommended time step?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2178704234, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQCOIJ3TRUTUCWZ3OX3ZIGAONAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZYG4YDIMRTGQ . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

Unfortunately, the problem occurred again. I have reinstalled numpy but it still gives an error, but this time the error is No module named 'exoplasim.pyfft991'. The resolution I ran this time is T127

[Aebestach]

T85 is normal, T127 is not, maybe I need to adjust the parameters?

[alphaparrot]

Well the immediate problem is your f2py workaround only compiled pyfft; pyfft991 is also needed to support all resolutions.

The bigger problem, however, seems to be that numpy 2.0 released two days ago, and breaks basically everything, including pythonic f2py functionality (see https://github.com/numpy/numpy/issues/25122). For now, downgrade to numpy<2.0 and re-run exoplasim.compile_pyfft(). For later, I'm opening a separate issue to deal with the f2py API changes. Again.

[Aebestach]

It's a numpy problem, I'm rerunning with T127 resolution and hope everything goes well

[Aebestach]

Does a situation like this mean that the variable needs to be modified?

[alphaparrot]

I haven't encountered that before; that looks like something that might result from a keyboard interrupt or running out of system resources (and getting killed by the OS)

On Wed, Jun 19, 2024, 9:17 PM Aebestach @.***> wrote:

image.png (view on web) https://github.com/alphaparrot/ExoPlaSim/assets/64588584/efd43f8c-9c4f-4562-a23f-63ce6c9bb3a6 Does a situation like this mean that the variable needs to be modified?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2179646912, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQC6N6XBNUPINCC2MDLZIIUUFAVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZGY2DMOJRGI . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

It has been 1 hour running at T127 resolution on the i9-13900HX, and the data for the first year has not been output yet Correction: It takes one hour to simulate one year Is there any platform that can run these codes? Do you have any recommendations, some cloud service providers?

[alphaparrot]

Like I said—these resolutions take days to weeks to run. You're not just increasing the size of the model domain, you're also increasing the number of timesteps.

You could try spinning up a beefy EC2 virtual machine in AWS, if money isn't an issue. Most people who run these codes at very high resolutions use supercomputers.

ExoPlaSim is not meant to make accessible the power of high-resolution GCMs. It's meant to provide the power of a very fast intermediate-complexity GCM that runs best at low and moderate resolution, for climate simulations, not weather simulations. What you're asking it to do is technically within its capabilities, but very much to an extreme, and quite possibly outside any reasonable person's patience.

On Thu, Jun 20, 2024, 12:47 AM Aebestach @.***> wrote:

It has been 1 hour running at T127 resolution on the i9-13900HX, and the data for the first year has not been output yet Is there any platform that can run these codes? Do you have any recommendations, some cloud service providers?

— Reply to this email directly, view it on GitHub https://github.com/alphaparrot/ExoPlaSim/issues/28#issuecomment-2179797809, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRQKQBI2CIW6BF63L7WRCDZIJNF7AVCNFSM6AAAAABJP65YJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZG44TOOBQHE . You are receiving this because you were mentioned.Message ID: @.***>

[Aebestach]

LOL, that's true. I'll go to AWS and have a look. Can ExoPlaSim adjust the input Coriolis parameter?

By the way, does this problem mean that the model has crashed and needs to be constantly adjusted and rerun? I set crashifbroken=True, but I don't see any specific crash logs. Can you tell me where they are?

Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#0  0x7fecb25c732a
#1  0x7fecb25c6503
#2  0x7fecb1c43f0f
#3  0x7fecb221cad0
#4  0x5641c2c2ea13
#5  0x5641c2c30cc9
#6  0x5641c2c33a0a
#7  0x5641c2c4d049
#8  0x5641c2c55ed8
#9  0x5641c2c022ff
#10  0x7fecb1c26c86
#11  0x5641c2c02349
#12  0xffffffffffffffff
--------------------------------------------------------------------------
mpiexec noticed that process rank 8 with PID 0 on node AebestachLaptop exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------
Command '['mpiexec -np 16 most_plasim_t127_l10_p16.x']' returned non-zero exit status 136.
Traceback (most recent call last):
  File "/home/aebestach/.local/lib/python3.6/site-packages/exoplasim/__init__.py", line 930, in _run
    subprocess.run([self._exec+self.executable],shell=True,check=True)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['mpiexec -np 16 most_plasim_t127_l10_p16.x']' returned non-zero exit status 136

[Aebestach]

https://github.com/alphaparrot/ExoPlaSim/assets/64588584/39cb22b3-d02f-4cfc-a77a-992e688aae2e

Similar to this running speed, I used the same file, and in Ubuntu-18.04, a .nc file was generated in 70 minutes, but in Ubuntu-22.04, it did not appear in 90 minutes. The main difference between the two is the running speed. In 18.04, the speed is very slow, but in 22.04, it is very fast. This is very abnormal, Even the CPU usage is only 2%.

[Aebestach]

I tested four versions of Ubuntu: 18.04 20.04 22.04 24.04. Only 18.04 and 20.04 were running normally, and the CPU usage was also consistent with the level when running normal programs. For 20.04, even though the CPU usage is 100%, the window still shows this