GEOS-FP data missing in GEOS_0.125x0.15625_AS

[xlwangg]

Your name

Xiaolin Wang

Your affiliation

Harvard University

What happened? What did you expect to happen?

Some I1dyn and A1dyn files are missing from the GEOS_0.125x0.15625_AS archive. Please see the list of missing timestamps below.

What are the steps to reproduce the bug?

N/A

Please attach any relevant configuration and log files.

No response

What GEOS-Chem version were you using?

14.4.1

What environment were you running GEOS-Chem on?

Local cluster

What compiler and version were you using?

gcc 12.0

Will you be addressing this bug yourself?

Yes

In what configuration were you running GEOS-Chem?

GCClassic

What simulation were you running?

Full chemistry

As what resolution were you running GEOS-Chem?

0.125x0.15625

What meterology fields did you use?

GEOS-FP

Additional information

No response

[msulprizio]

Thanks for reporting this. @yidant could you look into the missing files?

[yidant]

Thanks @xlwangg and @msulprizio! These files are produced! Feel free to reach out if any additional missing file is found.

[xlwangg]

Thanks very much @yidant ! The I1dyn file for 20210709 11 is still missing; all others are fine. Could you check this further?

[yidant]

Hi @xlwangg, it might take a few minutes to show up on the data portal. I can already see the file there. Could you refresh the page and check if you can see this file?

[xlwangg]

I see it now, thanks @yidant !!!

[xlwangg]

Hi @yidant, I noticed that the newly uploaded files have NaN values for lon_bnds and lat_bnds, which are causing issues with my subsequent processing. Could you look into this further?

[yidant]

Thanks @xlwangg for reporting this! It looks like all files created after 20230101 under GEOS_0.125x0.15625_AS are missing lon_bnds,lat_bnds. I can look into the source.

A workaround right now is to use ncks -v lon_bnds,lat_bnds <source_file> <target_file> to fill in these values from another file. I'll fix these files. Feel free to do the same if this issue happens to other files.

[xlwangg]

@yidant Thanks, this works!

[xlwangg]

Hi @yidant , I ran into the issue of missing files again while trying to run a long simulation at 12km over North America. There are hundreds of missing files between 2021-03-11 and 2024-09-30 (see attached Word document for the file list). If possible, would you consider adding warnings or an auto-completion feature to the routine process? I’ve attached my Python script for detecting missing files in a specific path. Missing_GEOS_0.125x0.15625_NA_files.docx

import xarray as xr
import datetime
import os

rootdir = '/n/holyscratch01/jacob_lab/Users/msulprizio/GEOS_0.125x0.15625_NA/GEOS_FP_DerivedWinds'

# Simulation time
startday = '20230101'
endday = '20241013'
Region = 'NA'
days = []
dt = datetime.datetime.strptime(startday, "%Y%m%d")
dt_max = datetime.datetime.strptime(endday, "%Y%m%d")
while dt <= dt_max:
    dt_str = str(dt)[0:10].replace("-", "")
    days.append(dt_str)
    delta = datetime.timedelta(days=1)
    dt += delta
nday = len(days)
hours = [i for i in range(24)]
hours = [f'{hour:02}' for hour in hours]

# For each day
for iday in range(0,nday):
    for ihour in hours:

        thisday = days[iday]
        data_pth = f"{rootdir}/Y{thisday[0:4]}/M{thisday[4:6]}/D{thisday[6:8]}"

        checknc = True
        if checknc == True:
            if not os.path.exists(f"{data_pth}/GEOS.fp.asm.A1dyn.0125x015625.{Region}.{thisday}_{ihour}30.V01.nc4"):
                print(f'not exist field for GEOS_0.125x0.15625_{Region} A1dyn:',thisday,ihour)
            if not os.path.exists(f"{data_pth}/GEOS.fp.asm.I1dyn.0125x015625.{Region}.{thisday}_{ihour}00.V01.nc4"):
                print(f'not exist field for GEOS_0.125x0.15625_{Region} I1dyn:',thisday,ihour)

print("Finished!")

[yidant]

Thanks @xlwangg! These missing files are now available.

The only exception are several files for 20210504. They seem to have failed dependencies and may require checking the raw files.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If there are no updates within 7 days it will be closed. You can add the "never stale" tag to prevent the issue from closing this issue.

[github-actions[bot]]

Closing due to inactivity