noaa-oar-arl / monetio

The Model and ObservatioN Evaluation Tool I/O package
https://monetio.readthedocs.io
MIT License
17 stars 30 forks source link

Metadata mismatch error #219

Open DCAIRMAN opened 2 days ago

DCAIRMAN commented 2 days ago

I'm new to the Monetio package, so I'm working through the practices examples on the ARL website. When I run this code (see below), I get this error message:

import numpy as np import pandas as pd import monetio as mio ##observations for MONET import matplotlib.pyplot as plt import seaborn as sns ##using this for better color palettes import cartopy.crs as ccrs ##map projections import cartopy.feature as cfeature ##poltical and geographic features

sns.set_style("ticks") ##plot configuration sns.set_context("talk") ##plot configure for size of text

dates = pd.date_range(start='2004-01-01',end='2004-12-31') df = mio.aqs.add_data(dates,daily=True,param=['VOC','OZONE'], download=True)

from numpy import sort

for i in sort(df.columns): print(i)

from monet.util import tools

new = tools.long_to_wide(df) new.head()

new.dropna(subset=["ISOPRENE"]).siteid.unique().shape

for i in sort(new.columns): print(i) new.describe()

new.index = new.time new["OZONE_ppb"] = new.OZONE * 1000 print(new.OZONE_ppb.mean())

f, ax = plt.subplots(figsize=(10, 4)) new.ISOPRENE.resample("D").mean().plot(ax=ax) plt.show()

ValueError: Metadata mismatch found in from_delayed.

Partition type: pandas.core.frame.DataFrame +--------------+--------+----------+ | Column | Found | Expected | +--------------+--------+----------+ | 'event_type' | object | float64 | +--------------+--------+----------+

Any insights

zmoon commented 1 day ago

Hi @DCAIRMAN thanks for this report. I just tried your AQS add-data case and was not able to reproduce.

I did this:

import monetio as mio
import pandas as pd

dates = pd.date_range(start='2004-01-01', end='2004-12-31')
df = mio.aqs.add_data(dates, daily=True, param=['VOC','OZONE'], download=False)

'event_type' is okay:

>>> df.event_type.value_counts()
None        1505837
Included      21200
Excluded          4
Name: event_type, dtype: int64

Note that df here is in the wide format already so long_to_wide(df) is not needed (and would fail). Sorry that the examples are not super up-to-date.

Did you install the current version monetio from conda-forge? If so, could you try running again? Sometimes this "Metadata mismatch found in from_delayed" error occurs when you try to access the data while changes are being made on the server (though I don't know what they would be changing in 2004 data).