simpeg / aurora

software for processing natural source electromagnetic data
MIT License
14 stars 2 forks source link

PKD SAO RR Test grabs multiple years of data #243

Closed kujaku11 closed 1 year ago

kujaku11 commented 1 year ago

@kkappler I'm running the parkfield\test_process_parkfield_run_rr.py test and it is downloading multiple years of data. PKD is downloading data from 2003-09-13 to 2005-03-15 and SAO 2003-10-29 to 2007-06-29. Somehow the data request is skipping the desired time block.

When I open the stationxml I get a warning from Obspy:

C:\Users\jpeacock\Anaconda3\envs\em\lib\site-packages\obspy\io\stationxml\core.py:91: UserWarning: The StationXML file has version 1.2, ObsPy can read versions (1.0, 1.1). Proceed with caution.
  warnings.warn("The StationXML file has version %s, ObsPy can "

The StationXML has:

<Network code="BK" startDate="1980-10-01T00:00:00.000000Z" endDate="3000-01-01T00:00:00.000000Z">
    <Description>Berkeley Digital Seismic Network (BDSN)</Description>
    <TotalNumberStations>161</TotalNumberStations>
    <SelectedNumberStations>5</SelectedNumberStations>
    <Station code="PKD" startDate="1996-09-06T16:28:00.000000Z" endDate="3000-01-01T00:00:00.000000Z">
      <Latitude unit="DEGREES">35.94517</Latitude>
      <Longitude unit="DEGREES">-120.5416</Longitude>
      <Elevation>583.0</Elevation>
      <Site>
        <Name>Bear Valley Ranch, Parkfield, CA, USA</Name>
      </Site>
      <CreationDate>1996-09-06T16:28:00.000000Z</CreationDate>
      <Channel code="BQ1" startDate="2003-09-12T18:54:00.000000Z" endDate="2005-03-15T16:45:00.000000Z" locationCode="">

An OverflowError: Python int too large to convert to C long is raised.

kujaku11 commented 1 year ago

@kkappler I found the issue. When a channel is initialized in MTH5 by just metadata alone, the size of the data is estimated based on start and end time as well as sample rate. Because the StationXML has all data the estimated size is mulitple years, turns out that is too large to be an int. I fixed the issue in MTH5.