Create a data package

[RamiKrispin]

Data requirements Frequency:

• [x] Hourly

• [x] Daily

• [x] Monthly

• [ ] Quarterly
  Structure:

• [x] Seasonal

• [x] Multiple seasonal

• [x] Linear trend

• [ ] Non-linear trend

Resources

• EIA API
• FRED
• SF Data Portal

[RamiKrispin]

US hourly demand for electricity by subregion

API: EIA

Dashboard: https://www.eia.gov/opendata/browser/electricity/rto/region-sub-ba-data

API URL: https://api.eia.gov/v2/electricity/rto/region-sub-ba-data/data/

METHOD: GET

SERIES DESCRIPTION: Hourly demand by balancing authority subregion. Source: Form EIA-930 Product: Hourly Electric Grid Monitor

Header:

"X-Params": {
    "frequency": "hourly",
    "data": [
        "value"
    ],
    "facets": {},
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "parent",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

[RamiKrispin]

US hourly demand and net generation of electricity by subregion

Dashboard: https://www.eia.gov/opendata/browser/electricity/rto/region-data

API URL: https://api.eia.gov/v2/electricity/rto/region-data/data/

METHOD: GET

SERIES DESCRIPTION: Hourly demand, day-ahead demand forecast, net generation, and interchange by balancing authority. Source: Form EIA-930 Product: Hourly Electric Grid Monitor

Header:

"X-Params": {
    "frequency": "hourly",
    "data": [
        "value"
    ],
    "facets": {
        "type": [
            "D",
            "NG"
        ]
    },
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

[RamiKrispin]

Monthly demand for natural gas

Dashboard: https://www.eia.gov/opendata/browser/natural-gas/sum/lsum

API URL: https://api.eia.gov/v2/natural-gas/sum/lsum/data/

METHOD: GET

Header:

"X-Params": {
    "frequency": "monthly",
    "data": [
        "value"
    ],
    "facets": {
        "series": [
            "N3010NY2"
        ]
    },
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

[RamiKrispin]

Vehicle Miles Traveled

Source: FRED

Link: https://fred.stlouisfed.org/series/TRFVOLUSM227NFWA

[RamiKrispin]

Total Vehicle Sales

Source: FRED

Link: https://fred.stlouisfed.org/series/TOTALNSA

[RamiKrispin]

Total Construction Spending: Total Construction in the United States

Source: FRED

Link: https://fred.stlouisfed.org/series/TTLCON

[RamiKrispin]

Air Traffic Passenger Statistics

Source: SF Data Portal

Link: https://data.sfgov.org/Transportation/Air-Traffic-Passenger-Statistics/rkru-6vcg

[RamiKrispin]

Air Traffic Landings Statistics

Source: SF Data Portal

Link: https://data.sfgov.org/Transportation/Air-Traffic-Landings-Statistics/fpux-q53t

[RamiKrispin]

Recommendation from LinkedIn post:

• https://db.nomics.world/
• https://datasetsearch.research.google.com/
• Eurostat Tourism - https://ec.europa.eu/eurostat/databrowser/view/TOUR_OCC_NIM__custom_1357282/default/table?lang=en
• Income and Wealth Inequality in America, 1949–2016 - https://www.journals.uchicago.edu/doi/10.1086/708815
• https://zenodo.org/record/2582968
• Open sky - https://opensky-network.org/
• California ISO - http://www.caiso.com/todaysoutlook/pages/default.aspx
• https://physionet.org/about/database/
• https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177312
• https://journals.humankinetics.com/view/journals/jab/33/6/article-p464.xml
• https://github.com/TenanATC/EMG

[RamiKrispin]

Recommendation from Twitter post:

• TFL data - https://t.co/sdzmokWiqW
• Road crash data - stats19 https://docs.ropensci.org/stats19/
• IPEADATA API https://github.com/gomesleduardo/ipeadatar
• Envirnmental data - https://twitter.com/USGS_DataSci/status/1384541933222178816
• Water data - https://waterdata.usgs.gov/blog/tags/dataretrieval/

[RamiKrispin]

Daily demand, day-ahead demand forecast, net generation, and interchange by balancing authority

Dashboard: https://www.eia.gov/opendata/browser/electricity/rto/daily-region-data

API URL: https://api.eia.gov/v2/electricity/rto/daily-region-data/data/

METHOD: GET

Header:

"X-Params": {
    "frequency": "daily",
    "data": [
        "value"
    ],
    "facets": {
        "timezone": [
            "Pacific"
        ],
        "type": [
            "D"
        ]
    },
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}