-
As a data scientist, when creating or changing a schema for ingesting or processing data, I want to be able to capture metadata about the database / table / columns. The data could include name, descr…
-
**Is your feature request related to a problem? Please describe.**
Right now, CEMS datatypes are not handled like FERC or EIA data. Datatypes are not defined using `codes.py` or `fields.py`, but rath…
-
I installed `catalystcoop.pudl-catalog` via mamba as part of a larger environment.yml file and had 2 issues:
1. sqlalchemy 2.0.15 was installed. I see that setup.py has been modified to limit it be…
-
# Summary of results:
See the job run logs and results [here](https://github.com/catalyst-cooperative/pudl-archiver/actions/runs/9740834873).
# Review and publish archives
For each of the follo…
-
EPA CEMS is bigger than laptop memory, there is no getting around that. But after loading, fully 50% of memory is taken by one column, 'unitid'. This column is a string dtype, but could probably be ch…
-
I am tinkering with IIIF image "discovery" for my manifest editor and was wondering if we can add content negotiation to the ib app to easily find/add images from the image server.
When ib gets a jso…
-
In order to make the CEMS data more usable for analyzing GHG emission factors at the generator level, I am beginning work on `pudl.analysis.emissions` which will create a "cleaned" CEMS dataframe that…
-
The EIA 861 brings a new kind of entity the Balancing Authority. It also brings a huge number of new tables related to Utilities, so it's important to integrate it into the post-Transform harvesting …
-
I wanted to suggest that a Python version of the R script be developed (or that this code base be transitioned to Python) to help facilitate contributions by the parts of the user community that use t…
-
### What's the use case?
[PUDL](https://github.com/catalyst-cooperative/pudl) has a few pre-configured jobs and about a dozen asset groups. For example, we have a job that runs the ETL for all availa…