-
### Objective
As a data scientist, I want to build a Python data loading script `src/data_ingestion.py` and an exploratory data analysis (EDA) notebook `eda/eda.ipynb`.
### Acceptance Criteria
- Da…
-
- [x] Update config.YAML
- [x] Update params.YAML
- [x] Research or analyis
- [x] Update the entity
- [x] Update the configuration manager in src config.
- [x] Update the components
- [x] Updat…
-
### What type of enhancement is this?
API improvement
### What does the enhancement do?
From one of our key user who only wants to generate derived data without persist original data. We will need …
-
- [ ] First we have to dump the dataset into **MySQL** dataset
- [ ] Create / Establish a connection with database .
- [ ] Load the data and crate **dataframe**
- [ ] Drop the unnecessary columns…
-
### Problem Statement
We have a lot of customers who run out of quota, or run into spikes, or otherwise see data that's missing. On Dashboards, this often results in mysterious or confusing behaviour…
-
## Expectation
As a user, I require access to the following datasets which I will access via the API:
1. Ground Observations (TAHMO stations) - csv
2. Ground Observations (TAHMO stations) - API
3. CB…
-
-
(Random thoughts as I'm struggling with slow ingestion times)
Speaking of multi-threading: further down the line the bottleneck for real-time use cases is probably gonna be the speed of ingestion w…
-
**Title**: Ingest LLM Data into Editable Table
**Description**: Develop a data ingestion pipeline to store new headlines and associated metadata in a structured, easily editable table.
**Dependencie…
-
I'm looking into the AIND format option for inputting data into the pipeline. The readme states
> aind: data ingestion used at AIND. The input folder must contain an ecephys subfolder which in turn…