-
**Description:**
The Extract-Transform-Load (ETL) design pattern is crucial for data integration and data warehousing processes. It involves extracting data from various sources, transforming it to fi…
-
As part of our pilot program with STLT's we need to understand the challenges that come with putting data into Maven or Clinisys assume our data is in JSON format the question we should answer is
1. W…
-
The question this issue is two fold. To start a discussion on how to transform existing datasets into a form that can work in a format that Citygram expects, and second to recommend a few tools that a…
-
### Discussed in https://github.com/apache/airflow/discussions/33556
Originally posted by **ntnhaatj** August 20, 2023
### Description
Hi, as [my issue was raised here](https://github.com/a…
-
Hello,
Trained the model on the imagenet30 dataset and inference it on a few images. However, the anomaly scores are coming in some weird numbers. Something like 84.31, 84.32, etc for both anomalous …
-
1. arXiv에서 논문들의 정보를 가져오는 데이터 파이프라인이 있으면 좋을 듯?
-
We need to develop a robust and scalable data ingest/ETL (Extract, Transform, Load) pipeline to facilitate the reading of eQTL (expression Quantitative Trait Loci) data from FTP sources, indexing it i…
-
Apache NiFi
Apache Airflow
pandas in Python
-
The data files in Participant Data are tricky to work with in Python requiring unclear expressions to access the data, eg
`age = data['data'][0,0]['individual'][0,0]['age'][0][0]`
In order…
-
https://www.youtube.com/watch?v=ad1GmW_TmYg