-
Tap is frequently used to pull data from the postgres replica and a common issue in this case is "canceling statement due to conflict with recovery" especially for a full table sync. It happens when W…
-
-
這個issue將被用做記錄kafka-spark-delta ETL進度,我將大致會將其切成四個部分逐一完成,每個部分我再根據工作量切成小的pr進行完成:
- [ ] 讀取prop file配置相關參數 :
透過 prop file 定義過程使⽤的所有參數,包括資料的路徑,資料樣式,目標 Kafka topic相關資訊等。
- [ ] CSV to Spark :
從CSV將資料匯入spa…
-
**Environment setup: AWS EMR serverless 6.9.0 version
Pyspark ETL job with multiple streaming queries, each streaming query writes to an iceberg table and redshift table, in microbatches, the trigger…
-
I have use case to capture the bad records and store it in a separate location for future reference. Is it possible to get the records which are not deserialised instead of dropping them?
-
These workflows:
- https://github.com/nextstrain/ncov-ingest/blob/master/.github/workflows/fetch-and-ingest-gisaid-master.yml
- https://github.com/nextstrain/ncov-ingest/blob/master/.github/workflo…
-
@yruslan
I have large set of input files for processing. I wanted to take the stream processing approach. But, when i tried with the same options that i have successfully used for variable length re…
-
@ask11 i need to push an update to the readme and docs, but i would like to get your feedback and enhancements on the issues in the first 5 milestones for this project.
we're converting the old ruby-…
-
I am trying to use ethereumetl stream with the following command line:
```
ethereumetl stream --provider-uri file:///mnt/disks/geth/geth/geth.ipc --output=postgresql+pg8000://daniel_d_kang:[password…
-
### What is the issue?
embedding model
When I submit a single fragment, it responds normally, but when I submit multiple fragments, an exception occurs.
I encountered this error on different Wind…