-
I have added `confluent-kafka`, `pymongo`, `pyodbc` and other packages in requirements.txt and and importing those in code inside the plugins directory. On local runner it is working fine.
But when…
-
I run the official code example in `intro.ipynb`：
```python
import dspy
lm = dspy.LM(model='openai/default', api_key=" ", api_base=" ",temperature=0.9, max_tokens=3000,)
colbertv2_wiki17_abstrac…
-
### Description
Airflow use `git-sync` to pull DAGs from git repository, I'm thinking about doing it in a more cloud native way.
So I thought of using k8s `custom resource definition` to do this.
W…
-
We want a generic way of for database connections like mysql, postgres, mongodb
This is done when
```[tasklist]
### Tasks
- [ ] https://github.com/stackabletech/documentation/pull/321
…
-
In case you have multiple Python files that form an application it is inconvenient to put everything in a ConfigMap.
It would be nice if you could use the same mechanism as implemented in Airflow usi…
-
## Context
In a recent conversation about unplanned EC2 instance restarts and how they might affect our infrastructure, we began discussing how our current DAGs might handle a random dropout. H…
-
One of the strengths of airflow is the way it democratizes orchestration through operators, and as a result many users have custom or less-popular operators as key components in their pipelines. In or…
-
### Description
Standardize similar AWS transfer operators to have similar functionality, as well as similar naming conventions.
This would start with updating transfer operators using S3 as a sou…
-
### Apache Airflow version
2.9.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
As per the confluence wiki, dynamic task mapping [users will have an option…
-
Hello,
I been searching in google for a couple of hours now but I cant find a workaround this error. I'm trying to use DockerOperator for airflow.
DAG:
```from airflow import DAG
from airflow…