Open anayyar82 opened 4 years ago
Hi @anayyar82 , you can use kubectl logs
to see detail logs inside the container.
Thanks @stwind Yeah i did but seems it stucked somehow.
kubectl logs -f airflow -c scheduler
[2020-04-06 14:33:31,551] {settings.py:253} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=1
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2020-04-06 14:33:33,244] {__init__.py:51} INFO - Using executor KubernetesExecutor
[2020-04-06 14:33:33,303] {scheduler_job.py:1344} INFO - Starting the scheduler
[2020-04-06 14:33:33,303] {scheduler_job.py:1352} INFO - Running execute loop for -1 seconds
[2020-04-06 14:33:33,305] {scheduler_job.py:1353} INFO - Processing each file at most -1 times
[2020-04-06 14:33:33,306] {scheduler_job.py:1356} INFO - Searching for files in /tmp/airflow/dags
[2020-04-06 14:33:33,308] {scheduler_job.py:1358} INFO - There are 0 files in /tmp/airflow/dags
[2020-04-06 14:33:33,317] {kubernetes_executor.py:805} INFO - Start Kubernetes executor
[2020-04-06 14:33:53,879] {kubernetes_executor.py:351} INFO - Event: and now my watch begins starting at resource_version: 0
[2020-04-06 14:33:36,929] {kubernetes_executor.py:733} INFO - When executor started up, found 0 queued task instances
[2020-04-06 14:33:53,933] {scheduler_job.py:1409} INFO - Resetting orphaned tasks for active dag runs
[2020-04-06 14:33:54,212] {dag_processing.py:556} INFO - Launched DagFileProcessorManager with pid: 29
[2020-04-06 14:33:54,664] {settings.py:54} INFO - Configured default timezone <Timezone [UTC]>
[2020-04-06 14:33:54,693] {settings.py:253} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=29
and for Webserver
kubectl logs -f airflow -c webserver
[2020-04-06 14:33:31,551] {settings.py:253} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=1
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
May be this helps
kubectl describe pod airflow
Name: airflow
Namespace: default
Priority: 0
Node: minikube/192.168.64.2
Start Time: Mon, 06 Apr 2020 09:33:11 -0500
Labels: run=airflow
Annotations: <none>
Status: Running
IP: 172.17.0.2
IPs:
IP: 172.17.0.2
Containers:
webserver:
Container ID: docker://f09249e339079acd78e2175ef690a853dd264664bf53ac826f743c46e83bc7b4
Image: anayyar82/airflow-aks1
Image ID: docker-pullable://anayyar82/airflow-aks1@sha256:7f459e5404ab93668ed3a9476ac02cda8acc691aae1b8b983e65611109c546be
Port: <none>
Host Port: <none>
Command:
airflow
webserver
State: Terminated
Reason: Error
Exit Code: 137
Started: Mon, 06 Apr 2020 09:33:30 -0500
Finished: Mon, 06 Apr 2020 09:33:53 -0500
Ready: False
Restart Count: 0
Environment:
AIRFLOW__CORE__LOAD_EXAMPLES: False
AIRFLOW__CORE__SQL_ALCHEMY_CONN: mysql://airflow:airflow@mysql.default/airflow
AIRFLOW__CORE__EXECUTOR: KubernetesExecutor
AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY: anayyar82/airflow-aks1
AIRFLOW__KUBERNETES__WORKER_CONTAINER_TAG: latest
AIRFLOW__KUBERNETES__DAGS_VOLUME_HOST: /tmp/airflow/dags
Mounts:
/tmp/airflow/dags from store (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-n8qt9 (ro)
scheduler:
Container ID: docker://8a6731a2e07caa9e0f58cbc26a94e7c897fc3354e2d8e593461db946c8fba763
Image: anayyar82/airflow-aks1
Image ID: docker-pullable://anayyar82/airflow-aks1@sha256:7f459e5404ab93668ed3a9476ac02cda8acc691aae1b8b983e65611109c546be
Port: <none>
Host Port: <none>
Command:
airflow
scheduler
State: Running
Started: Mon, 06 Apr 2020 09:33:30 -0500
Ready: True
Restart Count: 0
Environment:
AIRFLOW__CORE__LOAD_EXAMPLES: False
AIRFLOW__CORE__SQL_ALCHEMY_CONN: mysql://airflow:airflow@mysql.default/airflow
AIRFLOW__CORE__EXECUTOR: KubernetesExecutor
AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY: anayyar82/airflow-aks1
AIRFLOW__KUBERNETES__WORKER_CONTAINER_TAG: latest
AIRFLOW__KUBERNETES__DAGS_VOLUME_HOST: /tmp/airflow/dags
AIRFLOW__KUBERNETES__KUBE_CLIENT_REQUEST_ARGS:
Mounts:
/tmp/airflow/dags from store (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-n8qt9 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
store:
Type: HostPath (bare host directory volume)
Path: /tmp/airflow/dags
HostPathType:
default-token-n8qt9:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-n8qt9
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
sorry by just looking at the logs I am not quite sure what is the problem, maybe you could try running without k8s first?
Yeah without it's running fine. Could you please recheck if something is missing in the doc?
Seems airflow installation get stuck while running below command.
I see the airflow pod is in error and shows only below
running the code: