Runtime error: The dispatcher (i.e. the database management system) is turned off. Kubernetes

defyjoy commented 3 years ago

I was able to run the container for australia region and the docker container restarted after downloading entire db and the pod showed 1/1 on kubernetes . However when I check the logs and query for the api . I get strange error as -

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta http-equiv="content-type" content="text/html; charset=utf-8" lang="en"/>
  <title>OSM3S Response</title>
</head>
<body>

<p>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</p>
<p><strong style="color:#FF0000">Error</strong>: runtime error: The dispatcher (i.e. the database management system) is turned off. </p>

</body>
</html>

Traceback (most recent call last):
  File "data_task__by_nikhil.py", line 140, in <module>
    data_frame = extract_raw_data_from_OSM(query)
  File "data_task__by_nikhil.py", line 118, in extract_raw_data_from_OSM
    json_data = response.json()
  File "/usr/lib/python3.8/site-packages/requests/models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Below are the logs from the pod

It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
Update finished with status code: 3
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
10.240.0.115 - - [14/Nov/2020:13:22:49 +0000] "GET /api/status HTTP/1.1" 200 156 "-" "kube-probe/1.19"
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
10.240.0.115 - - [14/Nov/2020:13:22:59 +0000] "GET /api/status HTTP/1.1" 200 156 "-" "kube-probe/1.19"
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 1 to run the loop. Desired load is: 10%. Sleeping: 9
10.240.0.115 - - [14/Nov/2020:13:23:09 +0000] "GET /api/status HTTP/1.1" 200 156 "-" "kube-probe/1.19"
10.240.0.115 - - [14/Nov/2020:13:23:12 +0000] "GET /api/interpreter?data=%5Bout%3Ajson%5D%5Btimeout%3A50%5D%3B%28way%5B%22highway%22%5D%28around%3A50%2C-34.06221%2C150.82524%29%5Bmaxspeed%5D%3B%29%3Bout+body%3B HTTP/1.1" 504 635 "-" "python-requests/2.24.0"
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
10.240.0.115 - - [14/Nov/2020:13:23:19 +0000] "GET /api/status HTTP/1.1" 200 155 "-" "kube-probe/1.19"
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
runtime error: The dispatcher (i.e. the database management system) is turned off.
It took 0 to run the loop. Desired load is: 10%. Sleeping: 3

Can you please guide how and why is this happening and how to resolve this ?

wiktorn commented 3 years ago

Does your Kubernetes cluster allow creating of unix domain sockets? Can you post a response form query to /api/status ?

kmehta-nuro commented 3 years ago

Unrelated to your original question, but I am also trying to deploy on Kubernetes. Can you share your deployment.yaml? From the README, after initialization the Docker container stops — how did you start it up again? Kubernetes doesn't have anything like docker start

kap7475 commented 2 years ago

Yep I'm having the same issue saying the dispatcher is turned off. Helppppp ;( Can't seem to find anyone who has found a solution to this.

bellondr commented 9 months ago

I have the same issue I prepare data in pvc osm-data-pvc2

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    meta.helm.sh/release-name: overpass-api-server
    meta.helm.sh/release-namespace: osm
  labels:
    app.kubernetes.io/instance: overpass-api-server
    app.kubernetes.io/name: overpass-api-server
  name: overpass-api-server
  namespace: osm
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/instance: overpass-api-server
      app.kubernetes.io/name: overpass-api-server
  template:
    metadata:
      labels:
        app.kubernetes.io/instance: overpass-api-server
        app.kubernetes.io/name: overpass-api-server
    spec:
      containers:
      - name: overpass-api-server
        image: wiktorn/overpass-api
        imagePullPolicy: IfNotPresent
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 0
        resources:
          limits:
            cpu: "4"
            memory: "16Gi"
          requests:
            cpu: "4"
            memory: "16Gi"
        env:
        - name: OVERPASS_META
          value: "yes"
        - name: OVERPASS_MODE
          value: init
        - name: OVERPASS_PLANET_URL
          value: http://download.geofabrik.de/north-america/us-latest.osm.bz2
        - name: OVERPASS_RULES_LOAD
          value: "10"
        volumeMounts:
        - mountPath: "/db"
          name: osm-datadb
      volumes:
      - name: osm-datadb
        persistentVolumeClaim:
          claimName: osm-data-pvc2

error

bellondr commented 9 months ago

I successful deployed overpass-api in k8s cluster(aws eks). maybe overpass-api create unix domain sockets in /db/ dir. So s3 pvc is not work for overpass-api server I recomand to use efs pvc in aws eks cc @wiktorn

wiktorn / Overpass-API

Runtime error: The dispatcher (i.e. the database management system) is turned off. Kubernetes #53