kantancoding / microservices-python

262 stars 161 forks source link

Mysql cannot be connected using minikube's host.minikube.internal #16

Closed kshitizJ closed 1 year ago

kshitizJ commented 1 year ago

Error Image:

error

My server file image:

pythonfile

My auth-deployment file image:

auth-deployment-file

My auth-service file image:

auth-service

My auth-configmap file image:

auth-configmap

My Docker file for auth:

auth-docker-file

My all pods are running successfully:

pods running

I don't know what is the error. When id do systemctl status mysql it shows that my mysql is running successfully in my local device. My operating system is Ubuntu 22.04 LTS.

Please help me to resolve the issue.

kshitizJ commented 1 year ago

Ok, I will try this and let you know.

Thank you!

kshitizJ commented 1 year ago

I did what you said and I am getting this error:

Screenshot from 2022-12-02 20-06-10

kantancoding commented 1 year ago

Come on bro. At least attempt to fix it on your own. You aren't going to learn anything if you ask for the fix before even trying to figure out what the issue is. This is a pretty straightforward error. Don't take it the wrong way. I just want you to actually learn something πŸ˜‰

kshitizJ commented 1 year ago

I did some research but was unsuccessful.

What I did is, I updated the mysql pod and wrote MYSQL_ALLOW_EMPTY_PASSWORD to yes: Screenshot from 2022-12-02 23-42-27

Then I deleted the mysql yaml file by kubectl delete -f ./ and again ran kubectl delete -f ./

Then I started mysql at port 3306 (I don't know if it was necessary or not): Screenshot from 2022-12-02 23-42-43

Then I deleted all the manifest yaml file in auth using command kubectl delete -f ./ and made changes to config.yaml file as you suggested: Screenshot from 2022-12-02 23-43-38

Then I forwarded the port of auth pod: Screenshot from 2022-12-02 23-43-53

Then I made request from the command line but was unsuccessful again: Screenshot from 2022-12-02 23-50-49

By checking the logs I saw that I have same issue i.e. MySQLdb.OperationalError: (2005, "Unknown MySQL server host 'mysql' (-3)")

kantancoding commented 1 year ago

Can you use the tool I suggested in the video k9s. and check if the pod is running? You should see a pod with the Name of mysql. If not, something went wrong with kubectl apply. So Run it again on the mysql dir and show me the output

kshitizJ commented 1 year ago

I use kube lens application to check if the pod is running or not.

Here is the image of mysql that is running: Screenshot from 2022-12-03 14-16-08

kantancoding commented 1 year ago

Then I started mysql at port 3306 (I don't know if it was necessary or not):

Wait, I just noticed. why are you port-forwarding mysql? Please read the doc about what port-forward does. You are trying to connect to the mysql pod that is inside of the cluster with another pod that is inside of the cluster... So why port-forward?

kshitizJ commented 1 year ago

It is still not working without port-forwarding.

I am getting the same error.

kantancoding commented 1 year ago

So you can see that the mysql pod is there. And that the name is in fact mysql. And the error is that the mysql host with the name mysql doesn't exist. So seems to me you need to research how pods can communicate with other pods internally (within a kubernetes cluster).

Do some troubleshooting e.g. try to connect to the mysql db from your local by using port forwarding.. You need to try things.

That should help you figure out your issue. I think at this point, I've provided a lot of guidance but I think I'd be doing you a disservice if I keep walking you through every step so I'm going to let you figure this one out on your own. When you solve the problem, please let me know as I'll be happy to hear about it. But other than that, I'm going to let you troubleshoot on your own and learn.

kshitizJ commented 1 year ago

Ok, I will let you know once I troubleshoot the problem.

Thanks once again for the guidance.

kantancoding commented 1 year ago

Hey bro, any update? What have you been trying so far?

kshitizJ commented 1 year ago

Yes.

What I tried was, here on this page in the last section it says that if we are using mysql in linux than we have to disable the appArmor.

So first I check if there is something known as appArmor or not in my system and I found out that I don't have appArmor which is enabled by default.

So I disable the appArmor and again tried using host.minikube.internal for accessing the local mysql but it didn't connect.

Than I tried the mysql host which is running in the container but still no luck.

Should I upgrade my minikube to the latest version i.e. 1.28.0? As we can see downgrading the minikube version doesnot solves the problem. So I think that minikube version has nothing to do with host.minikube.internal.

kshitizJ commented 1 year ago

apiVersion: v1 kind: ConfigMap metadata: name: auth-configmap data: MYSQL_HOST: host.minikube.internal MYSQL_USER: root MYSQL_DB: auth MYSQL_PORT: '3306'

kubectl port-forward auth-67f87b9dc6-s7jbg 5000:5000 &

curl -X POST 'http://127.0.0.1:5000/login' -u "trump@gmail.com:admin123"

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VybmFtZSI6InRoZW9kb25kcmVAZ21haWwuY29tIiwiZXhwIjoxNjY5ODUxOTY2LCJpYXQiOjE2Njk3NjU1NjYsImFkbWluIjp0cnVlfQ.pOSQEACZrnni4NVP9WDr6pO1o6iK0ZWRPPYQWPdsQiI%

Which OS are you using @donwany

donwany commented 1 year ago

MAC OS

kantancoding commented 1 year ago

Than I tried the mysql host which is running in the container but still no luck.

What do you mean by no luck here? What happened?

kshitizJ commented 1 year ago

MAC OS

ok

kshitizJ commented 1 year ago

Than I tried the mysql host which is running in the container but still no luck.

What do you mean by no luck here? What happened?

Same error. MySQLdb.OperationalError: (2005, "Unknown MySQL server host 'mysql' (-3)")

andreariba commented 1 year ago

I got the same issue on πŸ˜„ minikube v1.28.0 on Ubuntu 22.04

andreariba commented 1 year ago

the workaround on AppArmor from https://github.com/kubernetes/minikube/issues/7401 and https://github.com/moby/moby/issues/7512#issuecomment-61787845 did not work for me and this issue could be linked to this https://stackoverflow.com/questions/50952240/connect-to-local-database-from-inside-minikube-cluster, not sure ...

lewiswon commented 1 year ago

I got the same issue on πŸ˜„ minikube v1.28.0 on Ubuntu 22.04

Your error may not same with this one, I have same error with you, I tried #4 method, and now is work.

andreariba commented 1 year ago

already tried not working .. i stopped trying and dropped the couse because apparently on ubuntu minikube is behaving very different than on mac os, so it requires some details than I cannot figure out ...

I got the same issue on πŸ˜„ minikube v1.28.0 on Ubuntu 22.04

Your error may not same with this one, I have same error with you, I tried #4 method, and now is work.

juanfrilla commented 1 year ago

@andreariba or @kshitizJ did you finally solve it? I have the same error Mysql connection cursor error on cur = mysql.connection.cursor() TypeError: 'str' object cannot be interpreted as an integer And when I cast the int, now my error is MySQLdb._exceptions.OperationalError: (2002, "Can't connect to MySQL server on 'host.minikube.internal' (115)")

andreariba commented 1 year ago

the course is basically mac os specific, and there are too many differences in the configuration of minikube and mysql for ubuntu, if you find a solution let me know

@andreariba or @kshitizJ did you finally solve it? I have the same error Mysql connection cursor error on cur = mysql.connection.cursor() TypeError: 'str' object cannot be interpreted as an integer And when I cast the int, now my error is MySQLdb._exceptions.OperationalError: (2002, "Can't connect to MySQL server on 'host.minikube.internal' (115)")

knowbits commented 1 year ago

Same error. MySQLdb.OperationalError: (2005, "Unknown MySQL server host 'mysql' (-3)")

Been having the same MySQL connection problems, @kshitizJ.

Finally got it working, the JWT is now returned (!) when calling: curl -X POST http://mp3converter.com/login -u georgio@email.com:Admin123

I suspect that trying to access a MySQL db installed in Windows/WSL/Ubuntu using "host.minikube.internal" might be the culprit. Solution: Run MySQL in Kubernetes as @selikapro suggested.

  1. Follow the instructions above by @selikapro and run mysql in the cluster, so create the "pod.yaml" as described in the post above. Also change auth/manifests/configmap.yaml as he describes there.
  2. NOTE: There is typo in that "pod.yaml", so change "Aauth123" to "Auth123".
  3. Then create a "service.yaml" file to expose the mysql pod as a service, so that it can be called using "mysql" from other pods in the cluster. See the "service.yaml" below.
  4. I also had to make one more change in the "init.sql" part of "pod.yaml": replace 'localhost' with '172.17.0.1'

PS! You might need to add a label to "pod.yaml", to be able to refer to it from "service.yaml":

"pod.yaml"

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  # NOTE: Added the following 2 lines: 
  labels:
    app: mysql
spec:
  containers:
  - name: mysql
    image: mysql        
    ports:
      - containerPort: 3306
    volumeMounts:
      - name: mysql-initdb
        mountPath: /docker-entrypoint-initdb.d
  volumes:
    - name: mysql-initdb
      configMap:
        name: mysql-initdb-config
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-initdb-config
data:
  initdb.sql: |
    CREATE USER 'auth_user'@'localhost' IDENTIFIED BY 'Auth123';
    CREATE DATABASE auth;
    GRANT ALL PRIVILEGES ON auth.* TO 'auth_user'@'localhost';
    USE auth;
    CREATE TABLE user (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, email VARCHAR(255) NOT NULL UNIQUE, password VARCHAR(255) NOT NULL);
    INSERT INTO user (email, password) VALUES ('georgio@email.com', 'Admin123');

The "service.yaml":

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  selector:
    app: mysql
  type: ClusterIP
  ports:
    - port: 3306
      targetPort: 3306
      protocol: TCP

The updated "auth/manifests/configmap.yaml"

apiVersion: v1
kind: ConfigMap
metadata:
  name: auth-configmap
data:
  MYSQL_HOST: mysql
  MYSQL_USER: auth_user
  MYSQL_DB: auth
  MYSQL_PORT: "3306"

Useful: Connect "mysql" tool to the Kubernetes pod

kantancoding commented 1 year ago

the course is basically mac os specific, and there are too many differences in the configuration of minikube and mysql for ubuntu, if you find a solution let me know

@andreariba or @kshitizJ did you finally solve it? I have the same error Mysql connection cursor error on cur = mysql.connection.cursor() TypeError: 'str' object cannot be interpreted as an integer And when I cast the int, now my error is MySQLdb._exceptions.OperationalError: (2002, "Can't connect to MySQL server on 'host.minikube.internal' (115)")

The fact that the entire architecture is run in a vm (minikube is basically just a virtual machine) means that the course is inherently not OS specific. The issue that people are experiencing here is that they can't connect to their localhost from within minikube. I provided the MacOS solution for this in the course and I assumed the solution would be similar for other operating systems but it seems that is not the case.

For that reason, I provided an alternative solution that allows you to run mysql within the cluster which @knowbits was able to successfully implement and he also provided a very well written detailed explanation of how. A course is meant to offer guidance for others to learn from. In programming, when doing literally anything, you will run into issues that require you to figure out ways to resolve them. This is where you learn the most.

Things in this field are very rarely a,b,c 1,2,3 and DONE without issues. Especially when somebody makes a 5 hour course that attempts to teach people how to build an entire architecture. The more complex something is, the more likely there are to be issues.

But I still think that everybody here can solve their issues by simply spinning up a mysql pod within the cluster. And even if you can't get it working, I still hope that you were able to learn something.

Also, the error TypeError: 'str' object cannot be interpreted as an integer has no relation to this issue. It's a completely different error.

juanfrilla commented 1 year ago

@knowbits thanks! Followed all the steps and it worked πŸ˜‰

knowbits commented 1 year ago

Great! It's such a good tutorial, so it's nice to be able to complete it πŸ˜…

juanfrilla commented 1 year ago

@knowbits I still have not been able to complete it, I'm almost done but struggling with the MongoDB stuff πŸ˜… I'm wondering if we need to create some others yml files for mongo configuration because I don't know if it would give error of "host.minikube.internal" unrecognized host or something like that

knowbits commented 1 year ago

I'm wondering if we need to create some others yml files for mongo configuration

Yes, I also had a full stop because of MongoDB being installed on localhost.

So, my plan is to run it in the cluster, same as we had to do with MySQL.

Let me know if you create yaml files to run MongoDB as a pod in Kubernetes.

juanfrilla commented 1 year ago

@knowbits finally could implement my MongoDB pod in kubernetes creating 4 yml files; a persistent volume, persistent volume claim, a service and a deployment following this tutorial https://levelup.gitconnected.com/deploy-your-first-flask-mongodb-app-on-kubernetes-8f5a33fa43b4 And changed "host.minikube.internal" by "mongo" in the mongo-related python code Now the upload endpoint is working and there I stayed, tomorrow I will continue

knowbits commented 1 year ago

Thanks for sharing, @juanfrillaπŸ™ Will give it a try tomorrow.

kshitizJ commented 1 year ago

This is for Linux Ubuntu 22.04 LTS

Hey, I got the JWT by making mysql cluster in kubernetes.

I followed @knowbits but failed to load mysql cluster because it was giving me errors.

Here is my mysql files for making a cluster:

deployment.yaml - for mysql

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  selector:
    app: mysql
  type: ClusterIP
  ports:
    - port: 3306
      targetPort: 3306
      protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - image: mysql
          name: mysql
          env:
            # Use secret in real usage
            - name: MYSQL_ROOT_PASSWORD
              value: password
          ports:
            - containerPort: 3306
              name: mysql
          volumeMounts:
            - name: mysql-persistent-storage
              mountPath: /var/lib/minesql
      volumes:
        - name: mysql-persistent-storage
          persistentVolumeClaim:
            claimName: mysql-pv-claim

volume.yaml - for mysql

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-pv-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 4Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 4Gi

I will also share my updated configmap.yaml and secret.yaml files for auth service

configmap.yaml - for auth

apiVersion: v1
kind: ConfigMap
metadata:
  name: auth-configmap
data:
  MYSQL_HOST: mysql
  MYSQL_USER: root
  MYSQL_DB: auth
  MYSQL_PORT: "3306"

secret.yaml - for auth

apiVersion: v1
kind: Secret
metadata:
  name: auth-secret
stringData:
  MYSQL_PASSWORD: password
  JWT_SECRET: sarcasm
type: Opaque

Now I can follow along with the tutorial. Thank you for your help. If I face any issue ahead then I will open new issue thread.

knowbits commented 1 year ago

@knowbits finally could implement my MongoDB pod in kubernetes

Thanks @juanfrilla! I followed the instructions and now mongodb works as well.

Here are the details of what is needed to run mongodb in Kubernetes.

kantancoding commented 1 year ago

You guys are great! Congrats πŸ˜„