search

canonical / microk8s

MicroK8s is a small, fast, single-package Kubernetes for datacenters and the edge.
https://microk8s.io
Apache License 2.0
8.51k stars 772 forks source link

Failed to enable kubeflow #1439

Closed lao-white closed 3 years ago

lao-white commented 4 years ago

inspection-report-20200728_120622.tar.gz

Couldn't contact api.jujucharms.com
Please check your network connectivity before enabling Kubeflow.
Failed to enable kubeflow
xelios commented 3 years ago

Same problem with Ubuntu 20.04.2 LTS (Focal Fossa) and microk8s v1.20.2 classic (but tested also with version 1.20/beta, edge and 1.19/stable):

Couldn't contact api.jujucharms.com from within the Kubernetes cluster

Iptables forward packets fine (and no firewall is active); testing connectivity works better:

kubectl exec dnsutils -it -- nslookup api.jujucharms.com Server: 10.152.183.10 Address: 10.152.183.10#53

Non-authoritative answer: Name: api.jujucharms.com Address: 162.213.33.78 Name: api.jujucharms.com Address: 162.213.33.79

Also tested with bundle lite or edge:

microk8s enable kubeflow --debug

Where is the problem?? Any suggestion? Help me, please.

Gabriele

inspection-report-20210308_150142.tar.gz

knkski commented 3 years ago

@xelios: Can you post the output from this command?

microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl api.jujucharms.com"
xelios commented 3 years ago

user@kubeflow:~$ microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl api.jujucharms.com" Error from server (InternalError): Internal error occurred: failed calling webhook "admission-webhook.kubeflow.org": Post "https://admission-webhook.kubeflow.svc:443/apply-poddefault?timeout=30s": service "admission-webhook" not found

kuppan4109 commented 3 years ago

@knkski Is there any perfect solution for this issue? Command : microk8s enable kubeflow Output : Enabling dns... Enabling storage... Enabling ingress... Enabling metallb:10.64.140.43-10.64.140.49... Waiting for other addons to finish initializing... Addon setup complete. Checking connectivity... Couldn't contact api.jujucharms.com Please check your network connectivity before enabling Kubeflow.

knkski commented 3 years ago

@xelios: Ah, it looks like there was an incomplete teardown of a previous Kubeflow deployment, as the admission-webhook shouldn't be deployed before the network check. The easiest way to fix this (assuming you don't have anything else in MicroK8s you need to keep) is to just reinstall the MicroK8s snap:

sudo snap remove microk8s --purge
sudo snap install microk8s --classic

Otherwise, you can list the existing mutating and validating webhooks with:

microk8s kubectl get mutatingwebhookconfigurations,validatingwebhookconfigurations

And then running microk8s delete on any webhooks with admission-webhook in the name.

knkski commented 3 years ago

@kuppan4109: Can you post the output from microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl api.jujucharms.com"?

You also might just need to do the same steps as above

knkski commented 3 years ago

@xelios: I have opened #2086, which should fix this issue

hfrechen commented 3 years ago

@knkski I get the same output as @kuppan4109 

Couldn't contact api.jujucharms.com
Please check your network connectivity before enabling Kubeflow.

I tried different stable / candidate builds from multiple versions but all fail with the same error message.

Output of the curl command, that you proposed, is listed below:

Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:7 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB]
Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [178 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [218 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [29.6 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1089 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [938 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [4301 B]
Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [681 kB]
Get:16 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [659 kB]
Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.6 kB]
Fetched 17.2 MB in 2s (7456 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Setting up libbrotli1:amd64 (1.0.7-6ubuntu0.1) ...
Setting up libsqlite3-0:amd64 (3.31.1-4ubuntu0.2) ...
Setting up libsasl2-modules:amd64 (2.1.27+dfsg-2) ...
Setting up libnghttp2-14:amd64 (1.40.0-1build1) ...
Setting up krb5-locales (1.17-6ubuntu4.1) ...
Setting up libldap-common (2.4.49+dfsg-2ubuntu1.7) ...
Setting up libkrb5support0:amd64 (1.17-6ubuntu4.1) ...
Setting up libsasl2-modules-db:amd64 (2.1.27+dfsg-2) ...
Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2build1) ...
Setting up libk5crypto3:amd64 (1.17-6ubuntu4.1) ...
Setting up libsasl2-2:amd64 (2.1.27+dfsg-2) ...
Setting up libroken18-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libkrb5-3:amd64 (1.17-6ubuntu4.1) ...
Setting up openssl (1.1.1f-1ubuntu2.2) ...
Setting up publicsuffix (20200303.0012-1) ...
Setting up libheimbase1-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libasn1-8-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libhcrypto4-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up ca-certificates (20210119~20.04.1) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Updating certificates in /etc/ssl/certs...
129 added, 0 removed; done.
Setting up libwind0-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libgssapi-krb5-2:amd64 (1.17-6ubuntu4.1) ...
Setting up libssh-4:amd64 (0.9.3-2ubuntu2.1) ...
Setting up libhx509-5-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libkrb5-26-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libheimntlm0-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libgssapi3-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libldap-2.4-2:amd64 (2.4.49+dfsg-2ubuntu1.7) ...
Setting up libcurl4:amd64 (7.68.0-1ubuntu2.4) ...
Setting up curl (7.68.0-1ubuntu2.4) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for ca-certificates (20210119~20.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://api.jujucharms.com/">here</a>.</p>
<hr>
<address>Apache/2.4.29 (Ubuntu) Server at api.jujucharms.com Port 80</address>
</body></html>
pod "connectivity-check" deleted
kuppan4109 commented 3 years ago

microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl api.jujucharms.com" @knkski Output of the Command : microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl api.juj ucharms.com" If you don't see a command prompt, try pressing enter. Reading package lists... Done N: See apt-secure(8) manpage for repository creation and user configuration deta ils. N: Updating from such a repository can't be done securely, and is therefore disa bled by default. E: The repository 'http://security.ubuntu.com/ubuntu focal-security InRelease' i s not signed. E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/InRele ase 403 Forbidden [IP: 23.185.0.4 80] N: See apt-secure(8) manpage for repository creation and user configuration deta ils. N: Updating from such a repository can't be done securely, and is therefore disa bled by default. E: The repository 'http://archive.ubuntu.com/ubuntu focal InRelease' is not sign ed. E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/InRelease 403 Forbidden [IP: 23.185.0.4 80] E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-updates/InReleas e 403 Forbidden [IP: 23.185.0.4 80] E: The repository 'http://archive.ubuntu.com/ubuntu focal-updates InRelease' is not signed. N: Updating from such a repository can't be done securely, and is therefore disa bled by default. N: See apt-secure(8) manpage for repository creation and user configuration deta ils. E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-backports/InRele ase 403 Forbidden [IP: 23.185.0.4 80] E: The repository 'http://archive.ubuntu.com/ubuntu focal-backports InRelease' i s not signed. N: Updating from such a repository can't be done securely, and is therefore disa bled by default. N: See apt-secure(8) manpage for repository creation and user configuration deta ils. pod "connectivity-check" deleted pod default/connectivity-check terminated (Error)

sakulh commented 3 years ago

Same problem. Any solution?

brettkoonce commented 3 years ago

Getting this with a clean install on 1.20 + 1.19 as well, 18.04 as the base.

rui-vas commented 3 years ago

@knkski would you know why this passed the tests?

brettkoonce commented 3 years ago

per discussion on slack with @RFMVasconcelos (thanks again!), i ran the following network test:

microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl https://api.jujucharms.com/charmstore/v5/istio-pilot-5/icon.svg"

with the following error:

dial tcp: lookup athena: Temporary failure in name resolution

In my case, the issue turned out to be that the local dns hostname didn't match the machine name (eg my machine name is athena, but i had the name athena-40g being assigned by dns), and then all sort of wierdness happened elsewhere/kubeflow was dying. Simply making the interface name match the host name resolved the issue and now all is working properly, thanks Rui!

kbegiedza commented 3 years ago

For me microk8s disable dns fixed thing.

xiaolougod commented 3 years ago

it seem python3 can not support https in https://api.jujucharms.com/charmstore/v5/istio-pilot-5/icon.svg. how to solve it?

achilles13 commented 3 years ago

Disconnecting from VPN while enabling kubeflow fixed this issue for me

mikel-brostrom commented 3 years ago

None of the solutions presented here worked for me.

The inspection report generated by microk8s.inspect gives me the following error under JUJU:

`ERROR` No controllers registered.

Please either create a new controller using "juju bootstrap" or connect to
another controller that you have been given access to using "juju register".

So I ran:

juju bootstrap

in order to create the controller. But I still get the

Couldn't contact api.jujucharms.com from within the Kubernetes cluster
Please check your network connectivity before enabling Kubeflow.

error When

microk8s enable kubeflow

elgiewindhu commented 3 years ago

I tried all the solutions (swapping the version of microk8s and deleting webhook and co.), but I still cannot enable kubeflow with same reason.

I'm behind the coorporate firewall and set the same proxies (including NO_PROXY) on etc/environment and var/snap/microk8s/current/args/containerd-env. I was able to do the command:

curl https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg,

That means there is connectivity for Ubuntu host. But I tried @knkski suggestion to run the check-connectivity() command inside the container with:

microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg"

But it seems that the container doesn't have any internet connection at all.

mavencode01 commented 3 years ago

@elgiewindhu Same issue we have, the containerd daemon is not using the host HTTP_PROXY and HTTPS_PROXY so it can't connect to the internet.

If you try setting the proxy in the connectivity-check ubuntu container, the container will work

microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash 

# inside the container now,
$ export http_proxy=....
$ export https_proxy=...
$ apt update && apt install -y curl && curl https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg

However, this proxy needs to injected into the container before with something like MutatingAdmissionWebhook.

I've tried to also inject the proxy ENV variables to the containers using k8s-env-injector but is not working in microk8s.

Hope someone can post a solution here !

xnuinside commented 3 years ago

sudo iptables -P FORWARD ACCEPT - helped me with same issue, maybe it can help someone

filiphagan commented 3 years ago

same issue on v1.22.3 sudo iptables -P FORWARD ACCEPT didn't work for me

banderlog commented 3 years ago

sudo iptables -P FORWARD ACCEPT Did not work for me, but reinstalling the microk8s and kubeflow did.

But, just for a single run, if you will try to stop microk8s and start again -- nothing will work

UPD: microk8s reset also works

Arthur-Matta commented 2 years ago

sudo iptable -p FOWARD ACCEPT didn't work for me also, at least not by iteself.

After some inspection I found out the problem was with name resolution between the nodes. I have a 3-nodes cluster and although every kubectl get <resource> show no sign of problems, the nodes couldn't properly communicate with each other as I found out when trying to get logs from a pod running on another node:

$ microk8s kubectl logs coredns-7f9c69c78c-q2dzz -n kube-system Error from server: Get "[...]": dial tcp: lookup kubeworker1: Temporary failure in name resolution

I executed microk8s inspect on all nodes. Only one reported a connectivity problems and suggested running sudo iptable -p FOWARD ACCEPT. I executed this command on all nodes. The problem persisted. I came accross a suggestion to edit the /etc/hosts file and It was only then that everything started working fine and the command microk8s enable kubeflow was successfull. For each node in my cluster, I had to edit the /etc/hosts file and add the ip of the other nodes in the cluster.

P.S.: running microk8s v1.21/stable on ubuntu 20.04

Ashok93 commented 2 years ago

per discussion on slack with @RFMVasconcelos (thanks again!), i ran the following network test:

microk8s.kubectl run --rm -it --restart=Never --image=ubuntu connectivity-check -- bash -c "apt update && apt install -y curl && curl https://api.jujucharms.com/charmstore/v5/istio-pilot-5/icon.svg"

with the following error:

dial tcp: lookup athena: Temporary failure in name resolution

In my case, the issue turned out to be that the local dns hostname didn't match the machine name (eg my machine name is athena, but i had the name athena-40g being assigned by dns), and then all sort of wierdness happened elsewhere/kubeflow was dying. Simply making the interface name match the host name resolved the issue and now all is working properly, thanks Rui!

Thanks! I also had issue with name resolution within the cluster nodes.

mohitv5 commented 2 years ago

Hello,

I am facing the same issue. I am installing Kubeflow on CentOS 7. microk8s 1.21.9 installs successfully and dns, dashboard and storage get enabled successfully. However, while enabling Kubeflow, I get the following error:

Enabling dns... Enabling storage... Enabling ingress... Enabling metallb:10.64.140.43-10.64.140.49... Waiting for other addons to finish initializing... Addon setup complete. Checking connectivity... Couldn't contact api.jujucharms.com Please check your network connectivity before enabling Kubeflow.

I am successfully able to connect to api.jujucharms.com using curl https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg

I disabled firewall on CentOS, changed iptables settings to FORWARD ACCEPT, tried version 1.20 and 1.18 of microk8s, but still not able to go past this error.

inspection-report-20220208_154232.tar.gz This is the inspection report.

Any help is highly appreciated.

hfrechen commented 2 years ago

Hello,

I am facing the same issue. I am installing Kubeflow on CentOS 7. microk8s 1.21.9 installs successfully and dns, dashboard and storage get enabled successfully. However, while enabling Kubeflow, I get the following error:

Enabling dns... Enabling storage... Enabling ingress... Enabling metallb:10.64.140.43-10.64.140.49... Waiting for other addons to finish initializing... Addon setup complete. Checking connectivity... Couldn't contact api.jujucharms.com Please check your network connectivity before enabling Kubeflow.

I am successfully able to connect to api.jujucharms.com using curl https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg

I disabled firewall on CentOS, changed iptables settings to FORWARD ACCEPT, tried version 1.20 and 1.18 of microk8s, but still not able to go past this error.

inspection-report-20220208_154232.tar.gz This is the inspection report.

Any help is highly appreciated.

I was facing the same issue last year, also running a CentOS 7 VM and thought at that time, the connection issue may be caused by some internal network restrictions. But recently I tried the same setup on a Debian 11 VM (same internal network) and installation finished without any problems. So maybe it has something to do with the OS?

M-I-Dx commented 2 years ago

I am also getting the same error but I can even access https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg using curl. Any idea why?

skabbit commented 2 years ago

Got the same issue. Tried 1.18/stable, full reinstall, iptable hack - nothing helps. https://github.com/ubuntu/microk8s/issues/url gives stable 404 response. Any idea?

mohamedniyaz1996 commented 2 years ago

I am too facing same error with 1.18,1.19,1.20 versions of microk8s.

Error on enabling kubeflow addon:

$ microk8s.enable kubeflow
Enabling dns...
Enabling storage...
Enabling ingress...
Enabling metallb:10.64.140.43-10.64.140.49...
Waiting for other addons to finish initializing...
Addon setup complete. Checking connectivity...
Couldn't contact api.jujucharms.com
Please check your network connectivity before enabling Kubeflow.
Failed to enable kubeflow

microk8s inspect:

$  microk8s inspect
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-flanneld is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-apiserver is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Service snap.microk8s.daemon-proxy is running
  Service snap.microk8s.daemon-kubelet is running
  Service snap.microk8s.daemon-scheduler is running
  Service snap.microk8s.daemon-controller-manager is running
  Service snap.microk8s.daemon-etcd is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy openSSL information to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster

Building the report tarball
  Report tarball is at /var/snap/microk8s/2647/inspection-report-20220217_212519.tar.gz

ping api.jujucharms.com is failing with packets not being received.

$ ping api.jujucharms.com

--- api.jujucharms.com ping statistics ---
17 packets transmitted, 0 received, 100% packet loss, time 16389ms

curl -v https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg seems to be not found error as the url is no longer valid it seems.

$ curl -v https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg
*   Trying 162.213.33.79...
* TCP_NODELAY set
* Connected to api.jujucharms.com (162.213.33.79) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=api.jujucharms.com
*  start date: Jan 13 06:20:36 2022 GMT
*  expire date: Apr 13 06:20:35 2022 GMT
*  subjectAltName: host "api.jujucharms.com" matched cert's "api.jujucharms.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET /charmstore/v5/~kubeflow-charmers/ambassador-88/icon.svg HTTP/1.1
> Host: api.jujucharms.com
> User-Agent: curl/7.58.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/1.1 404 NOT FOUND
< Date: Thu, 17 Feb 2022 21:28:25 GMT
< Server: gunicorn
< content-type: text/html
< content-length: 232
< snap-store-version: 46
< x-vcs-revision: ceb1c973
< x-request-id: A2D5214FBEEC0A325D7701BB620EBDF919FC12F2
< 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>
* Connection #0 to host api.jujucharms.com left intact

Any idea on how to overcome this?

realLyans commented 2 years ago

Anyone got solution for this? On Microk8s 1.21 and facing the same issue

skabbit commented 2 years ago

It is still an issue, but as I believe it is not Microk8s problem, but Kubeflow. Very unstable project, too young.

realLyans commented 2 years ago

I just get very frustrated after trying so many different ways to deploy kubeflow and none of them work... I guess maybe it is not the simple solution I am looking for

M-I-Dx commented 2 years ago

So my problem was solved by using this https://charmed-kubeflow.io/docs/quickstart tutorial. From what I can tell the old tutorial doesn't work anymore.

realLyans commented 2 years ago

So my problem was solved by using this https://charmed-kubeflow.io/docs/quickstart tutorial. From what I can tell the old tutorial doesn't work anymore.

I actually tried that this morning and it works all till the end but I cannot access my dashboard through the web...