search

recro / medulla

A Cloud Native Low-Code Platform
https://recro.github.io/docs/
Apache License 2.0
6 stars 4 forks source link

When Trying to run `tye deploy` during local development in order to install medulla on my local running cluster, it will try to push images to main image package and fails #263

Closed keithlogan94 closed 1 year ago

keithlogan94 commented 1 year ago

Is there an existing issue for this?

Describe the bug

When Trying to run tye deploy during local development in order to install medulla on my local running cluster, it will try to push images to main image package and fails.

Expected behavior

I want to be able to install medulla on my local running cluster for development, but it fails.

Steps to reproduce

Setup a local cluster to deploy medulla onto

tye deploy

Exceptions or error messages (if any)

    Loading Application Details...
    Verifying kubectl installation...
    Verifying kubectl connection to cluster...
    Processing Service 'shell'...
        Applying container defaults...
        Compiling Services...
        Publishing Project...
        Building Docker Image...
                #1 [internal] load build definition from Dockerfile
                #1 sha256:baabd500310b0c088fe8424e48c64c42930a97d137b197e0504a3c1c3e53aca6
                #1 transferring dockerfile: 157B done
                #1 DONE 0.0s

                #2 [internal] load .dockerignore
                #2 sha256:6471cace54b54b7d1372f23576df96b548f247b734f1ea52c343621243437f9e
                #2 transferring context: 2B done
                #2 DONE 0.0s

                #3 [internal] load metadata for mcr.microsoft.com/dotnet/aspnet:6.0
                #3 sha256:ac4494cbca04ddb415c76edcbcc7688784c2a6ea65dd656286c013738aa3b75f
                #3 DONE 1.0s

                #4 [1/3] FROM mcr.microsoft.com/dotnet/aspnet:6.0@sha256:99d7ef1ce989ace31dfef826a3e702606f19774665acd1d00cd3d29f3e6e3149       
                #4 sha256:1831754be27949f58db1bd6a34904cb89567d681837dcb6d87fd341e550f1f22
                #4 resolve mcr.microsoft.com/dotnet/aspnet:6.0@sha256:99d7ef1ce989ace31dfef826a3e702606f19774665acd1d00cd3d29f3e6e3149 done     
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 0B / 31.40MB 0.1s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 0B / 15.17MB 0.1s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 0B / 31.63MB 0.1s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 2.10MB / 15.17MB 0.2s
                #4 sha256:99d7ef1ce989ace31dfef826a3e702606f19774665acd1d00cd3d29f3e6e3149 1.82kB / 1.82kB done
                #4 sha256:9bbfbed0f66ed02e8de90ec46e45ea46367b1263604f6d9869b0e7a2a21c65db 1.37kB / 1.37kB done
                #4 sha256:fd48163689297fd6096083525da1a3caebacc60ab5110a3fdd7b59ed952042d4 3.25kB / 3.25kB done
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 3.15MB / 15.17MB 0.3s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 4.19MB / 15.17MB 0.4s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 2.10MB / 31.40MB 0.5s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 5.24MB / 15.17MB 0.5s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 2.10MB / 31.63MB 0.5s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 6.29MB / 15.17MB 0.6s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 4.19MB / 31.40MB 0.7s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 7.34MB / 15.17MB 0.7s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 8.39MB / 15.17MB 0.8s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 4.19MB / 31.63MB 0.8s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 6.29MB / 31.40MB 0.9s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 9.44MB / 15.17MB 0.9s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 10.49MB / 15.17MB 1.0s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 6.29MB / 31.63MB 1.0s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 8.39MB / 31.40MB 1.1s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 11.53MB / 15.17MB 1.1s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 12.58MB / 15.17MB 1.2s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 8.39MB / 31.63MB 1.2s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 10.49MB / 31.40MB 1.3s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 13.63MB / 15.17MB 1.3s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 14.68MB / 15.17MB 1.4s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 10.49MB / 31.63MB 1.4s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 12.58MB / 31.40MB 1.5s
                #4 sha256:7ed415b4bd19c2b83ef768757b22c5156111db042fd62be4263ba200b4c0c8d0 15.17MB / 15.17MB 1.4s done
                #4 sha256:7b9388913c3cc3dacffa41ae2bb30c18b54cc5f522fa6ef2faacf48b0dff6020 0B / 156B 1.5s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 12.58MB / 31.63MB 1.6s
                #4 sha256:7b9388913c3cc3dacffa41ae2bb30c18b54cc5f522fa6ef2faacf48b0dff6020 156B / 156B 1.5s done
                #4 sha256:871ef3419da3410a47aa97b7655d8543add053e27cac5c5922ff3ee1f75793cd 0B / 9.46MB 1.6s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 14.68MB / 31.40MB 1.7s
                #4 ...

                #6 [internal] load build context
                #6 sha256:c7761bcdf39fe1c9ecc8e31d38b96a2a1c8647e2da9b9124956d0c0a701a9a23
                #6 transferring context: 39.47MB 1.8s done
                #6 DONE 1.9s

                #4 [1/3] FROM mcr.microsoft.com/dotnet/aspnet:6.0@sha256:99d7ef1ce989ace31dfef826a3e702606f19774665acd1d00cd3d29f3e6e3149       
                #4 sha256:1831754be27949f58db1bd6a34904cb89567d681837dcb6d87fd341e550f1f22
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 16.78MB / 31.40MB 1.9s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 15.73MB / 31.63MB 1.9s
                #4 sha256:871ef3419da3410a47aa97b7655d8543add053e27cac5c5922ff3ee1f75793cd 2.10MB / 9.46MB 1.9s
                #4 sha256:871ef3419da3410a47aa97b7655d8543add053e27cac5c5922ff3ee1f75793cd 4.19MB / 9.46MB 2.0s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 20.97MB / 31.40MB 2.2s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 19.92MB / 31.63MB 2.2s
                #4 sha256:871ef3419da3410a47aa97b7655d8543add053e27cac5c5922ff3ee1f75793cd 6.29MB / 9.46MB 2.2s
                #4 sha256:871ef3419da3410a47aa97b7655d8543add053e27cac5c5922ff3ee1f75793cd 7.34MB / 9.46MB 2.3s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 23.07MB / 31.40MB 2.4s
                #4 sha256:357910a178d4d646acde36bf2c2a95fc59893a78c92a56a47ef8cc89627f850a 22.02MB / 31.63MB 2.4s
                #4 sha256:871ef3419da3410a47aa97b7655d8543add053e27cac5c5922ff3ee1f75793cd 8.39MB / 9.46MB 2.4s
                #4 sha256:31b3f1ad4ce1f369084d0f959813c51df0ca17d9877d5ee88c2db6ff88341430 27.26MB / 31.40MB 2.6s
                #8 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
                #8 exporting layers 0.1s done
                #8 writing image sha256:9ef25afea2bd7a10fef4f249755cc689a45efa65f1518c0cdcfe507b2245b0ff done
                #8 naming to ghcr.io/recro/medulla/shell:0.1.0 done
                #8 DONE 0.1s

                Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
            Created Docker Image: 'ghcr.io/recro/medulla/shell:0.1.0'
        Pushing Docker Image...
                denied: permission_denied: write_package
    Drats! 'deploy' failed:
            'docker push' failed.

Version number

0.1.0

Anything else?

No response

keithlogan94 commented 1 year ago

Main reason is because of permission denied

denied: permission_denied: write_package
AndrewBabbitt97 commented 1 year ago

Error is coming from docker, are you logged into GHCR?

keithlogan94 commented 1 year ago

Can we use tilt in addition to the for local development? I've already started configuring it and it works nicely. Also, it automatically applies changes to the helm chart

keithlogan94 commented 1 year ago

Should open source developers be able to overwrite our package registry in order to run medulla locally?

AndrewBabbitt97 commented 1 year ago

Tilt was removed in #38 when Tye was added. We moved to Tye due to it being more resource friendly as not everyone has decent machines to do development on. This is a larger concern on windows machines as a whole, as containers running on WSL using significantly more memory than Mac and Linux machines due to the nature of how Hyper-V reserves memory. This is only made worse when running a whole cluster on top of the medulla containers and potential auxiliary services such as Istio.

General direction has been do decrease the development footprint over time to make it easier for people to onboard with little to no k8s knowledge.

As for the package registry, there's no need for you to configure a registry if you are using Rancher Desktop as is recommended in the documentation: https://recro.github.io/docs/get-started/contributing

Rancher Desktop shares the container images between docker and k8s, meaning there is no need to push to a registry at all. As all built containers are directly available to containerd and dockerd inside of Rancher Desktop, this also means you can directly deploy the helm chart on the local k8s cluster. We don't expect general devs need to understand k8s and helm charts in order to develop code. A simple tye run --watch should suffice for most people.

keithlogan94 commented 1 year ago

Oh okay. Tye works well for local development when not testing the helm charts, but one thing that tilt has to its advantage is that any changes to the helm chart, or code will automatically rebuild both the images, and deploy changes to the helm chart. I think it would be nice to provide tilt as an additional workflow for development.

When developing in tye on helm charts, firstly, you have to apply all changes to the helm chart manually, and tye run will not work with a local kubernetes cluster as that would just run the processes directly on the host.

AndrewBabbitt97 commented 1 year ago

Oh okay. Tye works well for local development when not testing the helm charts, but one thing that tilt has to its advantage is that any changes to the helm chart, or code will automatically rebuild both the images, and deploy changes to the helm chart. I think it would be nice to provide tilt as an additional workflow for development.

When developing in tye on helm charts, firstly, you have to apply all changes to the helm chart manually, and tye run will not work with a local kubernetes cluster as that would just run the processes directly on the host.

Generally we can split the development into two areas:

  1. Code development
  2. Chart development

Tilt is fantastic for the second scenario, where you want to iterate on a helm chart as it adds a hot reload capability to the inner development loop. For the first scenario hot reload is preferred using Tye for the reasons I have already listed on my previous comment and just overall being faster than redeploying containers on k8s frequently, but also the ability to easily attach a debugger to running processes on the local machine.

Very open to improving the second scenario, as we already have a good development story for the first scenario. Open to adopting a Tiltfile under the contrib/ or chart/ directories with a focus on improving chart development workflows. tye build can be used to build containers, or even the new container build support in the .NET 7 SDK. More info can be found in this blog post.

keithlogan94 commented 1 year ago

Okay, I like using tilt for helm charts, and tye for everything else.