leoporoli
commented
12 months ago This is the error shown in the UI: Could not create an enclave, got: ConnectError: [unknown] NetworkError when attempting to fetch resource.
This is the request that didn't receive a response from the EM server
curl 'http://localhost:8081/kurtosis_enclave_manager.KurtosisEnclaveManagerServer/CreateEnclave' -X POST -H 'User-Agent: connect-es/0.13.2' -H 'Accept: */*' -H 'Accept-Language: en-GB,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br' -H 'Referer: http://localhost:9711/' -H 'connect-protocol-version: 1' -H 'content-type: application/json' -H 'Origin: http://localhost:9711/' -H 'Connection: keep-alive' -H 'Sec-Fetch-Dest: empty' -H 'Sec-Fetch-Mode: cors' -H 'Sec-Fetch-Site: same-site' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' --data-raw '{"enclaveName":"","apiContainerVersionTag":"","apiContainerLogLevel":"info","mode":"PRODUCTION"}'The engine server starts creating the enclave but for some reason, it doesn't finish.
These are the logs from the engine' GRPC server:
DEBU[2023-11-14T15:30:13Z][engine_connect_server_service.go:CreateEnclave] args: enclave_name:"" api_container_version_tag:"" api_container_log_level:"info" mode:PRODUCTION
DEBU[2023-11-14T15:30:13Z][docker_kurtosis_backend_enclave_functions.go:CreateEnclave] Creating Docker network for enclave 'be040fdc5efd47d7b8de96ebd25a9dae'...
DEBU[2023-11-14T15:30:13Z][docker_kurtosis_backend_enclave_functions.go:CreateEnclave] Docker network 'be040fdc5efd47d7b8de96ebd25a9dae' created successfully with ID 'b1cf10d6ef4ea4090a016215c52612df76e0b86d096010138889d8b73ae0079d'
DEBU[2023-11-14T15:30:13Z][docker_manager.go:FetchImage] Fetching image 'alpine:3.17' with image download mode: missing
DEBU[2023-11-14T15:30:13Z][docker_manager.go:getContainerHostConfig] Binds: [kurtosis-logs-collector-vol--be040fdc5efd47d7b8de96ebd25a9dae:/fluent-bit/etc]
DEBU[2023-11-14T15:30:13Z][docker_manager.go:CreateAndStartContainer] Created container with ID 'e8fabbbb801e3137adb25bff3f94322745b6ea3802b997ba4530118f94a7ac75' from image 'alpine:3.17'
DEBU[2023-11-14T15:30:14Z][fluentbit_configuration_creator.go:createFluentbitConfigFileInVolume] The Fluentbit config file with content '
[SERVICE]
log_level debug
http_server On
http_listen 0.0.0.0
http_port 9712
storage.path /fluent-bit/etc/storage/
[INPUT]
name forward
listen 0.0.0.0
port 9713
storage.type filesystem
[OUTPUT]
name forward
match *
host 172.17.0.2
port 9714
' was successfully added into the volume
DEBU[2023-11-14T15:30:14Z][docker_manager.go:FetchImage] Fetching image 'fluent/fluent-bit:1.9.7' with image download mode: missing
INFO[2023-11-14T15:30:14Z][docker_manager.go:pullImage] Pulling image 'fluent/fluent-bit:1.9.7'
DEBU[2023-11-14T15:30:23Z][log_file_manager.go:func2] Creating log file paths...
DEBU[2023-11-14T15:30:23Z][log_file_manager.go:func2] Successfully created log file paths. I tried to understand what was happening based on the logs from the engine I shared. Something that is strange for me is that the Engine GRPC server receives the request, even when it starts creating the enclave but for some reason, it doesn’t show anything after this line: Pulling image 'fluent/fluent-bit:1.9.7' doesn’t show any error and if something fails while it’s fetching the image we should see an error (I checked in the code it should be An error occurred fetching image '%v') So, the FE calls the EM, the EM calls the Engine, and the engine starts creating the enclave but then something happens (I’m not sure where I think that probably in the EM) and the Engine doesn’t show any error, the enclave is not created, and the FE shows an error (Could not create an enclave, got: ConnectError: [unknown] NetworkError when attempting to fetch resource.) because it doesn’t get a response from the EM.
Dartoxian
What's your CLI version?
0.85.18
Description & steps to reproduce
I am unable to create new enclaves in the em ui after I upgrade my local engine. These are the steps that I take:
sudo apt update && sudo apt install --only-upgrade kurtosis-clikurtosis enclave ls(just to trigger starting the engine with the new version and check it works)runbutton will spin, and eventually just stop spinning. No error is shown, but no enclave is created either.I can work around this issue by manually creating an enclave on the cli (
kurtosis run github.com/kurtosis-tech/postgres-package) - this seems to wake something up in my local instance and after this the web ui can be used to create enclaves.I run kurtosis on Ubuntu 22.
This issue was also seen on 0.85.10.
Desired behavior
After upgrading my engine I should be able to launch enclaves in the emui.
Additionally, the emui should show an error (rather than no message) when it does not create an enclave.
What is the severity of this bug?
Papercut; this bug is frustrating, but I have a workaround.
What area of the product does this pertain to?
Frontend: the Graphical User Interface, including the Enclave Manager