Closed tarockey closed 2 years ago
@tarockey thanks for raising! There appears to be a bug in conda - we are looking into a workaround.
@tarockey this has now been fixed. We changed the deployment to be a custom docker build - feast has dependencies on gcc and unixodbc-dev that needed to be installed on the docker image. The bug in Conda caused pip to hang when it hit the install error.
Summary: When attempting to create the conda environment "feast-env" as found in the training and deployment notebook, the environment creation stalls out after an hour during "installing pip dependencies" and never completes.
Details: First occurrence of this issue was during attempts to deploy a scoring container, which timed out after an hour. Looking at the logs, it simply stops at the line "Installing pip dependencies: ... working ...", with no further detail.
Attempting to build the conda environment from the "model_service_env.yml" in a local terminal fail in the same spot. Manually removing the pip packages from the yml file and manually installing those packages in the conda environment does not fail. Also, removing just the "feast-azure-provider" package causes to build to succeed, and it can then be manually installed. However, since Azure ML does not allow the pip installs to be done in separate steps, this is not a viable workaround for the container deployment.