snowflakedb / snowflake-ml-python

Apache License 2.0
38 stars 8 forks source link

Update conda-libmamba-solver and conda versions in conda.blz #44

Closed skupr-anaconda closed 11 months ago

skupr-anaconda commented 11 months ago

We found that the latest version of snowflake-ml-python cannot be built by conda-build 3.26.1 (with conda 23.7.4, conda-libmamba-solver 23.7.0) because your conda.blz has a hardcoded version 23.1.0 for conda and conda-libmamba-colver https://github.com/snowflakedb/snowflake-ml-python/blob/main/third_party/rules_conda/conda.bzl#L4-L8. As a result we got an error:

Starting local Bazel server and connecting to it...
Loading: 
Loading: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: snowflake/ml
Loading: 0 packages loaded
    currently loading: snowflake/ml
Analyzing: target //snowflake/ml:wheel (1 packages loaded, 0 targets configured)
Analyzing: target //snowflake/ml:wheel (90 packages loaded, 227 targets configured)
Analyzing: target //snowflake/ml:wheel (91 packages loaded, 227 targets configured)
Analyzing: target //snowflake/ml:wheel (91 packages loaded, 227 targets configured)
Analyzing: target //snowflake/ml:wheel (91 packages loaded, 227 targets configured)
Analyzing: target //snowflake/ml:wheel (91 packages loaded, 227 targets configured)
Analyzing: target //snowflake/ml:wheel (91 packages loaded, 227 targets configured)
Analyzing: target //snowflake/ml:wheel (91 packages loaded, 227 targets configured)
INFO: Repository build_env instantiated at:
  /Users/miniconda3/envs/fin/conda-bld/snowflake-ml-python_1694689494466/work/WORKSPACE:71:13: in <toplevel>
  /Users//miniconda3/envs/fin/conda-bld/snowflake-ml-python_1694689494466/work/third_party/rules_conda/defs.bzl:19:10: in conda_create
  /private/var/tmp/_bazel_/ec07aae0a2d8b2418eaa384a250f0962/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule conda_create_rule defined at:
  /Users/miniconda3/envs/fin/conda-bld/snowflake-ml-python_1694689494466/work/third_party/rules_conda/env.bzl:149:36: in <toplevel>
Analyzing: target //snowflake/ml:wheel (91 packages loaded, 227 targets configured)
ERROR: An error occurred during the fetch of repository 'build_env':
   Traceback (most recent call last):
        File "/Users/miniconda3/envs/fin/conda-bld/snowflake-ml-python_1694689494466/work/third_party/rules_conda/env.bzl", line 144, column 24, in _conda_create_impl
                _create_environment(rctx, env_name)
        File "/Users/miniconda3/envs/fin/conda-bld/snowflake-ml-python_1694689494466/work/third_party/rules_conda/env.bzl", line 88, column 30, in _create_environment
                _create_empty_environment(rctx, env_name)
        File "/Users/miniconda3/envs/fin/conda-bld/snowflake-ml-python_1694689494466/work/third_party/rules_conda/env.bzl", line 69, column 13, in _create_empty_environment
                fail("Failure creating empty environment.\nstdout: {}\nstderr: {}".format(result.stdout, result.stderr))
Error in fail: Failure creating empty environment.
stdout: 
stderr: warning  libmamba Could not parse state file: Could not load cache state: [json.exception.type_error.302] type must be string, but is null
warning  libmamba Could not parse state file: Could not load cache state: [json.exception.type_error.302] type must be string, but is null
warning  libmamba Could not parse state file: Could not load cache state: [json.exception.type_error.302] type must be string, but is null
warning  libmamba Could not parse state file: Could not load cache state: [json.exception.type_error.302] type must be string, but is null

Is it possible to get the latest versions of conda and conda-libmamba-colver instead of hardcoded ones?

skupr-anaconda commented 11 months ago

I tried to patch conda.blz to get the latest available version of conda and conda-libmabma-solver but I got another error:

[14 / 193] [Sched] Executing genrule //snowflake/ml/modeling/feature_selection:generate_select_fwe ... (16 actions, 2 running)

ERROR: /croot/snowflake-ml-python_1694697212710/work/snowflake/ml/modeling/xgboost/BUILD.bazel:6:19: Executing genrule //snowflake/ml/modeling/xgboost:generate_xgb_regressor failed: (Exit 1): bash failed: error executing command (from target //snowflake/ml/modeling/xgboost:generate_xgb_regressor) /bin/bash -c ... (remaining 1 argument skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging

Traceback (most recent call last):

  File "/root/.cache/bazel/_bazel_root/b8b4e8a1a8a12c85c3a3978d39b2d19c/sandbox/processwrapper-sandbox/4/execroot/SnowML/bazel-out/k8-opt-exec-DAD2D8CE/bin/codegen/estimator_autogen_tool.runfiles/SnowML/codegen/estimator_autogen_tool.py", line 17, in <module>

    from sklearn_wrapper_autogen import AutogenTool, GenMode

ModuleNotFoundError: No module named 'sklearn_wrapper_autogen'

I am very confused by this error too.

sfc-gh-wzhao commented 11 months ago

Hi, thank you for your feedback!

We have noticed, identified these 2 issues before and we have fixed them. They will be available in the next release.

sfc-gh-wzhao commented 11 months ago

Close the issue as the fix is released.