coiled / examples

Examples using Dask and Coiled
17 stars 3 forks source link

Remove earthaccess pip install from environment.yml #28

Open mrocklin opened 1 year ago

mrocklin commented 1 year ago
╭──────────────────────────────── Package Sync ────────────────────────────────╮
│ Fetching latest package priorities                    ━━━━━━━━━━━━━━ 0:00:00 │
│ Scanning 295 conda packages                           ━━━━━━━━━━━━━━ 0:00:00 │
│ Scanning 210 python packages                          ━━━━━━━━━━━━━━ 0:00:00 │
│ Running pip check                                     ━━━━━━━━━━━━━━ 0:00:00 │
│ Validating environment                                ━━━━━━━━━━━━━━ 0:00:01 │
│ Creating wheel for /Users/mrocklin/workspace/examples ━━━━━━━━━━━━━━ 0:00:00 │
│ Creating wheel for coiled                             ━━━━━━━━━━━━━━ 0:00:02 │
│ Creating wheel for dask-expr                          ━━━━━━━━━━━━━━ 0:00:03 │
│ Creating wheel for earthaccess                        ━━━━━━━━━━━━━━ 0:00:07 │
│ Uploading coiled_local_examples                       ━━━━━━━━━━━━━━ 0:00:00 │
│ Uploading coiled                                      ━━━━━━━━━━━━━━ 0:00:00 │
│ Uploading dask-expr                                   ━━━━━━━━━━━━━━ 0:00:00 │
│ Uploading earthaccess                                 ━━━━━━━━━━━━━━ 0:00:00 │
│ Requesting package sync build                         ━━━━━━━━━━━━━━ 0:00:00 │
╰──────────────────────────────────────────────────────────────────────────────╯

It's taking up seven seconds early on in cluster creation, which slows down demoing a bit.

Not urgent, but I'll be happy when things are released properly. I'm removing it from my local environment as I demo these days.

dchudz commented 1 year ago

for me earthaccess makes the conda install totally fail, not just be slow

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: - Ran pip subprocess with arguments:
['/opt/miniconda3/envs/coiled/bin/python', '-m', 'pip', 'install', '-U', '-r', '/Users/davidchudzicki/examples/condaenv.lkt4orrw.requirements.txt']
Pip subprocess output:
Collecting git+https://github.com/jrbourbeau/earthaccess.git@include-store (from -r /Users/davidchudzicki/examples/condaenv.lkt4orrw.requirements.txt (line 1))
  Cloning https://github.com/jrbourbeau/earthaccess.git (to revision include-store) to /private/var/folders/w7/90rc1pmj0jn8n0tpycfltwkc0000gp/T/pip-req-build-yufyxkk1

Pip subprocess error:
  Running command git clone --filter=blob:none --quiet https://github.com/jrbourbeau/earthaccess.git /private/var/folders/w7/90rc1pmj0jn8n0tpycfltwkc0000gp/T/pip-req-build-yufyxkk1
  WARNING: Did not find branch or tag 'include-store', assuming revision or ref.
  Running command git checkout -q include-store
  error: pathspec 'include-store' did not match any file(s) known to git
  error: subprocess-exited-with-error

  × git checkout -q include-store did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git checkout -q include-store did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

failed

CondaEnvException: Pip failed

@jrbourbeau

jrbourbeau commented 1 year ago

@dchudz see https://github.com/coiled/examples/pull/32 for a fix

dchudz commented 1 year ago

fwiw I suspect that solves my "it's broken" problem, and not Matt's "it's slow" problem

I don't have any opinion about Matt's problem though

jrbourbeau commented 1 year ago

Yes, agreed. When the new release is out soon the slow part should be addressed