microsoft / PlanetaryComputer

Issues, discussions, and information about the Microsoft Planetary Computer
https://planetarycomputer.microsoft.com/
MIT License
180 stars 7 forks source link

Request update of sits in R environment #321

Closed KBodolai closed 6 months ago

KBodolai commented 7 months ago

Hi, I recently ran into an issue trying to run some of the SITS examples in the MPC R environment. I described in detail in this issue in the sits repo.

I'm not a regular R user, I tried to install the newest version but it ran into dependency issues. Would it be possible to update the SITS version in the hub to 1.4.2?

I think there may be some other deeper issues with the R environment that I don't know enough to debug, sometimes when I start the server and I try to run a fresh R notebook, it seems to get caught up on connecting to the R kernel:

Screenshot 2024-02-16 at 15 32 52
KBodolai commented 7 months ago

if I understand correctly, we would need to update the conda-lock file here. I'm happy to work on it and do a PR myself, I'd just need some pointers as to how to test it well and make sure everything works, what I'd try right now is:

  1. Build the docker image without r-sits in the conda-lock
  2. test that it works as expected
  3. update the conda lock and submit a PR to that repo?

Thanks, K.

TomAugspurger commented 7 months ago

Sorry for not getting back to you earlier. I got a bit distracted trying to find the latest on https://r-spatial.org/r/2023/04/10/evolution3.html, since I'm not plugged into the R geospatial community that closely.

That's the right repo. The input is the r/environment.yaml at https://github.com/microsoft/planetary-computer-containers/blob/main/r/environment.yml, and the output of make r/conda-linux-64.lock is the lockfile which is passed into the container build (which is built on GitHub releases).

Feel free to give updating the environment.yaml a shot and let me know if you run into any issues.

KBodolai commented 7 months ago

No worries at all!

I'll have a look and try to update it and come back to you :)

KBodolai commented 6 months ago

Update: I tried using conda-lock (as per the makefile) to update the conda-lock and I'm guetting errored out every time, the trace is rather long, but I suspect the issue is here:

```bash STDERR: PackageRecord(_hash=5590636656325471586, name='__glibc', version='0', build='0', build_number=0, channel=Channel("@"), subdir='linux-64', fn='__glibc', md5='12345678901234567890123456789012', package_type='virtual_system') Traceback (most recent call last): File "/home/kristian/miniconda3/envs/pangeo/bin/conda-lock", line 10, in sys.exit(main()) ^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 1402, in lock lock_func( File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 1110, in run_lock make_lock_files( File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 393, in make_lock_files fresh_lock_content = create_lockfile_from_spec( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 836, in create_lockfile_from_spec deps = _solve_for_arch( ^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 748, in _solve_for_arch conda_deps = solve_conda( ^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/conda_solver.py", line 160, in solve_conda dry_run_install = solve_specs_for_arch( ^^^^^^^^^^^^^^^^^^^^^ File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/conda_solver.py", line 372, in solve_specs_for_arch proc.check_returncode() File "/home/kristian/miniconda3/envs/pangeo/lib/python3.12/site-packages/conda_lock/_vendor/poetry/utils/_compat.py", line 168, in check_returncode raise CalledProcessError( conda_lock._vendor.poetry.utils._compat.CalledProcessError: Command '['/home/kristian/miniconda3/condabin/mamba', 'create', '--prefix', '/tmp/tmp6wmd6iiy/prefix', '--dry-run', '--json', '--override-channels', '--channel', 'conda-forge', '--channel', 'file:///tmp/tmpk_a7pty2', 'python 3.*', 'pangeo-notebook >=2021.09.08', 'planetary-computer >=0.4.2', 'pystac-client >=0.5.0', 'pystac >=1.1.0', 'r-arrow', 'r-base', 'r-caret', 'r-data.table', 'r-dendextend', 'r-devtools', 'r-dtwclust', 'r-e1071', 'r-flexclust', 'r-gdalcubes', 'r-gdalutils', 'r-geor', 'r-geosphere', 'r-ggplot2', 'r-hdf5r', 'r-irkernel', 'r-keras', 'r-kohonen', 'r-lidr', 'r-lubridate', 'r-magrittr', 'r-mass', 'r-mgcv', 'r-ncdf4', 'r-nnet', 'r-proj4', 'r-ptw', 'r-randomfields', 'r-randomforest', 'r-ranger', 'r-raster', 'r-rcolorbrewer', 'r-reticulate', 'r-rgdal', 'r-rgeos', 'r-rlas', 'r-rnetcdf', 'r-rstac >=0.9.1_5', 'r-sf', 'r-signal', 'r-slider', 'r-sp', 'r-spacetime', 'r-spatstat', 'r-spdep', 'r-stars', 'r-sits', 'r-terra', 'r-tidync', 'r-tidyverse', 'r-tmap', 'r-xgboost', 'r-zoo']' returned non-zero exit status 1. ```

I think it's something to do with conda-lock and not the dependencies themselves, because I've managed to just install the environment.yml.

Could you check if running conda-lock works for you? If not, happy to keep investigating.

Sorry this is dragging a bit, and thanks for your time Tom!

TomAugspurger commented 6 months ago

I did a bit of work in https://github.com/conda-forge/r-sits-feedstock/pull/26 to get r-sits updated in conda-forge.

https://github.com/microsoft/planetary-computer-containers/pull/63 has an update.

It'll be a little bit of time before I can get this rolled out to the Hub.

KBodolai commented 6 months ago

Thanks Tom!

I'll stay tuned, do let me know if there's anything I can help with.

gilbertocamara commented 6 months ago

Dear Tom

Thanks for the support!

Prof Dr Gilberto Camara Senior Researcher National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/

On 4 Mar 2024, at 12:11, Kristian Bodolai @.***> wrote:

Thanks Tom! I'll stay tuned, do let me know if there's anything I can help with. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

TomAugspurger commented 6 months ago

sits should be updated on the PC Hub now. Please reopen this if you spot any issues.

KBodolai commented 6 months ago

Thanks a lot for your support Tom!