spkenv / spk

A Package Manager for high velocity software environments, built on spfs.
https://spkenv.dev
Apache License 2.0
38 stars 6 forks source link

spk install loses existing non-spk package digests from spfs runtime stack #759

Open dcookspi opened 1 year ago

dcookspi commented 1 year ago

The spk install ... command adds packages to the current runtime by adding requests for the new packages to requests for the existing packages. Then is solves for all of them and replaces the runtime's stack with the layers from the resolved solution. However in doing so, it loses any additional spfs layers/refs/digests that were in the runtime but did not come from the original solution's packages. Any edits in the upperdir are retained though.

This means someone who has entered a runtime via spk, has done some editing, has spfs committed a layer or platform, carried on editing/working, and then wants to include another package will lose their extra layer/s from the runtime. Getting their additional layer/s back on top of the runtime stack is convoluted with the existing commands.

It seems like spk install ... should replace the layers for the old packages with the layers from the new solution, but it should leave any other layer/s (really spfs refs) in the stack as they are (at the top of the stack).

Steps to replicate:

# enter a runtime
> spk env python
* You are now in a configured subshell *

# showing what's present in the runtime
> spk info
Installed Packages:
  runtime/stdfs:{build,run}/=1.0.0/3I42H3S6  highest 
  runtime/python:{build,run}/=3.9.7/XO645NLV highest 
 Number of Packages: 2
> spfs info
Active Runtime:
 id:: e30c2fa5-d2b2-4cca-84a9-5a73e508d23c
 editable:: true
stack
  - 4H3VQSHBTDI7JK4772NNDVO3RNHEVFDYD7EPG6DQ2UGSMT6MR6HA====
  - ORJ5AZE4MXJTGGFNDCLHZA6ZJKWXSJ2X7A7OPEHCVQ3RZY27ZTXA====

Run 'spfs diff' for active changes

# User does some editing, testing, and saves changes in a layer
> spfs edit
> touch /spfs/ook
> spfs diff
+/spfs/ook
> spfs commit layer
      ⠏ computing manifest [========================================]    1/1
      ⠏ committing blobs   [========================================]    1/1
      ⠏ committing data    [========================================]   0B/0B
> spfs info
Active Runtime:
 id:: e30c2fa5-d2b2-4cca-84a9-5a73e508d23c
 editable:: false
stack
  - HY6OZJ3JJCWQO6YZHOJG33CWZYDIYLWOY26ZJSFGJLXWIO6XOYXQ====   (the new layer)
  - 4H3VQSHBTDI7JK4772NNDVO3RNHEVFDYD7EPG6DQ2UGSMT6MR6HA====
  - ORJ5AZE4MXJTGGFNDCLHZA6ZJKWXSJ2X7A7OPEHCVQ3RZY27ZTXA====

Run 'spfs diff' for active changes

# Used forgot to add a package, adds it now...
> spk install python-pytest
The following packages will be installed:

  Requested:
    stdfs/1.0.0/3I42H3S6
    python/3.9.7/XO645NLV
    python-pytest/7.3.1/GMTG3CXY

Do you want to continue? [y/N]: y

# The new layer above is no longer in the runtime. Some of their changes are gone from the runtime.
> spfs info
Active Runtime:
 id:: e30c2fa5-d2b2-4cca-84a9-5a73e508d23c
 editable:: false
stack
  - ORJ5AZE4MXJTGGFNDCLHZA6ZJKWXSJ2X7A7OPEHCVQ3RZY27ZTXA====
  - 4H3VQSHBTDI7JK4772NNDVO3RNHEVFDYD7EPG6DQ2UGSMT6MR6HA====
  - MMPC3ROGQ3GFYRHAQJMNSQXIP6AU3OQOEHQHH664XGHBLJPMUGOA====
  - ZVNGCGDRWQ2W6MD6KFUFHSU52R7SE557JDIYX3GGIXMISNYGIDBQ====
  - VEYLNQTE3IWMT5I3MZOLQDW4KQRYT23XIKUHB6XWGDIBOKWA7VKQ====
  - EWISDHLUVU6LBLYGTM3SJKBFCNEPM3Q54SARROGJKDKN3SQ3HI5A====
  - KROKA6PBZKLD2EXTRMBPN6X6JJAKYGXSAT6FHJEJXE6HC3OSD2LQ====
  - DIAQBPVPQ2AJMTGIXT3X6MFO2UQU5R3K6ZKHKRJYKHMF5ZXC7C2A====
  - JRZCJXRNER5NVCXTM2DUVTBV3TTRWW5FPZ3IUSXOFY47X336M4UQ====
  - DOAQ4CS6HPGFJZQP2JSNCI2BDNYZJ55E6VU7P6IS7D4E4F5BZ7OQ====
  - RSVG6VTYG5XC6YBMG6ED2IET6E2M6JHT2LWS7JAKK4QZU55E75LQ====
  - HZL3HSIM24RT5HB7G34E3XKN3YLJW4QZEIWERW3AN73XZNL56A4Q====
  - FCOOCIQBH7ELC6VZHAJZKTESM67BIHGIWXVA2EIAFZ5HNDR7SHXQ====
  - MRZTI6ENG572PZXS43JJ45XXIZWLNHDVFNN7RVWLR37U6XN6DYHA====

Run 'spfs diff' for active changes
rydrman commented 1 year ago

Makes sense to me - I've been eyeing this install command for a re-vamp as well since we are looking at workflows which would require incremental updates to old solves and this command can only really add new compatible packages, but has a hard time with making any changes to existing packages such as changing the build or making downgrades