canonical / snapcraft

Package, distribute, and update any app for Linux and IoT.
https://snapcraft.io
GNU General Public License v3.0
1.17k stars 441 forks source link

Using (source-type: git; source: .) does not seem to ever realise files have changed, snap rebuild caches code changes #4883

Open lathiat opened 5 days ago

lathiat commented 5 days ago

Bug Description

It seems that when you specify source-type: git and source: . to use the git source from the local directory, that snapcraft never notices a change to the source and will re-used the cached source. This results in a re-built snap that has not integrated local code changes, which is very surprising.

I ran into this while working on the Simplestreams Snap (https://git.launchpad.net/simplestreams) which currently does this.

I would make a local change to the bin/sstream-query file, re-run snapcraft and then the change would not be included. The same is true if I actually make a git commit with the change. It seems that the PULL stage is presumed clean and is not re-run as the override-pull steps to set the version from the git hash also don't seem to get re-run, even if I've made an actual commit that changes it's output. Additionally, if you make a change to the override-pull script to force it to re-run the step then it will pickup both committed and non-committed changes in that case. So it doesn't seem to be a case of it comparing the git hash or ignore local changes.

If you change the source tree to use source-type: local this doesn't happen, and changing a file (whether committed or not) results in the pull stage re-running and the source not being cached.

Ideally this behaviour would be fixed, or possibly we should disallow this to stop people shooting themselves in the foot.

To Reproduce

  1. git clone https://git.launchpad.net/simplestreams
  2. Apply new snapcraft.yaml changes
    patch -Np1 <<<EOF
    diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
    index 75ac11a362ea..5b85628b84a9 100644
    --- a/snap/snapcraft.yaml
    +++ b/snap/snapcraft.yaml
    @@ -1,5 +1,5 @@
     name: simplestreams
    -base: core18
    +base: core22
     adopt-info: simplestreams
     summary: Library and tools for using Simple Streams data
     description: Library and tools for using Simple Streams data
    @@ -41,11 +41,8 @@ apps:
     parts:
       simplestreams:
         plugin: python
    -    python-version: python3
         source: .
         source-type: git
    -    constraints:
    -      - https://raw.githubusercontent.com/openstack/requirements/stable/ussuri/upper-constraints.txt
         python-packages:
           - python-glanceclient
           - python-keystoneclient
    @@ -58,8 +55,7 @@ parts:
           - libssl-dev
           - libxml2-dev
           - libxslt1-dev
    -    override-build: |
    -      snapcraftctl build
    -      last_release="$(git tag -l --sort=version:refname | grep -v ubuntu | tail -1)"
    -      current_shorthash="$(git log --format="%h" | head -1)"
    -      snapcraftctl set-version "${last_release}-${current_shorthash}"
    +    override-pull: |
    +      craftctl default
    +      TAG=$(git describe --tags --abbrev=10)
    +      craftctl set version="${TAG#v}"
    EOF
  3. Build the snap snapcraft
  4. Modify bin/sstream-query

    patch -Np1 <<<EOF
    diff --git a/bin/sstream-query b/bin/sstream-query
    index 65343173889c..62cbe208a16d 100755
    --- a/bin/sstream-query
    +++ b/bin/sstream-query
    @@ -28,6 +28,8 @@ import pprint
     import signal
     import sys
    
    +# Test change
    +
     FORMAT_PRETTY = "PRETTY"
     FORMAT_JSON = "JSON"
     DEFAULT_KEYRING = "/usr/share/keyrings/ubuntu-cloudimage-keyring.gpg"
  5. Re-run snapcraft
    rm *.snap
    snapcraft
  6. Install the snap snap install --dangerous ./simplestreams_*.snap
  7. Check if the file matches diff /snap/simplestreams/current/bin/sstream-query ./bin/sstream-query

Environment

Snapcraft: 8.2.12 (12024) tracking latest/stable Ubuntu 22.04 Build Provider: LXD

snapcraft.yaml

name: simplestreams
base: core22
adopt-info: simplestreams
summary: Library and tools for using Simple Streams data
description: Library and tools for using Simple Streams data

grade: stable
confinement: strict

layout:
  /usr/share/keyrings:
    bind: $SNAP/usr/share/keyrings

apps:
  sstream-mirror:
    command: bin/sstream-mirror
    plugs:
      - network
      - home
  sstream-mirror-glance:
    command: bin/sstream-mirror-glance
    plugs:
      - network
      - home
  sstream-query:
    command: bin/sstream-query
    plugs:
      - network
      - home
  sstream-sync:
    command: bin/sstream-sync
    plugs:
      - network
      - home
  json2streams:
    command: bin/json2streams
    plugs:
      - network
      - home

parts:
  simplestreams:
    plugin: python
    source: .
    source-type: git
    python-packages:
      - python-glanceclient
      - python-keystoneclient
      - python-swiftclient
    stage-packages:
      - gpgv
      - ubuntu-keyring
    build-packages:
      - libffi-dev
      - libssl-dev
      - libxml2-dev
      - libxslt1-dev
    override-pull: |
      craftctl default
      TAG=$(git describe --tags --abbrev=10)
      craftctl set version="${TAG#v}"

Relevant log output

N/A

Additional context

No response