search

thin-edge / thin-edge.io

The open edge framework for lightweight IoT devices
https://thin-edge.io
Apache License 2.0
221 stars 54 forks source link

debian software package does not install from url when version contains a colon character #2666

Closed reubenmiller closed 8 months ago

reubenmiller commented 8 months ago

Describe the bug

The tedge-apt-plugin fails to install a software package from a URL (e.g. a package hosted in Cumulocity IoT) if the software version contains a colon in its name due to apt-get install <file> ignoring the <file> argument if a colon is present.

apt interprets the colon as the Epoch separator, so it seems that apt-get install silently ignores the given file and does not exit.

The root cause is the fact that tedge-apt-plugin downloads the debian file from the given software url, and save the file using the path built from the name and version, e.g. /tmp/{name}_{version}.deb, and the filename is not sanitized to be compatible with apt/apt-get.

There is a similar issue found in another project:

Note

To Reproduce

  1. Create a software package in Cumulocity for vim

    • Name: vim
    • Version: 2:8.2.2434-3+deb11u1
    • Url: Upload a package that matches your device's CPU architecture from the debian website

  2. Install vim via the Cumulocity IoT Device Management Software page for the device under test

  3. Inspect the software management logs on the device to see if the apt-get install command actually tried installing the package or not (see below for an example where it doesn't process the given package)

    ----- $ Downloading: http://127.0.0.1:8001/c8y/inventory/binaries/33714613 to /tmp/vim_2:8.2.2434-3+deb11u1
----- $ /usr/bin/sudo "/etc/tedge/sm-plugins/apt" "update-list"
exit status: 0

stdout <<EOF
apt-get install args: ["install", "--quiet", "--yes", "/tmp/vim_2:8.2.2434-3+deb11u1.deb"]
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 39 not upgraded.
EOF

stderr <<EOF
EOF

Expected behavior

If the tedge-apt-plugin downloads the debian package from a given URL, then the temporary file created from the download should sanitize the version by substituting any unsupported characters.

Finding an example spec seems to be difficult, but this post might be helpful:

It might be enough to just replace : with the URL encoded %3a, but the results will have to be verified.

Screenshots

The following shows the output when trying to install a debian package from file with a colon in it (notice that there are no errors displayed):

# apt install '/tmp/vim_2:8.2.2434-3+deb11u1.deb'
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 39 not upgraded.

Renaming the file to replace the : with %3a, is enough for apt-get to process the package (though the install is still failing, however this is just because the dependencies aren't installed, and installing from file does not pull in dependencies automatically.)

# mv /tmp/vim_2:8.2.2434-3+deb11u1.deb /tmp/vim_2%3a8.2.2434-3+deb11u1.deb
# apt-get install /tmp/vim_2%3a8.2.2434-3+deb11u1.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'vim' instead of '/tmp/vim_2%3a8.2.2434-3+deb11u1.deb'
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
vim : Depends: vim-common (= 2:8.2.2434-3+deb11u1) but 2:9.0.1378-2 is to be installed
    Depends: vim-runtime (= 2:8.2.2434-3+deb11u1) but 2:9.0.1378-2 is to be installed
E: Unable to correct problems, you have held broken packages.

Environment (please complete the following information):

Property Value
OS [incl. version] Debian GNU/Linux 12 (bookworm)
Hardware [incl. revision] Raspberry Pi 5 Model B Rev 1.0
System-Architecture Linux rpi5-d83addab8e9f 6.1.0-rpi4-rpi-2712 #1 SMP PREEMPT Debian 1:6.1.54-1+rpt2 (2023-10-05) aarch64 GNU/Linux
thin-edge.io version tedge 1.0.0-rc.2~297+geb93afc

Additional context

reubenmiller commented 8 months ago

@Ruadhri17 We could live with a naive implementation for now rather than doing a full url encoding:

file

crates/core/plugin_sm/src/plugin.rs
fn sm_path(name: &str, version: &Option<String>, target_dir_path: impl AsRef<Path>) -> PathBuf {
    let mut filename = name.to_string();
    if let Some(version) = version {
        filename.push('_');
        filename.push_str(version.as_str());
    }

    target_dir_path.as_ref().join(sanitize_filename(&filename))
}

fn sanitize_filename(filename: &String) -> String {
    // Replace special characters
    // see https://github.com/thin-edge/thin-edge.io/issues/2666
    filename.replace(":", "__")
}
reubenmiller commented 8 months ago

Implemented in https://github.com/thin-edge/thin-edge.io/pull/2683

See the system test included in the PR. A dummy package was included in the tests to test the exact scenario highlighted in this test:

gligorisaev commented 8 months ago

QA has thoroughly checked the bug and here are the results: