AppImage / appimagetool

A low-level tool to generate an AppImage from an existing AppDir
96 stars 15 forks source link

Runtime download step does not truly validate downloaded file - 404 "Not Found" deemed success #57

Closed spiderkeys closed 3 months ago

spiderkeys commented 3 months ago

Using release:

/opt/mr/linuxdeploy/appimagetool-x86_64.AppImage --verbose ./squashfs-root/
appimagetool, continuous build (git version b9b26c3), build 146 built on 2024-05-19 18:33:03 UTC

I run into a situation where the download of the type2-runtime results in an 404, but AppImageTool doesn't seem to mind. What ends up happening is that instead of the runtime getting inserted at the beginning of the AppImage, the text "Not Found" is inserted (and thus the appimage isnt a valid ELF file:

Embedding ELF...
Marking the AppImage as executable...
Embedding MD5 digest
Platforms other than 32-bit/64-bit are currently not supported!Could not find section .digest_md5 in runtime

This is the verbose output of the download step:

Generating squashfs...
Downloading runtime file from https://github.com/AppImage/type2-runtime/releases/download/continuous/runtime-x86_64
libcurl's default CA certificate bundle file /etc/ssl/certs/ca-certificates.crt was found on this system
libcurl's default CA certificate bundle directory /etc/ssl/certs was found on this system
* Host github.com:443 was resolved.
* IPv6: (none)
* IPv4: 140.82.116.4
*   Trying 140.82.116.4:443...
* Connected to github.com (140.82.116.4) port 443
* ALPN: curl offers h2,http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=github.com
*  start date: Mar  7 00:00:00 2024 GMT
*  expire date: Mar  7 23:59:59 2025 GMT
*  subjectAltName: host "github.com" matched cert's "github.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo ECC Domain Validation Secure Server CA
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA256
*   Certificate level 1: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 2: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using ecdsa-with-SHA384
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://github.com/AppImage/type2-runtime/releases/download/continuous/runtime-x86_64
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: github.com]
* [HTTP/2] [1] [:path: /AppImage/type2-runtime/releases/download/continuous/runtime-x86_64]
* [HTTP/2] [1] [accept: */*]
> GET /AppImage/type2-runtime/releases/download/continuous/runtime-x86_64 HTTP/2
Host: github.com
Accept: */*

* old SSL session ID is stale, removing
< HTTP/2 404 
< server: GitHub.com
< date: Thu, 01 Aug 2024 04:18:46 GMT
< content-type: text/plain; charset=utf-8
< vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, Accept-Encoding, Accept, X-Requested-With
< cache-control: no-cache
< strict-transport-security: max-age=31536000; includeSubdomains; preload
< x-frame-options: deny
< x-content-type-options: nosniff
< x-xss-protection: 0
< referrer-policy: no-referrer-when-downgrade
< content-security-policy: default-src 'none'; base-uri 'self'; connect-src 'self'; form-action 'self'; img-src 'self' data:; script-src 'self'; style-src 'unsafe-inline'
< content-length: 9
< x-github-request-id: B512:380098:27C4A27:2865ADD:66AB0CA6
< 
* Connection #0 to host github.com left intact
Downloaded runtime binary of size 9
Size of the embedded runtime: 9 bytes
spiderkeys commented 3 months ago

By manually downloading and specifiying the runtime-fuse3-x86_64 file under the old release on the type2-runtime repo, I am able to successfully create and run the appimage:

/opt/mr/linuxdeploy/appimagetool-x86_64.AppImage --runtime-file ./runtime-fuse3-x86_64 ./squashfs-root/
# ...
Embedding ELF...
Marking the AppImage as executable...
Embedding MD5 digest
Success
stsp commented 3 months ago

Same problem here. Failure to d/l the runtime results in "success" with corrupted output.

stsp commented 3 months ago

By manually downloading and specifiying the runtime-fuse3-x86_64 file under the old release on the type2-runtime repo, I am able to successfully create and run the appimage:

I can additionally confirm that manually downloading the latest runtime, still gives the non-working result. Only literally following @spiderkeys suggestion (using old runtime) I was able to create a working appimage.

probonopd commented 3 months ago

@TheAssassin I think we need to add explicit HTTP status code handling; would this be the correct way?

I must say that having to create classes for curl ourselves is a big burden and errors and unhandled cases can slip in easily.

class GetRequest {
private:
    // ... (rest of the class remains the same)

    long _httpStatus;

public:
    // ... (rest of the class remains the same)

    long httpStatus() const {
        return _httpStatus;
    }

    CurlResponse perform() {
        auto result = curl_easy_perform(this->_handle);
        _httpStatus = getOption<long>(CURLINFO_RESPONSE_CODE);
        return {result == CURLE_OK, getOption<curl_off_t>(CURLINFO_CONTENT_LENGTH_DOWNLOAD_T), _buffer};
    }
};

bool fetch_runtime(char *arch, size_t *size, char **buffer, bool verbose) {
    // ... (rest of the function remains the same)

    try {
        GetRequest request(url, verbose);

        auto response = request.perform();

        if (response.statusCode() != 200) {
            std::cerr << "Error: Received HTTP status code " << response.statusCode() << std::endl;
            return false;
        }

        std::cerr << "Downloaded runtime binary of size " << response.contentLength() << std::endl;

        // ... (rest of the function remains the same)
    } catch (const CurlException& e) {
        std::cerr << "libcurl error: " << e.what() << std::endl;
        return false;
    }
}
TheAssassin commented 3 months ago

I can't review this easily without seeing a diff. Adding a status code check is a good idea nevertheless.

I must say that having to create classes for curl ourselves is a big burden and errors and unhandled cases can slip in easily.

Using libcurl directly is extremely hard to get right. The classes help accomplishing that goal by introducing a memory safe layer in between that also greatly improves the API. Unfortunately, there are no good HTTP libraries for C++ and no proper C++ wrappers for libcurl other than cpr which you can't just install from Ubuntu or alike as easily. I think the amount of code is maintainable. Plus, it's idiomatic C++, so contributors could easily provide fixes if they understand C++.

probonopd commented 3 months ago

Thanks for implementing a proper solution to check the status code @TheAssassin

TheAssassin commented 3 months ago

It's pretty much what you suggested. By coincidence, actually. It turned out almost identical even though I didn't look at the code you provided. There were just a few things missing.