rokucommunity / roku-deploy

An npm module for zipping and deploying to Roku devices.
MIT License
42 stars 19 forks source link

Support for Raspberry Pi (Arm32) on `retrieveSignedPackage` #179

Open livecano opened 2 days ago

livecano commented 2 days ago

I ran into an issue while trying to execute retrieveSignedPackage. The GetToFile function in rokuDeploy needs further customization to handle devices with low CPU and memory, as is the case with a Raspberry Pi.

Below is an optimized version of the GetToFile function that "optionally" can also compare a checksum to verify the integrity of the signed package. I tested it on a Pi, and it works as expected.

async function optimizedGetToFile(requestParams, filePath, expectedChecksum) {
    await fsExtra.ensureFile(filePath);

    // Handle partial downloads
    const headers = fs.existsSync(filePath)
        ? { Range: `bytes=${fs.statSync(filePath).size}-` }
        : {};
    requestParams.headers = { ...requestParams.headers, ...headers };

    let writeStream;

    return new Promise((resolve, reject) => {
        try {
            writeStream = fs.createWriteStream(filePath, { flags: 'a', highWaterMark: 64 * 1024 });

            // // Calculate checksum while writing
            // const hash = crypto.createHash('sha256'); // Change to match your checksum algorithm

            // writeStream.on('finish', async () => {
            //     // Verify checksum after the file is fully written
            //     try {
            //         const calculatedChecksum = await computeChecksum(filePath);
            //         if (calculatedChecksum === expectedChecksum) {
            //             console.log('Checksum verified successfully.');
            //             resolve(filePath);
            //         } else {
            //             reject(new Error('Checksum verification failed.'));
            //         }
            //     } catch (err) {
            //         reject(err);
            //     }
            // });

            writeStream.on('finish', () => resolve(filePath));
            writeStream.on('error', (error) => reject(error));

            const req = request.get(requestParams);

            req.on('error', (err) => reject(err));

            req.pipe(writeStream);

            let downloadedBytes = 0;
            let lastLogged = 0;

            req.on('data', (chunk) => {
                downloadedBytes += chunk.length;
                if (downloadedBytes - lastLogged >= 64 * 1024) {
                    console.log(`Downloaded ${downloadedBytes} bytes`);
                    lastLogged = downloadedBytes;
                }
            });

            req.on('socket', (socket) => {
                socket.setTimeout(requestParams.timeout || 300000);
                socket.on('timeout', () => {
                    req.abort();
                    reject(new Error(`Request timed out after ${requestParams.timeout || 300000}ms`));
                });
            });

            req.on('end', () => {
                console.log(`Download completed: ${filePath}`);
            });
        } catch (err) {
            reject(err);
        }
    }).finally(() => {
        if (writeStream) {
            try {
                writeStream.close();
            } catch (err) {
                // Ignore stream close errors
            }
        }
    });
}
TwitchBronBron commented 6 hours ago

From what I can tell, the only relevant changes are to set highWaterMark and also support calling the function multiple times so you can download the file in chunks.

However, I don't know why either of those matter. The entire signed package should be 4MB or less, which every PI since the first release has enough RAM to do that.

Could you better explain what you're trying to accomplish here, and how your changes mitigate the issue? How would consumers expect to use your proposed changes? (do they need to call the function multiple times?)