holepunchto / hyperdrive

Hyperdrive is a secure, real time distributed file system
Apache License 2.0
1.86k stars 134 forks source link

Get the list of versions for a file? #251

Closed KrishnaPG closed 1 year ago

KrishnaPG commented 4 years ago

Lets say a directory has ten files. But only one was is getting modified. So, I would expect the version for that file keeps getting incremented / changed. If we know that file name, can we get the list of all version history for that file (similar to git ) ?

Is there an API for this? or any other low-level way?

pfrazee commented 4 years ago

IIRC there are two ways to do this (correct me maf or andrew if I'm wrong)

  1. Get the full history list for the drive and then filter it down to the files you're interested in
  2. Use diffing with the filename as a prefix to iteratively walk backwards through the history of an individual file, manually constructing its history.
KrishnaPG commented 4 years ago

Thank you @pfrazee

... walk backwards through the history of an individual file

Wondering how to access the history of a file ? Is there some API? Or any other low-level mechanism?

...Get the full history list for the drive and then filter it down to the files you're interested in ...

The archive.version seems to be pointing to the latest version, and do not see any way of accessing the full history of drive (in the readMe docs).

Where can I find more information this? Any links or pointers to information on this version management api for drive and files would greatly help. Specifically I am looking for help with versioning of files / content.

xoryouyou commented 3 years ago

I've stumbled upon the same issue.

It would be great to have some function like drive.promises.getVersions(filename) -> [versions]

I've build myself a small but not ideal helper function.

async function getVersions(drive_path, versions, filename, filetype) {

    let file_in_versions = [];
    // versions start at 1 see https://github.com/hypercore-protocol/hyperdrive/blob/4a10b3810c655a92b919cf4e9e927cc08322673c/index.js#L107
    for (let i = 1; i <= versions; i++) {
        // create new drive
        let drive = new Hyperdrive(drive_path);
        // wait for ready
        await drive.promises.ready()
        // check out specific version
        let old = drive.checkout(i);
        try {
            // check if file exists
            await old.promises.stat(filename, filetype)
            // push to found versions
            file_in_versions.push(i);
        } catch (error) {
            if (error.code == 'ENOENT') {
                // file does not exist
                continue;
            }
            // error out if something else happened
            throw (error);
        }
        // close the drive so we don't run into 
        // "MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 append listeners added to Hypercore"
        drive.close();
    }
    // return the found versions
    return file_in_versions;
}

let versions = await getVersions('./data/test-drive', drive.version, '/file1.txt', 'utf8');
console.log("Versions:", versions);

I hope this helps somebody but I think some feature like this should be included into the hyperdrive api.

KrishnaPG commented 2 years ago

Thank you @xoryouyou

It would be great if hyperdrive supports file diff streams and changelogs natively. Currently they operate at the drive level, but given that a file is a basic unit of access, it would be great to support change notifications, diff streams at the file level natively.

For example, sharing a file at a specific version and subscribing to its change notifications.