PR: steal-tree

[matthewp]

@frank-dspeed commented on Mon Nov 27 2017

Sorry for the not so well explained PR information but i was forced here https://github.com/canjs/can-component/issues/173 to talk about it and so that this is complet

This issue tracks the upcoming pr for steal tree that aims to create nativ tar.gz bundles with file meta and browser support that can be expanded into a object with original folder file names so we work on same structure in bundles and development.

it returns nativ blobs that can be returned with mime type as extra url data object that can directly be integrated into a website this allows also assets bundles.

Includes versioning features.

about the name steal-fs would also be cool or steal-tar

The Service worker untar implamentation is already done now working on a way to run in single th env efficent will take some weeks

StorageObj.getJs() === the script as js export as when you would import it
StorageObj.getBlob('mime')

Documentation

The module is a function that returns a modified Promise with a progress callback. This callback is executed every time a file is extracted. The standard Promise.then method is also called when extraction is done, with all extracted files as argument. The extraction is done in a Web Worker to allow the main UI thread to continue.

Example:

// Load the source ArrayBuffer from a XMLHttpRequest (or any other way you may need).
var sourceBuffer = [...];

untar(sourceBuffer)
.progress(function(extractedFile) {
    ... // Do something with a single extracted file.
})
.then(function(extractedFiles) {
    ... // Do something with all extracted files.
});

// or

untar(sourceBuffer).then(
    function(extractedFiles) { // onSuccess
        ... // Do something with all extracted files.
    },
    function(err) { // onError
        ... // Handle the error.
    },
    function(extractedFile) { // onProgress
        ... // Do something with a single extracted file.
    }
);

File object

The returned file object(s) has the following properties. Most of these are explained in the Tar wikipedia entry.

• name = The full filename (including path and ustar filename prefix).
• mode
• uid
• gid
• size
• modificationTime
• checksum
• type
• linkname
• ustarFormat
• buffer An ArrayBuffer with the contents of the file.
• blob A Blob object with the contents of the file.

• getBlobUrl() A unique ObjectUrl to the data can be retrieved with this method for easy usage of extracted data in <img> tags etc.

      document.getElementById("targetImageElement").src = file.getBlobUrl();

• readAsString() Parse the file contents as a UTF-8 string.
• readAsJSON() Parse the file contents as a JSON object.

If the .tar file was in the ustar format (which most are), the following properties are also defined:

• version
• uname
• gname
• devmajor
• devminor
• namePrefix

This kind of operations support a lot of amazing behaviors

---

@matthewp commented on Mon Nov 27 2017

This issue is to create tar files from bundles? Why?

---

@frank-dspeed commented on Mon Nov 27 2017

@matthewp we create tar files with the unmodifyed graph and then have path mappings 1:1 to real pathes and can set mappings to the files in that tar archive to the right vars in the right context

---

@frank-dspeed commented on Mon Nov 27 2017

@matthewp aka no transpilling simply the same files we use in dev we can use in production we bundle all up into tar and then extract on client in a web worker and use the scripts from there as blobs. or other method the steal-tree supports graphAsTar.toString and other methods like .getBlob('mime-type')

---

@matthewp commented on Mon Nov 27 2017

Ok, so this is a web packaging format. You can try it, my guess is that it will be much slower than bundling though. Going to move to the rfcs repo.

[frank-dspeed]

@matthewp more then that it should be the commun formart for steal !!! you can simply drop the tar step i propose to keep the filestructure the original one ! + meta if you then produce a bundle version out of that i am fine with that but a cached untared archiv is much faster as you think as we don't apply compression as http gz does that :)

[frank-dspeed]

@matthewp i propose with this to be more exactly to create and also bundle steal map data to the propertys the scripts are needing maybe we can have a short hangout?