Pull .zip from file system

[nikitassaraf]

Hello everyone!

I am trying to add an ability to MeshLabJS to pull .zip from the server instead of having a client to upload it. Has anyone tried doing it? Any leads will be highly appreciated!

Thank you!

[Gabryxx7]

I worked on .zip support in my textureSupport branch. It allowed for .zip files uploading from client through javascript. The .zip archive would then be extracted by calling the openMeshZip() function in C++. Since everything works in javascript, you might want to look at some way to download files from the server in Js, or look for some server-side file browser in AJAX. Supposedly you have a server in PHP or NodeJS, you could fill in an HTML list with all the files in a directory in your server. For instance PHP would fill in a list with all the files in /var/www/meshUpload so then from JS you could just give the option to select the file and download it from the server with ajax.

ThreeJS already provides functions to read mesh files from the server, e.g var loader = new THREE.XHRLoader(...);. For a .zip file, you might be able to read the file with an XHR and read it as an arraybuffer:

var loadZipFromServer = function(filename) {
            var xhr = new XMLHttpRequest();
            xhr.open('GET', filename, true);
            xhr.responseType = 'arraybuffer';
            xhr.onload = function(e) {
                var buffer = this.response;
                //  Emscripten need a Arrayview so from the returned arraybuffer we must create a view of it as 8bit chars
                var int8buf = new Int8Array(buffer);
                FS.createDataFile("/", filename, int8buf, true, true);
                resOpen = mf.cppMesh.openMeshZip(file.name);
            };

            xhr.send();
        };        
loadZipFromServer('fileName.zip');

I cannot test it, but I remember doing a similar thing in File.js, have a look at the loadMeshDataFromFile() function, you might be able to reuse it. Hope this helps :)

[nikitassaraf]

Thank you very much Gabriele!! I was thinking on similar lines. I tried to fix and your the function loadZipFromServer() but I am not able to find mf object.

[nikitassaraf]

Hi Gabriele,

I have used the code for laodZipFromServer() and I am able to get the arraybuffer of the .zip file that is to be visualized. However, i am not able to execute this line resOpen = mf.cppMesh.openMeshZip(file.name); this is error that I see - File.js:121 Uncaught ReferenceError: mf is not defined at XMLHttpRequest.xhr.onload (File.js:121). It is conceivable that mf is not defined, but I am unable to tell what mf is. Is it a integer value/string/ anything else? Can you please tell me what 'mf' is and where do i get it from as you have used it in this function too -

function loadMeshDataFromFile(file, mf, onLoaded) { var fileReader = new FileReader(); fileReader.readAsArrayBuffer(file); fileReader.onloadend = function (fileLoadedEvent) { console.log("Read file " + file.name + " size " + fileLoadedEvent.target.result.byteLength + " bytes"); console.timeEnd("Read mesh file"); // Emscripten need a Arrayview so from the returned arraybuffer we must create a view of it as 8bit chars var int8buf = new Int8Array(fileLoadedEvent.target.result); FS.createDataFile("/", file.name, int8buf, true, true); console.time("Parsing Mesh Time"); // console.log("File extension: " +file.name.split('.').pop()); var resOpen = -1; if (file.name.split('.').pop() === "zip") resOpen = mf.cppMesh.openMeshZip(file.name, mf.name); //extract data to a layer folder else resOpen = mf.cppMesh.openMesh(file.name);

        if (resOpen !== 0) {
            console.log("Ops! Error in Opening File. Try again.");
            FS.unlink(file.name);
            onLoaded(false);
        }
        console.timeEnd("Parsing Mesh Time");
        FS.unlink(file.name);
        onLoaded(true, mf);
        MLJ.core.Scene.addStateToHistory();
    };
}

[Gabryxx7]

Hi Sara, I haven't worked on the project since I implemented the textureSupport as a student in 2017 so forgive me if I might not be very clear. Looking at the code, mf is the Scene layer. In the file called Scene.js there is a function called this.createLayer() which takes a file name and returns a new layer, which is then saved in a variable called mf: var mf = MLJ.core.Scene.createLayer(file.name); In the file File.js. In short what's happening is that, upon clicking the open file option on the SceneBar, a list of files gets passed to openMeshFile() and for each file, a new layer in the scene is created by passing the filename to the createLayer() function. A scene Layer contains information such as its name (which is actually the mesh filename without extension), and a pointer to the cpp mesh created, to be edited in C++ with the vcglib. Initially, the cppMesh is empty, and this is where loadMeshDataFromFile() steps in. After the layer has been created with its mesh in ThreeJS and the empty one in cpp, the loadMeshDataFromFile() uses the client-side file and a reference to the newly created layer to fill in the cppMesh information. It simply takes the filename, reads it from the client, copies it to the server File System (FS) in Emscripten so that the C++ functions can now read that file from the FS and use the vcglib to read the mesh data. You'll notice, in fact, a call to openMesh() or openMeshZip() just like mf.cppMesh.openMesh(file.name);, where the empty cppMesh in the passed layer mf is just filling itself with information from the mesh file (or from a .zip archive with .mtl and mesh file).

In short, just make sure you are creating a new Layer in the scene from the mesh you want to open, so that it can be easily managed my MLJ and make sure it has a cppMesh that can be edited and manipulated with vcglib.

[nikitassaraf]

Thanks a ton, Gabriele! I understand that you have a busy schedule and I try not to bother you as much. I appreciate your help though! It was very difficult for me to find out the source of "mf". I get the direction now. However, do you have any kind on documentation about the code that you can share with me?

[nikitassaraf]

Well! I have edited the function loadZipFromServer()

function loadZipFromServer() { var xhr = new XMLHttpRequest(); xhr.open('GET', URL, true); xhr.responseType = 'arraybuffer'; var filename = 'I75_1cm_m100_11GCP_noGTG.zip'; xhr.onload = function(e) { var buffer = this.response; // Emscripten need a Arrayview so from the returned arraybuffer we must create a view of it as 8bit chars var int8buf = new Int8Array(buffer); FS.createDataFile('/', filename, int8buf, true, true);

            var mf = MLJ.core.Scene.createLayer(filename);
            mf.fileName = filename;
            var resOpen = -1;
            if (filename.split('.').pop() === "zip")
                resOpen = mf.cppMesh.openMeshZip(filename, mf.name); //extract data to a layer folder
            else
                resOpen = mf.cppMesh.openMesh(filename);

            if (resOpen != 0) {
                console.log("Ops! Error in Opening File. Try again.");
                FS.unlink(filename);
            //     onLoaded(false);
             }
            console.timeEnd("Parsing Mesh Time");
            FS.unlink(filename);
            // onLoaded(true, mf);
            MLJ.core.Scene.addStateToHistory();
        };
        xhr.send();
    };

The zip folder here is sent to openMeshZip() and this function does whats it is supposed to do. I entered breakpoints at every stage and i can see that my .zip is later sent to openMesh(). However, i cannot see the model and the material is deleted immediately! do you have any idea about this?

[Gabryxx7]

It's hard to reproduce the issue unfortunately. The material file as the .obj one are supposed to be deleted right away as they are now loaded in memory and shouldn't stay in the Emscripten FS. The fact that they are deleted with Success it means they were correctly loaded in the FS, which is good! I think now the issue is on the Javascript side. Are you able to see the new layer in MLJ? I think what is happening is that you are forgetting to trigger the $(document).trigger("MeshFileOpened", [meshFile]); which you can see is in the onLoad parameter of loadMeshFromFile() called in openMeshFile() in File.js. In fact you commented out onLoad(), which should trigger the MeshFileOpened event which in turn is handled by:

        $(document).on("MeshFileOpened",
                function (event, layer) {
                    MLJ.core.Scene.addLayer(layer);
                });

In Scene.js. This actually adds the new layer to scene so that it can be rendered. So in short, I believe you can actually just replace:

// onLoaded(true, mf);

With:

$(document).trigger("MeshFileOpened", [mf]);

Although the correct way would be to define the callback when calling loadZipFromServer().

Again my knowledge of the system is a bit rusty but I hope this will set you in the right direction :)

[nikitassaraf]

Phew! What a day! Well i can now pull a .zip from the server instead of having someone upload it! Works great!

Thank you very much Gabriele! You've been very helpful and understanding! Appreciate your time and efforts.

[Gabryxx7]

I'm glad it works :) Maybe try to mark the issue as solved? Or label it as an enhancement. It would be useful if you can share how you managed to do that for future reference, or at least the process of doing that :)

[nikitassaraf]

Hi Gabriele,

I had a chat with about professor about this enhancement. As its going to be a part of my thesis, it is advised that I post this enhancement after my defense. Would that be okay?