Use BrowserFS to get a web page running

ppegusii commented 10 years ago

I get the following error when trying to get the BrowserFS example working using localstorage.

{"type":1,"code":"ENOENT","message":"/: No such file or directory."}

This is the HTML document.

<!DOCTYPE html>
<html>
<head>
<!--<script src="compiled/lib/jquery-2.1.1.min.js" type="text/javascript"></script>-->
<!--
  <script data-main="release/interpret-pyc" src="node_modules/requirejs/require.js"></script>
  -->
  <script src="compiled/lib/browserfs.min.js" type="text/javascript"></script>
</head>
<body>
  <h1>Python Interpreter Test</h1>
  <input type="file" onchange="handleFiles(this.files)"></input>
  <div id='out'>
  </div>
  <script type="text/javascript">
    BrowserFS.install(window);
    var lsfs = new BrowserFS.FileSystem.LocalStorage();
    BrowserFS.initialize(lsfs);
    var fs = require('fs');
    fs.writeFile('/sample.txt', 'Testing!', function(err){
      if(err){ console.log(JSON.stringify(err)); }
    });
/*
    function handleFiles(fileList){
      console.log(fileList[0]);
      var reader = new FileReader();
      reader.onload = (function(theFile){
        return function(pyc){
          console.log(pyc);
          console.log(theFile.name);
          console.log(pyc.target.result);
          BrowserFS.install(window);
          var lsfs = new BrowserFS.FileSystem.LocalStorage();
          BrowserFS.initialize(lsfs);
          var fs = require('fs');
          fs.writeFile('test.txt', 'Testing!', function(err){
            if(err){ console.log(err); }
          });
          //fs.writeFile('/'+theFile.name, pyc.target.result, function(err){
          //  if(err){ console.log(err); }
          //});
        };
      })(fileList[0]);
      reader.readAsBinaryString(fileList[0]);
    }
*/
  </script>
</body>
</html>

jvilk commented 10 years ago

I just tried this with browserfs.min.js from 0.3.5, and... that code works with no modifications!

One weird possibility is that local storage contains garbage data. Can you try flushing it with localStorage.clear() to see if this fixes the issue?

jvilk commented 10 years ago

(Built code located here: https://github.com/jvilk/BrowserFS/releases/tag/v0.3.5 )

ppegusii commented 10 years ago

localStorage.clear() fixed the problem. Thanks!

ppegusii commented 10 years ago

I'm using FileReader.readAsText(File) on a file uploaded from an <input type="file"></input> tag. That method of FileReader should use UTF-8 encoding. I attach a handler to the onload event of the FileReader which receives a ProgressEvent. In that handler, I write to localstorage using BrowserFS's fs.writeFile('filename', ProgressEvent.target.result, 'utf8', cb). The problem is I seem to be using incorrect character encodings or the wrong readAs... method of FileReader. I've tried some other readAs... and fs.writeFile encoding combinations, but can't find one that works.

jvilk commented 10 years ago

utf8 should be the string to identify UTF-8 encoding. utf is ambiguous, as there's utf8, utf16, and utf32.

ppegusii commented 10 years ago

Sorry that was a typo. I did specify 'utf8'.

jvilk commented 10 years ago

Have you opened the debugger and verified that result is a string? What's the error? (Chrome DevTools are quite useful for this sort of thing.)

jvilk commented 10 years ago

Oh, and I should mention:

fs.writeFile('filename', ProgressEvent.target.result, 'utf8', cb)

This states, "take the input JavaScript string, and write it to a file in UTF-8 format". So if it's not a JavaScript string, it won't work.

ppegusii commented 10 years ago

I have verified that result is a string with typeof. The error occurs during parsing. Python Marshal prefixes serialized objects with a utf8 character that indicates the type of the object that follows. When parsing, the parser reads a utf8 character from the file that is not in the set of type characters. This does not happen when running the parser in node on the same pyc file.

jvilk commented 10 years ago

Python Marshal prefixes serialized objects with a utf8 character that indicates the type of the object that follows.

What error are you getting? Where is it getting thrown? Are you trying to write a binary file using UTF-8 encoding? Not all bytes are valid UTF-8 characters, and my UTF-8 encoding function verifies that it is not writing invalid characters. (EDIT: With that said, if it's already a JavaScript string (which are UTF-16), then... maybe there's a bug in my code.)

Also, you should use reader.readAsArrayBuffer;, which makes result an ArrayBuffer. The BrowserFS Buffer implementation has a zero-copy constructor that takes an array buffer (just use new Buffer(result)).

ppegusii commented 10 years ago

The error is Uncaught TypeError: undefined is not a function. It is thrown here, when the character extracted from the buffer is not a key in typeParserMap. So the result of the lookup is undefined and then I happily call it.

If you still think it's a problem BrowserFS should have detected. I'll give you any more information you need.

I will refactor my code to to accept a NodeBuffer, rather than a file system. Muchas gracias!

jvilk commented 10 years ago

Oh, then it's likely not a code in my bug, just that you were doing the wrong thing w.r.t. writing the file in UTF-8 format, since pyc files are binary files. Meaning, you were:

Reading a pyc file using a FileReader.
Instructing the FileReader to interpret the file as a UTF-8 string, and return it as a JavaScript string (equivalent to you opening the pyc file in a text editor).
Instructing BFS to write the JavaScript string as a UTF-8 string.

Not all bytes are valid UTF-8 characters, so who knows what FileReader does when the file is invalid (it might just drop those bytes). So the written file on the BFS side in this procedure is unlikely to match the original file.

Reading the file as an array buffer correctly preserves the data.

ppegusii commented 10 years ago

That makes sense.

ppegusii commented 10 years ago

Refactored to use NodeBuffer. All set.

nbr / ts-python-interpreter

Use BrowserFS to get a web page running #8