briangonzalez / gulp-file-contents-to-json

Slurp in some files, output a JSON representation of their contents.
12 stars 5 forks source link

Add support for binary files (base64) #1

Open alexewerlof opened 9 years ago

alexewerlof commented 9 years ago

This plugin works great for us embedding .glsl files (and generally other source codes). However, we really need to embed binary files as well. Currently we use a base64 encoder to convert .raw file contents into json like this:

var files = fs.readdirSync(path.binaries);
    files.forEach(function (filename) {
      if(/\.raw$/.test(filename)) { // if filename ends with .raw
        var data = fs.readFileSync(path.binaries + filename); // read raw binary file to a buffer object
        fs.writeFileSync(path.binaries + filename + '.txt', data.toString('base64')); // encode the binary data as base64 and write back to file suffixed with .txt
      }
    });

And then use your plugin to add the file contents into json:

    return gulp.src(path.binaries + '*.txt')
        .pipe(rename({extname: ''})) // Removes .txt
        .pipe(fc2json(path.bundleFile))
        .pipe(header('binaries = '))
        .pipe(gulp.dest(path.src + 'binaries/'));

However this two step process is tedious and error prone. It would be really awesome if the plugin could deal with binary files as well.

briangonzalez commented 9 years ago

Hmm, would you care to take a stab at this?

alexewerlof commented 9 years ago

sure.

alexewerlof commented 9 years ago

I didn't understand the role of guid and nconf in the code. Would you explain?

briangonzalez commented 9 years ago

Nconf is a in-memory/disk store which does a stellar job of storing values for IDs that look like this: foo:bar:baz.txt. I translate / to :, which nconf likes, then I store the value, then later, I ask nconf for the value for that ID and write it to disk.

Take this example:

var path = 'foo/bar/baz';
var id = path.replace(file.base, '').split('/').join(':');  // 'foo/bar/bax.txt' => 'foo:bar:baz.txt'
var contents = "Hello world.";
nconf.set(id, contents);

nconf.get(id, contents); 

// { foo: { bar: { baz: "Hello, world." }  } }

Since nconf uses store.json to persist contents to disk, I've created a guid to namespace the current project's data to avoid conflicts with other gulp tasks running in the same folder.