aheckmann / gridfs-stream

Easily stream files to and from MongoDB
MIT License
615 stars 120 forks source link

writeStream hangs in silence #75

Closed vrghost242 closed 9 years ago

vrghost242 commented 9 years ago

versions: ├── mongodb@2.0.28 └─┬ mongoose@3.8.27 └── mongodb@1.4.32 └── gridfs-stream@1.1.1 node v0.12.0

Trying to write a simple function to write a file to a database using writeStream, and does not seem to be able to get it to do anything. I have tested everything I can think of, and the file can be read by the fs stream (tested piping it to stdout). But whenever I try the below code, all that happens is that it gets to the pipe, then silence.

` var mongoose = require("mongoose"); var fs = require("fs"); var mongo = require('mongodb'); var Grid = require('gridfs-stream');

mongoose.connect('mongodb://localhost:27017/BabySounds'); var db = mongoose.connection;

function loadFile(dataCollection, cuID, file){ console.log("Will create id " + cuID + " in the collection " + dataCollection + " and store " + file);

db.once('connected', function (condata) {
    console.log('The bloody database is open!');
    var gfs = new Grid(mongoose.connection.db, mongoose.mongo);
    var writeStream = gfs.createWriteStream({
        _id: cuID,
        filename: file,
        mode: 'w'
    });
    console.log('And now lets write the thing');
    readStream = fs.createReadStream(file);
    readStream.pipe(writeStream);
   writeStream.on('data', function (chunk){
        console.log('Writing some data, just dont know what');
    });
    writeStream.on('end', function (filen) {
        console.log('Written file ' + filen.name);
    });
    writeStream.on('error', function (err) {
       console.log('Got the following error: ' + err);
    });

});

} `

If you call the function loadFile with a collection, id and valid filename, it seems like it gets as far as to readStream.pipe(writeStream);, but after that nothing.

Reggino commented 9 years ago

If you add the eventlisteners to the readStream, do you see anything?

Stuff will be easier for me to debug whenever you add a PR with a unit test...

vrghost242 commented 9 years ago

I tried, and the read stream does get a end of file. Added the following to the end writeStream.on('error', function (err) { console.log('Got the following error: ' + err); }); readStream.on('end', function (filer) { console.log('Got to the end of the input file!'); }); And it does get to the end. I also found that the files does end up in the database, just that the built in mongo viewer in webstorm did not update it's view. So the files end up there, the code gets to the end, but the writestream never receives an end statement.

Happy to do a PR, but not certain what that means:)

Reggino commented 9 years ago

PR => Pull Request :-)

So basically the read-stream does emit the end event, but the writeStream doesn't...

That seems to make sense, since the interface of stream.Writable states that a Writable Stream should emit drain, finish, pipe, unpipe and error.

Does that clarify things?

vrghost242 commented 9 years ago

This does make perfect sense. Seems like most examples I have found indicates that it should be end, and I really could not find anyones stating that there needed to be a finish not end.

Thank you so very very much.