Am I trying to get data wrong? Is it only accessible via streams?

[brucejo75]

I tried fc.findOne, expecting that the data would be the result of what I selected. Looking more closely at the docs I think I need to use fc.findOneStream? Now I am digging into how to use streams, blech. Async is great, but I value meteor for the fact that they take care of the fiberization of the code so I get best of both worlds and I would rather avoid async programming if I can.

My application involves files that are typically much less than 100K. It would be great to have a simple way to read the files once they are in the collection. I realized looking at the docs and looking at the sample that the "get data" scenario is not really covered well.

Am I missing something? Is there a way to simply get the data?

Thanks!

[vsivsi]

Hi, MongoDB gridFS on node.js (and therefore Meteor) using the native driver is inherently asynchronous and stream driven. Streams are a key part of (and one of the best ideas in) node.js.

Implementing this package without using streams would be untenable, from an implementation complexity, efficiency and best practices / pattern perspective.

Meteor.js omits any support for this type of thing because it doesn't play especially well with Fibers. Programming with Fibers is still async programming, it is just hiding many of the messier details from you. Unfortunately, the fiber abstraction breaks for efficiently moving around large amounts of data, which is necessary for many real world server applications.

If your application doesn't require that, you are free to forgo using gridFS through a package like file-collection and just store your files as binary buffers within ordinary MongoDB documents. So long as your files are never larger than around 15MB, that is a perfectly valid (and arguably much simpler) solution.

gridFS is nothing more than a set of conventions that build on top of that base functionality to make larger file sizes and the efficiency/practicality of streaming to support those larger files possible.

See: http://docs.meteor.com/api/ejson.html#EJSON-newBinary

[brucejo75]

OK, so I am not missing anything...

I took a look at GridFS and it is pretty sparse on how to retrieve the files too. It is a real problem in using the interface for uploaded data that a user would want to process. Have you considered a convenience function?

e.g. You could have findOneData that returns the data if the file is below a certain size (e.g. less than one chunk or some small number of chunks). Otherwise it fails out.

I suppose I could write this accessing the chunk data directly, I will put it in my sample...and you can take a look.

PS: I have figured out how to use the stream, so this is mostly an enhancement request based on the time it took me to figure this out.

[vsivsi]

Turning a readable stream into a buffer is a pretty simple node.js operation. It is dangerous if you don't know how large the stream will ultimately be, but for a gridFS file, that is not the case. Here are some packages to try or mine for insights. https://www.npmjs.com/search?q=stream+into+a++buffer

[brucejo75]

Thanks! I like the looks of stream-with-known-length-to-buffer I will stick it in my sample...

[brucejo75]

In my sample, and with that I think I am done! Closing the question. Thanks much.