search

zenogantner / MyMediaLite

recommender system library for the CLR (.NET)
http://mymedialite.net
499 stars 192 forks source link

Item recommendation could support streaming output, simple format #430

Closed jkleint closed 10 years ago

jkleint commented 10 years ago

Being able to stream recommendations to another process would save the time required to write out and read back in large temp files. Unfortunately, the I/O routines used don't seem to support streaming writes. For example, running item_recommendation with --prediction-file=/dev/stdout crashes with The stream does not support seeking at System.IO.FileStream.SetLength in MyMediaLite.ItemRecommendation.Extensions.WritePredictions. There doesn't seem to be any need for seeking here, but I'm unfamiliar with the .NET I/O subsystem.

It would also be nice to have the output in a form that's easy to parse, say maybe one user item score triple to a line.

zenogantner commented 10 years ago

Will have a look at this.

zenogantner commented 10 years ago

So, there is seeking, because files are not opened for appending by StreamWriter() when using the default constructor. Just opening with the append flag does not seem to work, so I created a fix that uses Mono's Unix API to open files in /dev.

Can you verify the fix, please?

Maybe we should also think about generally offering appending instead of overwriting. What do you think?

zenogantner commented 10 years ago

I moved the second thing to a separate issue, closing this ticket. Please re-open if the solution does not work for you.

jkleint commented 10 years ago

Thanks, I will give it a try. If you can write to stdout, you can already append with the shell: >>. No need to complicate the code.