Let's kill stderr

[piranna]

Not sure if runtime or NodeOS purpose, but whatever.

stderr is too much unix, it's a gross-grain console level while we have a more fine grained one on the console object (log, info, warning, error, and sometimes also debug and Python logging module has exception). I think it would be better to have only just a stdout stream and no stderr stream at all, so this way all messages go to just one only place. Later this messages can be filtered both "online" by setting the log level as a environment variable instead of redirect the content of stderr (2 > /dev/null) and also "offline" if the console content is output to a file and later filtered, for example if it's added some kind of metadata to the log lines, like a word at beggining or a special invisible character (unicode has several of them). This way it's stored the full log in just one file and in order, but later you can select what messages level you want to read.

[heapwolf]

is this a joke? otherwise i totally don't understand why you wouldn't want stderr. its an incredibly useful concept.

[RangerMauve]

I don't think that the kernel should have whatever log filtering format be a part of it. Also, getting rid of stderr might make it harder to integrate with node in the future.

[piranna]

No, it's not a joke. And yes, it's useful, but as I told before, it's useful as a gross-grain console level mechanism, while Javascript has a fine-grain one that's more versalite, that's why I think we should use it instead. You can always use console.warning() and console.err() as your stderr, only that's more explicit.

[piranna]

@RangerMauve, I don't think this would make this more dificult to integrate since you can always redirect the output and set to it a log level... Just think diferent :-)

[RangerMauve]

@piranna if we're thinking different, why even have an stdout? Javascript doesn't have a concept of pipes between processes out of the box. It does have the console, but that can just be a module that has nothing to do with the *nix shell's way of doing things.

[heapwolf]

The point of stdout, stderr and stdin is to allow the output of one program to be the input of another.

Having a broken version of this model doesn't make any sense. But I think it's interesting to consider a new model. How would that new model work?

[RangerMauve]

What if the communication was done through references to streams. A JS process can pass anything as an argument to another process, right? Then we could just have the communication between them be done entirely in Javascript rather than with some construct specific to the kernel.

[RangerMauve]

One thing that this might mean is that we could transfer structured data like JS objects of primitives rather than just text/binary streams. This could make inter-process communication a lot more powerful.

[piranna]

Sorry for the delay.

Seems there are some consensus that this is not a kernel issue, but instead a userspace one (that I think is good :-) ). It's cool the idea of being console an external module so you can change it's behaviour, in fact it's a wrapper on top of process stdout and stderr...

On bash.js (no public code yet) I was trying to create Node.js versions of command line tools as functions, returning objects for ls, grep, sort... as "stdout" instead of plain ol' strings and throwing exceptions as "stderr", and set this functions "stdin" by calling them setting the input data as 'this' (ls().sort(), or var a = ls('-a'); grep.call(a, '^\.')) and showing their results on console in a nicer way with util.inspect as REPL does. Main problem with this approach is, besided the fact that there could be some heterogeneous results, that seems basic commands can output results both to stdout and stderr (exec ls with some existent and non-existent directories). This could be fixed by returning Error objects, that lead me to think why to have several output streams (stdout and stderr) while they could be output to just only one with some metadata of what notify level (debug, log, info, warning, error) they are. This seems more easy and flexible, and could easily being done with the objectMode of Node.js streams that seems similar to what @RangerMauve is talking about...

[iefserge]

Yea. The idea is to put most of the services into userspace. This includes standard streams which could be provided as part of the program environment.

I like the idea of bash.js. Runtime.js would definitely need something like this at some point. As well as easy way to write Bash-like scripts in JavaScript.

[piranna]

I think I've done an error with bash.js, since the most versatil interface are streams: you can be able to do async operations with streams, but how do you do it returning a value? Only option is to use promises, that are not ready yet... I could publish my current code so you can inspect it and give me advices if you want... The most advanced is ls though, and this is because it has a lot of code inherited from a patch I did for shell.js...

[iefserge]

Maybe use object streams instead of strings. Like ls returns a stream of objects where each one contains info about file. And, grep, for example, can parse objects too. This basically means, you can write objects to stdout.

Not sure if it's a good idea though.

[RangerMauve]

I don't really see what potential downsides can exist to having an object stream. It'd mean you wouldn't need to parse things, and I reckon it wouldn't be too slow either.

[piranna]

Seems you both got the point of returning objects :-) Return a stream of objects would be an alternative to return an array of objects or a generator or a promise, only I wanted to use primitive objects (arrays) as possible. Do you think a stream of objects is an acceptable trade-off for an "infinite array" as Python generators would do?

I've uploaded my current code to https://github.com/piranna/bash.js, please look at ls and cat since they are the most advanced ones to get an idea what I'm trying to achieve. ls return a list of stat objects with a name entry (I'm not sure if it should be map of name:stat, but whatever), and representation is done by Node.js REPL internal calling to objects inspect method, that according to the ls flags it shows data in one or other format. It's said, inspect owns the CLI representation of the data, the V in MVC :-)

[iefserge]

One problem with strings stdout stream is colors support. I still can't find good solution for this.

[heapwolf]

So basically, standard stream provide a lot of useful compatibility for OS developers. This is not to say that there isn't room for innovation in the process communication model. I'll close this for now and we can continue to discuss process model in another in a new, more specific thread.