Small amount of documentation for contributors (e.g. how to write a transform)

[rufuspollock]

@andylolz I'd like to add a transform. I looked at code https://github.com/okfn/datapipes/blob/master/lib/operators.js (which looks excellent) but wasn't entirely sure what I need to do to add an

I wonder if a few short comments at the top of the file (or documenting one operator in detail) would make this a lot easier for contributors.

Alternatively I wonder if our support for arbitrary map functions will obviate the need for this (will we soon have a function called mapToTransform that turns a standard map function into a transform?)

@andylolz wdyt?

[andylolz]

I wonder if a few short comments at the top of the file (or documenting one operator in detail) would make this a lot easier for contributors.

Absolutely. I’ll add that in.

will we soon have a function called mapToTransform that turns a standard map function into a transform?

In the map op pull request there’s a transform that calls different bits of a standard map op in different operations. It works that way because we are operating in a sandbox. A mapToTransform function sounds cool, but I am not sure we can do that while still using the sandbox.

[andylolz]

Hopefully 79792aab is useful. I’ll add a bit more…

[andylolz]

@rgrp would a mapToTransform function mean that internal operators could be defined in exactly the same way as external operators (i.e. those run through map)? If so, I think I really like that idea.

[davidmiller]

Not sure I understand what the idea is :s

[rufuspollock]

@davidmiller logic here is writing some docs to explain how to create new transforms.

@andylolz re mapToTransform I guess I'm thinking of a function that takes a map function and does the work to turn it into an appropriate stream transform (i.e. take care of the boilerplate). e.g. suppose i had:

// question: do map functions have to be sync or can they be async??
// if async we would have mapRow(row, idx, callback)

// stupid map function that makes first column value = 'hello'
function helloMap(row, idx) {
  row[0] = 'hello'
  return row;
}

operators['hello'] = mapToTransform(helloMap);

[rufuspollock]

@andylolz is this being addressed by #105?

[rufuspollock]

FIXED. @andylolz I'm closing this as think we covered this both in #105 and in https://github.com/okfn/datapipes/commit/9bbb841b00f6b48bd4c7445f560fb1066d1c33fc