projectglow / glow

An open-source toolkit for large-scale genomic analysis
https://projectglow.io
Apache License 2.0
262 stars 107 forks source link

pipe transformer support for file input/output for command line apps #529

Closed MikePone closed 5 months ago

MikePone commented 1 year ago

Currently it appears that the pipe transformer only support input/output from standard in/out for command line apps. This is a feature request to have the pipe transformer also support command line apps that require input/output through named files on disk rather than standard in/out. Most of the bioinformatics tools we use are built to accept file input/output and not standard in/out.

MikePone commented 1 year ago

One workaround I thought of was to create a shell script that accepts standard in, writes it to a file, calls the command and reads the output file and sends it back to standard out. This would essentially "wrap" any file based command with standard in/out so glow could use it. Glow would call this shell script in the pipe transformer.

williambrandler commented 1 year ago

@MikePone what you suggest is the right answer, cat the output file to STDOUT

There's an example of how to do this in the Glow docs here:

https://glow.readthedocs.io/en/latest/tertiary/pipe-transformer.html#tertiary-pipe-transformer-plink https://glow.readthedocs.io/en/latest/_static/notebooks/tertiary/pipe-transformer-plink.html