[FEATURE] file validators should handle stdin and stdout properly

[h-2]

Does this problem persist on the current main?

• [X] I have verified the issue on the current main

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

Passing "-" to an input_file_validator fails with

what():  Validation failed for positional option 1: The file "-" does not exist!`

Passing "/dev/stdin" also fails:

what():  Validation failed for positional option 1: Expected a regular file "/dev/stdin"!

Expected Behavior

Both of these values should work.

Both of them should also be exempt from extension-checks (or there should be another validator called input_file_or_stdin_validator or maybe even just input_validator because it covers all possible inputs?).

P.S.: independent of "/dev/stdout", I think it is too restrictive to only allow regular files. Being able to open FIFOs is useful, too.

Steps To Reproduce

try it :)

Environment

- Operating system: Ubuntu 22.04
- Sharg version: 89bb8e71172ceea20b95beed5e071fe62b13d91c
- Compiler: gcc-11.3

Anything else?

I like the config being switched to designated initialisers 👍🏻

Suggestion: the documentation would be easier to read if validators and exceptions were in their own folder, because there would be fewer items cluttering the top-level of the documentation. They could even have their own namespace (sharg::validator::input_file ...).

[smehringer]

Hi @h-2, these are good points! We'll discuss them soon.

[eseiler]

Just dumping an idea: Maybe we could do something like sharg::input_validator{sharg::file_mode::file_only} and sharg::input_validator{..::all}.similar to the output_file_validator and with sensible names and default. Then we should change the output validator too.

I agree with the documentation suggestion of adding some more folders.

[smehringer]

Core Meeting 29.09.2022

Tasks after decisions:

• [ ] drop the restriction that input_file_validator must be a is_regular_file to also allow FIFO, sockets etc.
• [ ] input_file_validator does not check extensions for - and /dev/stdin
• [ ] output_file_validator does not check extensions for - and /dev/stdout

This is not API relevant because file validators are still experimental.

[smehringer]

@h-2 In your specific use case, Do you want to use stdin together wit seqan3 files? Because how do you determine the format, if previously, when giving a normal filename, automatic format detection based on extension is happening?

Do you know why we do not have automated format detection? > -> FASTA @ -> FASTQ Sam / BAM has a magic string in the beginning

[h-2]

I am not using it with SeqAn3 but with BioC++ I/O where the format can be selected manually (and will likely offer magic byte detection as well).

[smehringer]

Thanks for the quick reply! Does BIO not have file extension detection?

[h-2]

Thanks for the quick reply! Does BIO not have file extension detection?

It does... but it also allows specifying the format manually. To me this unrelated to argument parsing.

[eseiler]

So, this means you want to use - as input with an input_file_validator, but not with format validation?

I.e., it would be good enough for input_file_validator{} to work, but input_file_validator{{"fa", "fasta"}} not?

[h-2]

So, this means you want to use - as input with an input_file_validator, but not with format validation?

What I meant was, that the detection in the reader is independent of the argument parsing.

I.e., it would be good enough for input_file_validator{} to work, but input_file_validator{{"fa", "fasta"}} not?

I would hope that this works. I think - should just be ignored when verifying extensions, i.e. it should just be successfull.

[h-2]

Any news on this?

I would actually recommend just creating additional validators. It's very straightforward:

class input_file_or_stdin_validator : public sharg::input_file_validator
{
private:
    using base_t = sharg::input_file_validator;
public:

    using base_t::base_t;

    virtual void operator()(std::filesystem::path const & file) const override
    {
        if (file != "-" && file != "/dev/stdin")
            sharg::input_file_validator::operator()(file);
    }
};

class output_file_or_stdout_validator : public sharg::output_file_validator
{
private:
    using base_t = sharg::output_file_validator;
public:

    using base_t::base_t;

    virtual void operator()(std::filesystem::path const & file) const override
    {
        if (file != "-" && file != "/dev/stdout")
            sharg::output_file_validator::operator()(file);
    }
};