the tokenizer should parse bibliographic metadata from the supplied input file. (this is expanding the functionality of the tokenizer beyond a pure tokenizer service, but it make sense from an efficiently perspective, so that we don't have to read and parse the input files multiple times)
the tokenizer should parse bibliographic metadata from the supplied input file. (this is expanding the functionality of the tokenizer beyond a pure tokenizer service, but it make sense from an efficiently perspective, so that we don't have to read and parse the input files multiple times)