eBay / tsv-utils

eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.
https://ebay.github.io/tsv-utils/
Boost Software License 1.0
1.43k stars 80 forks source link

bufferedByLine does not work with File due to @safe <> @system conflict #282

Closed tastyminerals closed 4 years ago

tastyminerals commented 4 years ago

This is the first time I am using tsv-utils. I am trying to compile the following chunk of code:

import std.stdio;
import std.typecons : Yes, No;
import tsv_utils.common.utils : bufferedByLine;

enum testFile = "test.tsv";

void main(string[] args)
{
    foreach (line; testFile.File().bufferedByLine!(Yes.keepTerminator))
    {
        writeln(line);
    }
}

and get the following exception:

> dub build --single preprocessor.d --compiler=ldc2 --force
Performing "debug" build using ldc2 for x86_64.
mir-core 1.1.2: building configuration "library"...
mir-algorithm 3.7.28: building configuration "default"...
preprocessor ~master: building configuration "application"...
C:\Users\tasty\AppData\Local\dub\packages\tsv-utils-1.6.0\tsv-utils\common\src\tsv_utils\common\utils.d(899,38): Error: @safe function tsv_utils.common.util
s.bufferedByLine!(cast(Flag)true, char, cast(ubyte)10u, 131072LU, 16384LU).bufferedByLine.BufferedByLineImpl.popFront cannot call @system function std.stdio
.File.rawRead!ubyte.rawRead
C:\ldc2-1.20.0-windows-x64\bin\..\import\std\stdio.d(1076,9):        std.stdio.File.rawRead!ubyte.rawRead is declared here
preprocessor.d(29,40): Error: template instance tsv_utils.common.utils.bufferedByLine!(cast(Flag)true, char, cast(ubyte)10u, 131072LU, 16384LU) error instan
tiating
ldc2 failed with exit code 1.

Am I using it correctly?

jondegenhardt commented 4 years ago

Thanks for the report. Yes, far as I can tell it you are using it correctly. When I compile your main I do not see this bug. (I'm calling dmd and ldc directly.)

Looking at the phobos library, this looks like a difference between rawRead on Unix/MacOs vs Windows. (tsv-utils are tested on Unix and MacOS, but not Windows.) On Windows there is a call to an @system function. On Unix and other non-Windows platforms rawRead calls trustedFread, which has been marked @trusted, so it can be called from @safe code.

I should probably remove the @safe attribute on the popFront call. I'll do that as part of the next point release. It'll be a few days before I can make this change though. If you want to try making the change locally, the place to make it is here

jondegenhardt commented 4 years ago

Hopefully PR #283 addresses this case. dub won't pick up the new version until I tag a new version, and I might not get to for a few days. (There's some additional work on my part.)

jondegenhardt commented 4 years ago

@tastyminerals - A new release (v1.6.1) with the change has been tagged and is available via dub. Hopefully this enables compilation.

I'm closing this for now. When you get a chance I'm hoping you can test in your environment and report if it worked or not. If it didn't work, reopen this issue or a new issue as appropriate.

Thanks again for the report.