search

dlangBugzillaToGithub / migration_test

0 stars 0 forks source link

Efficient std.conv.to conversions #676

Open dlangBugzillaToGithub opened 13 years ago

dlangBugzillaToGithub commented 13 years ago

tomeksowi reported this on 2011-02-08T12:47:58Z

Transfered from https://issues.dlang.org/show_bug.cgi?id=5548

CC List

Description

Currently most to!T conversions allocate behind the scenes. I propose that the conversions where T is dynamically sized (like an array) get a speedy overload of the form:

void to(T, S, O)(S source, O output) if (isOutputRange!(O, T));

Example: to!string(9234820934, appender);

For further investigation: consider assuming the output range to be buffered.
dlangBugzillaToGithub commented 13 years ago

schveiguy commented on 2011-02-08T12:52:10Z

Do we not have the functionality for this (at least for strings) in std.format?

Are there use cases for this beyond strings?
dlangBugzillaToGithub commented 12 years ago

k.hara.pg commented on 2012-06-14T02:33:25Z

I agree with Steven. If you want to represent objects with output range of characters, you can use std.format.formatValue family directly.
dlangBugzillaToGithub commented 11 years ago

blah38621 commented on 2013-11-01T08:26:19Z

After a bit of testing, std.format.fomatValue is not a valid alternative to having std.conv.to provide overloads that accept an output buffer. The reason for this is simple, the following code is 3x slower than simply using to!string(int):

auto toStr = benchmark!(() {
    import std.format;
    import std.range : Appender;

    auto ret = Appender!string();
    auto fmt = FormatSpec!char("%s");
    ret.reserve(4096);
    for (auto i = 0; i < ObjectCount * 11; i++)
    {
        ret.formatValue(i, fmt);
        ret.clear();
    }
})(1);

writefln("Took %s ms (%s) to serialize 100k SimpleObjects with an average payload of %s bytes (%s).", res[0].msecs, toStr[0].msecs, cast(real)totalPayload / ObjectCount, totalPayload);

In my tests where ObjectCount was 100k, it takes 400ms for to!string(int) to create all the strings, and 1100ms for formatValue to do the same. formattedWrite is even worse, 1500ms. In my current implementation of a dynamic JSON (de)serializer, more than half of my time is eaten up by converting integers to strings when performing deserialization. I use a pre-allocated output range as the destination, so I know I'm not doing any allocations within my code.
dlangBugzillaToGithub commented 7 years ago

jack commented on 2017-02-02T16:03:27Z

Definitely agree that this should exist. Currently in the array overloads, there are a lot of uses of functions that use appender internally and then return the managed array with app.data(), only to be appended to another appender in the parent function. Having OutputRange overloads would represent a significant performance boost.

Once https://github.com/dlang/phobos/pull/5018 is pulled, I'll start working on this.

BTW, in the mean time, converting to string can be used with OutputRanges via std.range.put and std.conv.toChars.