search

pokryfka / aws-xray-sdk-swift

Unofficial AWS X-Ray SDK for Swift.
Apache License 2.0
14 stars 4 forks source link

Consider using PureJSONSwift and/or IkigaJSON #18

Closed pokryfka closed 4 years ago

pokryfka commented 4 years ago

Consider using using PureJSONSwift and/or IkigaJSON

The only place JSON encoder is used is XRayUDPEmitter which also uses SwiftNIO.

and XRayLogEmitter which creates an instance an instance of JSON encoder with "pretty" formatting

XRayLogEmitter could be moved to XRayTesting target with dependency on Foundation

pokryfka commented 4 years ago

Consider JSONLogHandler for LogEmitter (?)

ktoso commented 4 years ago

Yeah it would be great to avoid the foundation one, for a number of reasons: 1) perf and 2) avoiding the foundation dependency.

Ping @fabianfett since we talked a while ago about xray and purejson :)

fabianfett commented 4 years ago

@ktoso We are way ahead!

pokryfka commented 4 years ago

@ktoso @fabianfett is there a good explanation why the JSON encoder from Foundation is so slow on Linux (performance is similar on macOS)

got the answer via different channel

pokryfka commented 4 years ago

Result of CodingPerfTests

swift:5.2-amazonlinux2 running in Docker with 2 CPUs and 4 GiB memory on macOS 10.15.6 with 3 GHz 6-Core Intel Core i5, 32 GB:

Number of invocations: 10000
------------------------------------------
JSONValue to bytes
PureSwift                     | took: 0.54722s
------------------------------------------
Encoding
Foundation                    | took: 12.13563s
Ikiga                         | took: 0.38421s
PureSwift                     | took: 1.17607s
------------------------------------------
Reading
PureSwift on [UInt8]          | took: -0.00000s
PureSwift on Foundation.Data  | took: 0.69957s
PureSwift on NIO.ByteBuffer   | took: 2.38591s
------------------------------------------
Parsing
Foundation on Foundation.Data | took: 2.03149s
PureSwift on [UInt8]          | took: 0.50969s
PureSwift on Foundation.Data  | took: 0.50999s
PureSwift on NIO.ByteBuffer   | took: 0.53209s
------------------------------------------
Decoding
Foundation on Foundation.Data | took: 9.99868s
Foundation on NIO.ByteBuffer  | took: 9.98646s
IkigaJSON on [UInt8]          | took: 2.49468s
IkigaJSON on Foundation.Data  | took: 2.50112s
IkigaJSON on NIO.ByteBuffer   | took: 2.50956s
PureSwift on [UInt8]          | took: 1.07665s
PureSwift on Foundation.Data  | took: 1.08228s
PureSwift on NIO.ByteBuffer   | took: 1.09722s

for reference macOS 10.15.6, Swift 5.2.4 on 3 GHz 6-Core Intel Core i5, 32 GB:

Number of invocations: 10000
------------------------------------------
JSONValue to bytes
PureSwift                     | took: 0.46937s
------------------------------------------
Encoding
Foundation                    | took: 2.71739s
Ikiga                         | took: 0.50643s
PureSwift                     | took: 1.48155s
------------------------------------------
Reading
PureSwift on [UInt8]          | took: 0.00000s
PureSwift on Foundation.Data  | took: 0.36042s
PureSwift on NIO.ByteBuffer   | took: 2.37401s
------------------------------------------
Parsing
Foundation on Foundation.Data | took: 0.46644s
PureSwift on [UInt8]          | took: 0.64662s
PureSwift on Foundation.Data  | took: 0.65942s
PureSwift on NIO.ByteBuffer   | took: 0.66094s
SwiftyJSON                    | took: 1.99765s
------------------------------------------
Decoding
Foundation on Foundation.Data | took: 3.09398s
Foundation on NIO.ByteBuffer  | took: 3.11534s
IkigaJSON on [UInt8]          | took: 3.06283s
IkigaJSON on Foundation.Data  | took: 3.06483s
IkigaJSON on NIO.ByteBuffer   | took: 3.06710s
PureSwift on [UInt8]          | took: 1.61549s
PureSwift on Foundation.Data  | took: 1.61314s
PureSwift on NIO.ByteBuffer   | took: 1.61946s

Takeaways:

pokryfka commented 4 years ago

PureSwift JSON encoder returns an array of bytes:

public struct PSJSONEncoder {

    public var userInfo: [CodingUserInfoKey : Any]

    public init()

    public func encode<T>(_ value: T) throws -> [UInt8] where T : Encodable

    public func encodeAsJSONValue<T>(_ value: T) throws -> PureSwiftJSON.JSONValue where T : Encodable
}

IkigaJSON JSON encoder uses Foundation.Data

public struct IkigaJSONEncoder {

    public var userInfo: [CodingUserInfoKey : Any]

    /// These settings influence the encoding process.
    public var settings: IkigaJSON.JSONEncoderSettings

    public init()

    public func encode<E>(_ value: E) throws -> Data where E : Encodable
}
pokryfka commented 4 years ago

(Average) results in seconds of PerformanceTests in https://github.com/pokryfka/aws-xray-sdk-swift/pull/42 from check_run_id=934347557

Test test-macos test-linux (swift:5.2) test-linux (swiftlang/swift:nightly-5.3-bionic)
testEncodingUsingFoundationJSON 0.692 0.417 0.413
testEncodingUsingIkigaJSON 0.564 0.062 0.076
testEncodingUsingPureSwiftJSON 0.750 0.079 0.097

Note relative standard deviation is ~40% on macos and ~3% on linux (?).

pokryfka commented 4 years ago

Did some testing, key takeaways:

@fabianfett @ktoso

Using Foundation JSON encoder does not make sense in library targeted for Linux because of its much worse performance.

However I am not sure how much the dependency on Foundation.Data is an issue. For example I did not test if, and by how much, it would reduce the binary size, loading time and so.

ktoso commented 4 years ago

Using Foundation JSON encoder does not make sense in library targeted for Linux because of its much worse performance.

Yep. It's something we're painfully aware of;

I don't have a concrete plan yet about what we'll do there but yes it's a huge pain point for the server ecosystem and we'll need to improve the situation here. Status quo basically is: avoid Foundation (in general) on server if possible, and definitely avoid Foundation's coders if you can by depending on Pure or Ikiga.

PureSwift JSON encoder returns an array of bytes: [...] [...] However I am not sure how much the dependency on Foundation.Data is an issue.

If possible avoiding foundation entirely (so also Data) would be good here... Sadly we don't have a different "currency type" for "bag of bytes" today, so that makes interop painful (just array of bytes or NIO types remain). Hopefully we'll address this at some point though; for now you could do the same as Pure does; after all, these types never show up in user API for you right?

However I am not sure how much the dependency on Foundation.Data is an issue. For example I did not test if, and by how much, it would reduce the binary size, loading time and so.

It is definitely noticeable and painful to even link Foundation, it hurts cold startup time a lot. I don't have the numbers on me right now, but Fabian I think tested that at some point.

pokryfka commented 4 years ago

@ktoso

If possible avoiding foundation entirely (so also Data) would be good here... Sadly we don't have a different "currency type" for "bag of bytes" today, so that makes interop painful (just array of bytes or NIO types remain). Hopefully we'll address this at some point though; for now you could do the same as Pure does; after all, these types never show up in user API for you right?

if not for the Foundation dependency IkigaJSON seems better choice:

It is definitely noticeable and painful to even link Foundation, it hurts cold startup time a lot. I don't have the numbers on me right now, but Fabian I think tested that at some point.

@fabianfett

I would be very interested to see any results here. This is especially important for lambdas.

Maybe will try to play with that a bit in my Swift lambda template/playground.

pokryfka commented 4 years ago

@fabianfett @ktoso

IMO any swift-server library using JSON encoder/decoder should:

  1. include default implementation
  2. dont say "dont use the default implementation!" (see Swift AWS Lambda Runtime)
  3. considering the issue is not resolved, provide an easy way to change the JSON encoder/decoder

did some reorganization as far aws-xray-sdk-swift is concerned: https://github.com/pokryfka/aws-xray-sdk-swift/pull/42

there is new AWSXRaySDK providing JSON encoder implementation (PureSwiftJSON atm)

planning to do more performance testing of how Foundation affects Lambda cold start as well as IkigaJSON vs PureSwiftJSON;

IkigaJSON seems to be about 20% faster than PureSwiftJSON atm, not sure how much of an effort is removing dependency on Foundation in IkigaJSON; or perhaps bring in some of the performance from IkigaJSON to PureSwiftJSON