fluid-json

A JSON library written in pure Kotlin.

Table of Contents

• Installation
• Basic Usage
• Annotation Customization
• Examples
• Manual Coding
• Error Handling
• Ktor Serialization
• Modules
• Testing
• Type Mapping
• Architecture
• Future Planning

Installation

build.gradle.kts:

plugins {
    kotlin("kapt")
}

dependencies {
    kapt("io.fluidsonic.json:fluid-json-annotation-processor:1.5.0")
    implementation("io.fluidsonic.json:fluid-json-coding-jdk8:1.5.0")
}

If you cannot use Java 8, e.g. when supporting Android API 25 or below, replace fluid-json-coding-jdk8 with fluid-json-coding.

If you're using IntelliJ IDEA (not Android Studio) then you have to manually enable the following project setting in order to use annotation processing directly within the IDE (this is an open issue in IntelliJ IDEA):
Preferences > Build, Execution, Deployment > Build Tools > Gradle > Runner > Delegate IDE build/run actions to gradle

Basic Usage

fluid-json uses @Json-annotations for automatically generating codec classes at compile-time which are responsible for decoding and encoding from and to JSON.
You can also create these codecs on your own instead of relying on annotation processing.

import io.fluidsonic.json.*

@Json
data class Event(
    val attendees: Collection<Attendee>,
    val description: String,
    val end: Instant,
    val id: Int,
    val start: Instant,
    val title: String
)

@Json
data class Attendee(
    val emailAddress: String,
    val firstName: String,
    val lastName: String,
    val rsvp: RSVP?
)

enum class RSVP {
    notGoing,
    going
}

Next you create a parser and a serializer that make use of the generated codecs:

import io.fluidsonic.json.*

fun main() {
    val data = Event(
        attendees = listOf(
            Attendee(emailAddress = "marc@knaup.io", firstName = "Marc", lastName = "Knaup", rsvp = RSVP.going),
            Attendee(emailAddress = "john@doe.com", firstName = "John", lastName = "Doe", rsvp = null)
        ),
        description = "Discussing the fluid-json library.",
        end = Instant.now() + Duration.ofHours(2),
        id = 1,
        start = Instant.now(),
        title = "fluid-json MeetUp"
    )

    val serializer = JsonCodingSerializer.builder()
        .encodingWith(EventJsonCodec, AttendeeJsonCodec)
        .build()

    val serialized = serializer.serializeValue(data)
    println("serialized: $serialized")

    val parser = JsonCodingParser.builder()
        .decodingWith(EventJsonCodec, AttendeeJsonCodec)
        .build()

    val parsed = parser.parseValueOfType<Event>(serialized)
    println("parsed: $parsed")
}

Prints this:

serialized: {"attendees":[{"emailAddress":"marc@knaup.io","firstName":"Marc","lastName":"Knaup","rsvp":"going"},{"emailAddress":"john@doe.com","firstName":"John","lastName":"Doe","rsvp":null}],"description":"Discussing the fluid-json library.","end":"2019-03-05T00:45:08.335Z","id":1,"start":"2019-03-04T22:45:08.339Z","title":"fluid-json MeetUp"}

parsed: Event(attendees=[Attendee(emailAddress=marc@knaup.io, firstName=Marc, lastName=Knaup, rsvp=going), Attendee(emailAddress=john@doe.com, firstName=John, lastName=Doe, rsvp=null)], description=Discussing the fluid-json library., end=2019-03-05T00:45:08.335Z, id=1, start=2019-03-04T22:45:08.339Z, title=fluid-json MeetUp)

(nope, no pretty serialization yet)

Annotation Customization

In this section are a few examples on how JSON codec generation can be customized.

The full documentation on all annotations and properties controlling the JSON codec generation can be found in the KDoc for @Json.

Collect all generated codecs in one codec provider

All codecs in your module generated by annotation processing can automatically be added to a single codec provider which makes using these codecs much simpler.

@Json.CodecProvider
interface MyCodecProvider : JsonCodecProvider<JsonCodingContext>

fun main() {
    val parser = JsonCodingParser.builder()
        .decodingWith(JsonCodecProvider.generated(MyCodecProvider::class))
        .build()
    // …
}

Customize the generated codec

@Json(
    codecName = "MyCoordinateCodec",            // customize the JsonCodec's name
    codecPackageName = "some.other.location",          // customize the JsonCodec's package
    codecVisibility = Json.CodecVisibility.publicRequired  // customize the JsonCodec's visibility
)
data class GeoCoordinate2(
    val latitude: Double,
    val longitude: Double
)

Customize what constructor is used for decoding

@Json(
    decoding = Json.Decoding.annotatedConstructor  // require one constructor to be annotated explicitly
)
data class GeoCoordinate3(
    val altitude: Double,
    val latitude: Double,
    val longitude: Double
) {

    @Json.Constructor
    constructor(latitude: Double, longitude: Double) : this(
        altitude = -1.0,
        latitude = latitude,
        longitude = longitude
    )
}

// input:  {"latitude":50.051961,"longitude":14.431521}
// output: {"altitude":-1.0,"latitude":50.051961,"longitude":14.431521}

Customize what properties are used for encoding (opt-in)

@Json(
    encoding = Json.Encoding.annotatedProperties  // only encode properties annotated explicitly
)
data class User(
    @Json.Property val id: String,
    @Json.Property val name: String,
    val passwordHash: String
)

// input:  {"id":1,"name":"Some User","passwordHash":"123456"}
// output: {"id":1,"name":"Some User"}

Customize what properties are used for encoding (opt-out)

@Json
data class User(
    val id: String,
    val name: String,
    @Json.Excluded val passwordHash: String
)

// input:  {"id":1,"name":"Some User","passwordHash":"123456"}
// output: {"id":1,"name":"Some User"}

Encode extension properties

@Json
data class Person(
    val firstName: String,
    val lastName: String
)

@Json.Property
val Person.name
    get() = "$firstName $lastName"

// input:  {"firstName":"Marc","lastName":"Knaup"}
// output: {"firstName":"Marc","lastName":"Knaup","name":"Marc Knaup"}

Customize JSON property names

Some prefer it that way ¯\_(ツ)_/¯.

@Json
data class Person(
    @Json.Property("first_name") val firstName: String,
    @Json.Property("last_name") val lastName: String
)

// input/input: {"first_name":"John","last_name":"Doe"}

Inline a single value

@Json(
    representation = Json.Representation.singleValue  // no need to wrap in a structured JSON object
)
class EmailAddress(val value: String)

// input:  "e@mail.com"
// output: "e@mail.com"

Prevent encoding completely

@Json(
    encoding = Json.Encoding.none,              // prevent encoding altogether
    representation = Json.Representation.singleValue  // no need to wrap in a structured JSON object
)
class Password(val secret: String)

// input:  "123456"
// output: not possible

Prevent decoding completely

@Json(
    decoding = Json.Decoding.none  // prevent decoding altogether
)
class Response<Result>(val result: result)

// input:  not possible
// output: {"result":…}

Add properties depending on the context

@Json(
    decoding = Json.Decoding.none,                // prevent decoding altogether
    encoding = Json.Encoding.annotatedProperties  // only encode properties annotated explicitly
)
data class User(
    @Json.Property val id: String,
    @Json.Property val name: String,
    val emailAddress: String
)

@Json.CustomProperties  // function will be called during encoding
fun JsonEncoder<MyContext>.writeCustomProperties(value: User) {
    if (context.authenticatedUserId == value.id)
        writeMapElement("emailAddress", value = value.emailAddress)
}

@Json.CodecProvider
interface MyCodecProvider : JsonCodecProvider<MyContext>

data class MyContext(
    val authenticatedUserId: String?
) : JsonCodingContext

fun main() {
    val serializer = JsonCodingSerializer
        .builder(MyContext(authenticatedUserId = "5678"))
        .encodingWith(JsonCodecProvider.generated(MyCodecProvider::class))
        .build()

    println(serializer.serializeValue(listOf(
        User(id = "1234", name = "Some Other User", emailAddress = "email@hidden.com"),
        User(id = "5678", name = "Authenticated User", emailAddress = "own@email.com")
    )))
}

// input:  not possible
// output: [{"id":"1234","name":"Some Other User"},{"id":"5678","name":"Authenticated User","emailAddress":"own@email.com"}]

Annotate types without having the source code

If a type is not part of your module you can still annotate it indirectly in order to automatically generate a codec for it. Note that this currently does not work correctly if the type has internal properties or an internal primary constructor.

@Json.CodecProvider(
    externalTypes = [
        Json.ExternalType(Triple::class, Json(
            codecVisibility = Json.CodecVisibility.publicRequired
        ))
    ]
)
interface MyCodecProvider : JsonCodecProvider<JsonCodingContext>

Examples

Have a look at the examples directory. If you've checked out this project locally then you can run them directly from within IntelliJ IDEA.

Manual Coding

Instead of using annotations to generate codecs, JSON can be written either directly using low-level APIs or by manually creating codecs to decode and encode classes from and to JSON.

Simple Parsing

… = JsonParser.default.parseValue("""{ "hello": "world", "test": 123 }""")

// returns a value like this:
mapOf(
    "hello" to "world",
    "test" to 123
)

You can also accept a null value by using parseValueOrNull instead.

Full example

Simple Serializing

JsonSerializer.default.serializeValue(mapOf(
    "hello" to "world",
    "test" to 123
))

// returns a string:
// {"hello":"world","test":123}

Full example

Using Reader and Writer

While the examples above parse and return JSON as String you can also use Reader and Writer:

val reader: Reader = …
… = JsonParser.default.parseValue(source = reader)

val writer: Writer = …
JsonSerializer.default.serializeValue(…, destination = writer)

Full example for Reader and for Writer

Parsing Lists and Maps

You can also parse lists and maps in a type-safe way directly. Should it not be possible to parse the input as the requested Kotlin type a JsonException is thrown. Note that this requires the -coding library variant.

val parser = JsonCodingParser.default

parser.parseValueOfType<List<*>>(…)              // returns List<*>
parser.parseValueOfType<List<String?>>(…)        // returns List<String?>
parser.parseValueOfType<Map<*, *>>(…)             // returns Map<*,*>
parser.parseValueOfType<Map<String, String?>>(…)  // returns Map<String,String?>

Note that you can also specify non-nullable String instead of nullable String?. But due to a limitation of Kotlin and the JVM the resulting list/map can always contain null keys and values. This can cause an unexpected NullPointerException at runtime if the source data contains nulls.

Full example for Lists and for Maps

Streaming Parser

JsonReader provides an extensive API for reading JSON values from a Reader.

val input = StringReader("""{ "data": [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] }""")
JsonReader.build(input).use { reader ->
    reader.readFromMapByElementValue { key ->
        println(key)

        readFromListByElement {
            println(readInt())
        }
    }
}

Full example using higher-order functions and using low-level functions

Streaming Writer

JsonWriter provides an extensive API for writing JSON values to a Writer.

val output = StringWriter()
JsonWriter.build(output).use { writer ->
    writer.writeIntoMap {
        writeMapElement("data") {
            writeIntoList {
                for (value in 0 .. 10) {
                    json.writeInt(value)
                }
            }
        }
    }
}

Full example using higher-order functions and using low-level functions

Type Encoder Codecs

While many basic Kotlin types like String, List, Map and Boolean are serialized automatically to their respective JSON counterparts you can easily add support for other types. Just write a codec for the type you'd like to serialize by implementing JsonEncoderCodec and pass an instance to the builder of either JsonCodingSerializer (high-level API) or JsonEncoder (streaming API).

Codecs in turn can write other encodable values and JsonEncoder will automatically look up the right codec and use it to serialize these values.

If your codec encounters an inappropriate value which it cannot encode then it will throw a JsonException in order to stop the serialization process.

Because JsonEncoderCodec is simply an interface you can use AbstractJsonEncoderCodec as base class for your codec which simplifies implementing that interface.

data class MyType(…)

object MyTypeCodec : AbstractJsonEncoderCodec<MyType, JsonCodingContext>() {

    override fun JsonEncoder<JsonCodingContext>.encode(value: MyType) {
        // write JSON for `value` directly using the encoder (the receiver)
    }
}

Full example

Type Decoder Codecs

While all JSON types are parsed automatically using appropriate Kotlin counterparts like String, List, Map and Boolean you can easily add support for other types. Just write a codec for the type you'd like to parse by implementing JsonDecoderCodec and pass an instance to the builder of either JsonCodingParser (high-level API) or JsonDecoder (streaming API).

Codecs in turn can read other decodable values and JsonDecoder will automatically look up the right codec and use it to parse these values.

If your codec encounters inappropriate JSON data which it cannot decode then it will throw a JsonException in order to stop the parsing process.

Because JsonDecoderCodec is simply an interface you can use AbstractJsonDecoderCodec as base class for your codec which simplifies implementing that interface.

data class MyType(…)

object MyTypeCodec : AbstractJsonDecoderCodec<MyType, JsonCodingContext>() {

    override fun JsonDecoder<JsonCodingContext>.decode(valueType: JsonCodingType<MyType>): MyType {
        // read JSON using and create an instance of `MyType` using decoder (the receiver)
    }
}

A JsonDecoderCodec can also decode generic types. The instance passed to JsonCodingType contains information about generic arguments expected by the call which caused this codec to be invoked. For List<Something> for example a single generic argument of type Something would be reported which allows for example the list codec to serialize the list value's directly as Something using the respective codec.

Full example

Type Codecs

If you want to be able to encode and decode the same type you can implement the interface JsonCodec which in turn extends JsonEncoderCodec and JsonDecoderCodec. That way you can reuse the same codec class for both, encoding and decoding.

Because JsonCodec is simply an interface you can use AbstractJsonCodec as base class for your codec which simplifies implementing that interface.

Full example

Coding and Streaming

You can use encoding and decoding codecs not just for high-level encoding and decoding using JsonCodingSerializer and JsonCodingParser but also for streaming-based encoding and decoding using JsonEncoder and JsonDecoder.

Full example

Thread Safety

All implementations of JsonParser, JsonSerializer, JsonCodecProvider as well as all codecs provided by this library are thread-safe and can be used from multiple threads without synchronization. It's strongly advised, though not required, that custom implementations are also thread-safe by default.

All other classes and interfaces are not thread-safe and must be used with appropriate synchronization in place. It's recommended however to simply use a separate instance per thread and not share these mutable instances at all.

Error Handling

Errors occurring during I/O operations in the underlying Reader or Writer cause an IOException.
Errors occurring due to unsupported or mismatching types, malformed JSON or misused API cause a subclass of JsonException being thrown.

Since in Kotlin every method can throw any kind of exception it's recommended to simply catch Exception when encoding or decoding JSON - unless handling errors explicitly is not needed in your use-case. This is especially important if you parse JSON data from an unsafe source like a public API.

Default JsonException subclasses

Ktor Serialization

You can use this library with ContentNegotiation of Ktor.

build.gradle.kts:

dependencies {
    implementation("io.fluidsonic.json:fluid-json-ktor-serialization:1.5.0")
}

Setting up your HttpClient:

HttpClient {
    install(ContentNegotiation) {
        fluidJson(
            parser = JsonCodingParser.builder().decodingWith(…).build(),
        serializer = JsonCodingSerializer.builder().encodingWith(…).build(),
        )
    }
}

Modules

Testing

This library is tested automatically using extensive unit tests. Some parser tests are imported directly from JSONTestSuite (kudos to Nicolas Seriot for that suite).

You can run the tests manually using Tests run configuration in IntelliJ IDEA or from the command line by using:

./gradlew check

Type Mapping

Basic Types

Encoding

The default implementations of JsonWriter and JsonSerializer encode Kotlin types as follows:

Decoding

The default implementations of JsonReader and JsonParser decode JSON types as follows:

Extended Types

The following classes of the can also be decoded and encoded out of the box.
For types in the java.time package the -coding-jdk8 library variant must be used.

Architecture

• JsonReader/JsonWriter are at the lowest level and read/write JSON as a stream of JsonTokens:
  • part of -basic library variant
  • character-level input/output
  • validation of read/written syntax
  • one instance per parsing/serializing (maintains state & holds reference to Reader/Writer)
• JsonParser/JsonSerializer are built on top of JsonReader/JsonWriter and read/write a complete JSON value at once.
  • part of -basic library variant
  • completely hides usage of underlying JsonReader/JsonWriter
  • encoding is performed using the actual type of values to be encoded
  • decoding is performed using the type expected by the caller of JsonParser's parse… methods and only available for basic types
  • instance can be reused and creates one JsonReader/JsonWriter per parsing/serialization invocation
  • ease of use is important
• JsonDecoder/JsonEncoder are built on top of JsonReader/JsonWriter and decode/encode arbitrary Kotlin types from/to a stream of JsonTokens:
  • part of -coding library variant
  • most read/write operations are forwarded to the underlying JsonReader/JsonWriter
  • some read/write operations are intercepted by JsonEncoder to encode compatible types using codecs
  • implementations provided by JsonDecoderCodecs and JsonEncoderCodecs
  • inspired by MongoDB's Codec and CodecRegistry
  • one instance per parsing/serialization invocation (holds reference to JsonReader/JsonWriter)
• JsonCodingParser/JsonCodingSerializer are built on top of JsonDecoder/JsonEncoder and read/write a complete JSON value at once.
  • part of -coding library variant
  • completely hides usage of underlying JsonDecoder/JsonEncoder
  • encoding is performed using the actual type of values to be encoded using a matching JsonEncoderCodec implementation
  • decoding is performed using the type expected by the caller of JsonParser's parse… methods and a matching JsonDecoderCodec implementation
  • instance can be reused and creates one JsonDecoder/JsonEncoder per parsing/serialization invocation
  • ease of use is important

Most public API is provided as interfaces in order to allow for plugging in custom behavior and to allow easy unit testing of code which produces or consumes JSON.

The default implementations of JsonDecoder/JsonEncoder use a set of pre-defined codecs in order to support decoding/encoding various basic Kotlin types like String, List, Map, Boolean and so on. Codecs for classes which are available only since Java 8 are provided by the -coding-jdk8 library variant.

Recursive vs. Non-Recursive

While codec-based decoding/encoding has to be implemented recursively in order to be efficient and easy to use it's sometimes not desirable to parse/serialize JSON recursively. For that reason the default container codecs like MapJsonCodec also provide a nonRecursive codec. Since they read/write a whole value at once using JsonReader's/JsonWriter's primitive read*/write* methods they will not use any other codecs and thus don't support encoding or decoding other non-basic types.

JsonCodingParser.nonRecursive and JsonCodingSerializer.nonRecursive both operate on these codecs and are thus a non-recursive parser/serializer.

Classes and Interfaces

Future Planning

This is on the backlog for later consideration, in no specific order:

• Add KDoc to all public API
• Add performance testing
• Add low-level support for BigDecimal / BigInteger
• Add pretty serialization

License

Apache 2.0