Adding standard Json AST - https://github.com/mdedetrich/scala-json-ast

[mdedetrich]

READ THIS FIRST

This discussion is about designing a library for a representation of JSON that can be shared between many different implementations of JSON parsing/pretty printing/transformation/.... As it's a SLIP (and not a SIP, so no language changes!), this shared API may eventually ship as a module that's included in the standard set of Scala libraries (e.g., it would be like a modern, minimalist scala-xml without the language support).

Everyone's welcome to join the discussion, provided your contribution is truly a contribution to the precise topic of this SLIP, as I tried to capture it above. This SLIP is not trying to solve the general problem of how to modularize the standard library, for example. I'm happy to refine my summary if needed. As always, there are plenty of other channels to discuss other (related) topics.

To ensure a productive discussion, please be kind to each other, stay as focussed as possible, and refrain from comments with very little technical content. As we are still refining this SLIP, it's too early to vote.

Thanks, @adriaanm

[lvicentesanchez]

A JSON module that evolves independently is just a library, I don't see the point of creating yet another library instead of choosing one of the existing ones as the preferred/endorsed. It seems a waste of "resources".

[larsrh]

I don't see any technical arguments supporting the other position, and thus it seems to me like a narrowed vision that can't get us anywhere.

@niseh, please re-read this thread. You don't appear to be interpreting the arguments charitably.

[kings13y]

+1 I'd like a small modular core, but either RI or preconfigured bundles for when I don't want the hastle of picking the best of breed, or to point newcomers at to be up and running quickly. Don't want another maven like grab bag of snowflakes, but I do think diverse and dedicated community could build better dedicated libs than could be reasonable expected of a core team.

[non]

I think an interesting question is whether library/framework authors who desire a unified JSON AST are already planning to move to json4s-ast from their own libraries, or only plan to do so if it is included in the standard library.

I helped to design these ASTs, I think they are reasonable, and I plan to add support for them in Jawn (along with all the other ASTs that are supported). However, I would feel more comfortable with this proposal if the library had more usage in the wild and some non-snapshot releases.

There's one point in the SLIP text that is worth pointing out:

The issue presents itself more in frameworks/libraries that have to deal with JSON. As a quick example, slick-pg, an extension ontop of slick for Postgres extensions, needs to provide implements of the JSON type for each of the commonly used JSON libraries, instead of just needing to support one.

My interpretation here is that if this SLIP were accepted, then libraries like slick-pg would only support json4-ast as opposed to other libraries that it currently does support. So it does seem like libraries like argonaut, rapture, rojoma, circe, jawn, etc. would be at a significant disadvantage when competing with json4s-ast if this proposal were accepted, whereas now everyone competes on a (relatively) level playing field.

This isn't necessarily entirely bad: libraries need to justify their existence versus some status quo. But it does mean that we should be very careful about standardizing on something that is not yet proven, since it will be difficult/impossible to change our minds later.

[odersky]

@larsrh Why do you say "prescribing"? Nothing prevents alternatives to flourish. There are at least two reasons I can think of to put it in the standard distrubution: (1) It means that other modules in the standard distribution can depend on it, without fear of being rendered inoperable by a change. (2) it gives one suggestion to newcomers and generally non-experts what libraries might work well together. If you do not believe in these reasons, why do you invest time in Typelevel?

[fommil]

@tixxit if somebody wants support for numbers that can't be represented as BigDecimal then perhaps JSON isn't the right protocol in the first place!! My main concern in financial data.

[gpampara]

I'm very much against this being in the stdlib. I simply don't understand the need.

If newcomers come to scala and ask about anything, what is so wrong with having a wiki/web page stating a list of recommended libraries? There could be a whole list of JSON related libraries that users could look at and choose themselves, thus allowing the community to benefit from the competition. Having something in the stdlib for JSON (or anything else) is prescribing what a user should be using.

Instead of pigeon-holing them into something that is "standard" and possibly underwhelming, support the community projects that already exist.

[fommil]

@odersky 3) corporate uptake. We don't all have access to maven central! But this is not true for modules.

[DarkDimius]

I'd like to have a actively maintained replacement for scala.util.parsing.json. It does not need to part of main artifact of standard library: it could be a separate maven artifact with own releases. But, for convenience, I would want it to be part of standard distribution, as JSON became very common as an interchange format and I'd like to be able to have a library that just works everywhere including standard REPL, both for me and for newcomers.

But as far as I understood the SIP, it's not about user-facing JSON library, but about common interchange format for multiple already existing JSON libraries. In that case, this is entirely a different situation, as the format has no use in standard distribution if there is no implementation that uses it.

[larsrh]

@odersky As has already been pointed out, third-party libraries have a hard time competing against the standard library. Compare the amount of supported alternative JSON libraries with the same for XML. Also, there are large parts of the ecosystem which do not depend or do not interact with typelevel libraries, and this is just fine, so I don't see the point.

[non]

We have an example where Scala went in the other direction: actors. Akka actors proved to be the most successful actor library and are now a de facto "standard" despite being a separate library. They were so successful that the competing standard library implementation was removed. Similarly, my understanding is that Akka's design for Futures and Promises was the basis for SIP-14, which gave us the modern implementation of futures.

Akka was is (and was) the status quo actor library for Scala. If there is no status quo JSON library for Scala then it's interesting to wonder why that is. Is the problem too easy? Too large a design space? Too many competing requirements? Something else?

[larsrh]

@non I'm confused – are Akka actors a module, just like parser combinators?

[nafg]

I think we should also make it easier for people to add libraries to their project. Maybe something like http://ls.implicit.ly/#finding? Maybe @softprops can comment

[non]

@larsrh I just meant that we used to have built-in actors, and now they are just provided by a separate de-factor standard (Akka) even though it is a separate library.

[mandubian]

@non I think Json multi libraries prove there are different approaches about it... the smallest common denominator is the AST and having contributed to a Json library and a validation one, I must say I wouldn't have been against having a common AST because Json is a very stupid format and I'm not interested in it...

[odersky]

@non Completely agree that if there is a standard library in a domain out there we should not duplicate it in stdlib. In fact the migration from scala.actors to Akka was done with the active support of the standard library maintainers. Phillip Haller, who originally did scala-actors, spent a lot of effort to make sure the migration to Akka was as smooth as possible. But Akka is huge compared to the modules we are talking about. If you think you need an actor library, you'll probably find out quickly that Akka is the one to turn to. But for JSON parsers no such reference implementation exists, so confusion ensues.

[channingwalton]

I am interested in the 'batteries included' notion. Is it to help newcomers, and also provides comfort that the batteries have a reasonable longevity and support.

They certainly are not going to solve any problems as far as projects finding multiple libs for json or anything else, it'll just add one more to the mix.

Martin, what is the problem being solved by 'batteries included" as you see it?

[non]

I do think that @nafg is on to something. Most languages I've used (besides Java of course) have a reasonable directory of libraries (CPAN, PyPI, rubygems, npm, hackage, opam, etc.) so users don't tend to have problems finding these things, and it's relatively easy to see which are the most active, complete, well-supported, etc.

[non]

I guess what I am trying to say is that if no one is agreeing on a standard JSON library in the wild, I think it's worth asking why that is before immediately proposing a new library as a standard. Otherwise we run the risk of duplicating the XKCD comic about competing standards [1].

Are there libraries out there who expose a JSON library dependency and are intending to move to json4s-ast on its own merits?

[1] https://xkcd.com/927/

[fommil]

@non its because it's an interesting intellectual exercise to create another one. Just like logging. And that hurts everybody. It is exactly the sort of thing that I'd like to see standardised at the AST level. But putting it in a module is useless because that's just another maven central artefact.

[jeantil]

I agree that the current Json implementation in the std lib has to go.

Not having a standardized AST makes it harder to switch serializer and parser implementation (to and from string), It also makes it harder to switch typeclass implementation (to and from class/caseclass) as each AST as its own idioms to manually create a tree. and of course they are close but not identical.

Not having a default parser/serializer in the std lib is not necessarily a big issue. I believe newcomers will simply google json with scala anyway and pick the first library returned.

What the std lib and the community should strive for is to make it easy to switch to a different implementation once they realize their initial choice is not the one best suited for their problem.

[He-Pin]

Why there are so many json lib in scala,a fork and a fork of fork？just for fun？or just for practice or just make a voice？how many star and how many contributor and how activity them are？how wildly they are used？are they depended by spark or something like those？I think we should be nice to the new comers and normal usage,I really agree with @non about his cats,why they try do add more document and test,that's the way to think about users,even them have not nake money from it.

90% use case are simple,why coulden't you try to share something and make it more nice to new comers,I really want scala to be more useable and nice for a greater and bigger community,for good thing for the whole world,not only some of the expects.

[odersky]

@non Completely happy to co-opt an existing library. But having some automatism is important. Let's face it: Our average user does not care at all what kind of JSON library is provided and what the tradeoffs between possible alternative libraries are. They just want to have some way to parse JSON if they need it. So any library that is not completely broken will fill this need. The importance is simply to have something always available.

If I would try out another language, did not find a standard JSON parser, and was pointed to an extensive body of discussions on Stackoverflow (or, worse, Twitter!) discussing the merits of the various contenders, I would be gone at this very instant.

[henridf]

If I would try out another language, did not find a standard JSON parser, and was pointed to an extensive body of discussions on Stackoverflow (or, worse, Twitter!) discussing the merits of the various contenders, I would be gone at this very instant.

Exactly the feelings of this dabbling Scala user (who won't be gone in an instant due to an ongoing project, but still remains baffled that this discussion should even happen...)

[adriaanm]

I agree with @odersky and @non, with the clarification that the standard library's goal is to standardize, not to exclude alternatives, and that the promoted/"standard" JSON API is a module with a default implementation, that should be easily swapped out for another module.

The abstractions should be standard, the choice of implementation is up to you. Especially in the JSON space, having a standard AST would be a huge boon to all the other libraries, as they could compete on performance etc, rather than on "what you happened to standardize on and now you can't move because migration is too expensive".

[odersky]

@larsh

As has already been pointed out, third-party libraries have a hard time competing against the standard library. Compare the amount of supported alternative JSON libraries with the same for XML.

I think that has to do with the relative complexity of XML and JSON. Stdlib had both. XML was problematic in some aspects but hard to do better in all dimensions. JSON was super-easy to improve upon. But as I wrote before, it's not a matter of "competition" but of maximal usefulness for the community as a whole. If you disagree with that opinion, I'd like to point to the "Lisp Curse" as a supporting argument:

http://www.winestockwebdesign.com/Essays/Lisp_Curse.html

[sjrd]

because Javascript objects are un-ordered and that's that.

Not anymore. Current engines all agree on keeping the order of insertion, and this de facto behavior has been spec'ed in ES 6, so we can actually rely on it.

[adriaanm]

/cc @jroper, who did some work on a standard json api for play recently, I believe

[He-Pin]

just take a look at golang,in the src/ encoding ，they have json,csv ...

[mdedetrich]

@lihaoyi

IMHO that makes it considerably less useful. I don't find myself passing around parsed but un-converted JSON ASTs around enough for it to be a hassle; most of my work involves going straight from String -> case class or case class -> string, but maybe other people have different usage patterns.

We currently have to interopt between 3 different JSON AST's in our codebase, mainly because we have a client side REST API that was written in Spray, and another that was written Scalatra (json4s). There is an older one that also uses Finatra (which we want to remove)

Any new library that uses JSON in some way, and wants to be used by the community, has to support like 5 JSON AST's, all of which differ in trivial ways

What's the exact treatment of numbers? Ints? Longs? Doubles/ BigNums? Strings? In Javascript people always parse doubles, but I've seen various JSON libraries parse different things.

According to the JSON spec (referenced in the SLIP), JSON numbers have unlimited precision, in other words they are unbounded real numbers (hence why both this SLIP, and Jawn, uses String as an underlying representation of JNumber, at least for the fast versions)

Scala.js isn't just about js.Arrays, we have js.Dictionarys too; would we encode the JSON dicts as those? That wouldn't work with the fast/safe distinction, because Javascript objects are un-ordered and that's that.

Are you talking about safe or fast? Fast has ordering, but thats only because it uses an Array (Array or js.Array). Array is the fastest, most memory efficient data structure for speed, particularly when you are building a JArray/JObject. The safe version has a proper implementation of what JArray/JObject are meant to represent

In Scala.js you'd typically want to wrap "raw" Javascript objects coming from JSON.parse. This is partly for performance, partly due to the fact that other JS libraries will be giving you these things. You hardly ever will want to parse them yourself into your own AST if you can help it.

Its more for libraries that use a JValue, and want to target both JVM and Javascript.

Nobody uses any of the listed libraries in Scala.js, and this proposal as discussed won't be of any use to anyone using Scala.js AFAICT. Cross compiling the AST would be of course trivial, but I do not see any use case as proposed (and I do think Scala.js needs a nice way of handling JSON!)

Originally one of the reasons was triviality, but the other reason is that I would (when Scala.js ecosystem grows enough) use some of the other libraries (that would use this AST) to do my client side JSON querying, mainly because its typesafe

This could very well live outside the standard lib and get adopted by various libraries. In particular, the consumers of this AST are few enough that it's actually feasible to talk to all of them and get them to try and migrate. This is in contrast to e.g. collections where the consumers are every-scala-programmer-ever.

If this is an official module rather than in stdlib, I don't really have many issues with that

@fommil

Why not just pick one that already works well? spray-json in really good. Importantly it uses arbitrary precision for numbers and reminds us all that JSON maps do not preserve ordering. If you really must create a new AST (replacing this one tiny file of awesome) then please do not forget these points.

The fast version here is almost the same as spray-json. The safe version is meant to represent a valid JSON structure that has good all round performance for querying values (i.e. eC lookup time in JObject)

@larsrh

If it's being put into a module, users would still need to declare it as a dependency in their build definition (just like for XML and parser combinators). What's the point prescribing one library when there are others out there which can be imported in exactly the same way?

It has an "official" label on it, but also the AST is designed to basically never change, so it has the feature of stability

@OlegIlyenko Precisely what this is trying to solve

@non

My interpretation here is that if this SLIP were accepted, then libraries like slick-pg would only support json4-ast as opposed to other libraries that it currently does support. So it does seem like libraries like argonaut, rapture, rojoma, circe, jawn, etc. would be at a significant disadvantage when competing with json4s-ast if this proposal were accepted, whereas now everyone competes on a (relatively) level playing field.

This is one of the main justifications, however thats why json4s-ast (current reference library) was made with the huge amount of feedback from all (or at least most of) those library creators. This is just meant to be an AST, and the AST for JSON is downright trivial. If you look at the implementations for those AST's, they basically are almost practically the same (if you split them into fast or safe versions).

Its like having 3 different versions of String

@adriaanm

The abstractions should be standard, the choice of implementation is up to you. Especially in the JSON space, having a standard AST would be a huge boon to all the other libraries, as they could compete on performance etc, rather than on "what you happened to standardize on and now you can't move because migration is too expensive".

Precisely

@sjrd

Not anymore. Current engines all agree on keeping the order of insertion, and this de facto behavior has been spec'ed in ES 6, so we can actually rely on it.

Yes, but the JSON standard as is about unordered objects

@odersky Regarding using a parser as part of the module. I am not in disagreement of that, however it was outside of the scope for this SLIP, and there is a huge (and amount of good competition) for parsers. If scala wants to pick one of the current existing parsers so users have a solution, I am not against that.

[dwijnand]

Trying to be "batteries included" but then actually be a module means:

• you still need to actually declare it as a dependency, like an other library
• but you would get it in the distribution REPL (scala on the command-line)

:-1: I don't think this should be a part of the Scala distribution, I think it should be part of the larger ecosystem.

I hope json-ast picks up interest again and reaches success as I think its goals are important.

[cb372]

I don't understand the benefit of adding a json library to the stdlib.

• It's more bloat for the core Scala developers to maintain, even if it's in a separate jar
• Anything tied to the Scala release cycle will inevitably evolve/fix bugs/etc more slowly than an independent library
• What's the point of a batteries included solution that quickly becomes outdated and superseded by one or more libraries?

As for the argument for adding just an AST to the stdlib, I don't really think this is necessary either. I don't think I've ever had to interop between two different json libs in the same Scala project, so who cares whether different libs share the same AST?

Also I don't feel like the current Scala ecosystem is as bad as people make out. Sure, there are a few different libs available, but they serve different purposes: json4s is for people who just want to get shit done, play-json is more type-safe, argonaut is for people who love FP, etc.

[mdedetrich]

It's more bloat for the core Scala developers to maintain, even if it's in a separate jar

If you have a look at the implementation, and the spec for JSON, its been virtually unchanged for years. In fact, I don't see the current source changing (apart from version bumps) for years

What's the point of a batteries included solution that quickly becomes outdated and superseded by one or more libraries?

There is a lowest common denominator for interpretability. Thats like arguing "why do we have a common String". Have a look at how much pain multiple "string" types has caused in the Haskell community (text vs List[Char] vs others)

As for the argument for adding just an AST to the stdlib, I don't really think this is necessary either. I don't think I've ever had to interop between two different json libs in the same Scala project, so who cares whether different libs share the same AST?

I do this all the time. Its not even so much about interoperability, but the fact it makes it that much harder for library creators (that have their library using JSON in some way) need to target 5 different AST's that differ slightly.

Let me point out to everyone, that what is now 5 AST's, used be 2/3. That number keeps on increasing, and we don't need to get to a ridiculous situation in 5 years time where "if you want to do something with JSON, you need to target/maintain 8 JSON AST implementations"

Also I don't feel like the current Scala ecosystem is as bad as people make out. Sure, there are a few different libs available, but they serve different purposes: json4s is for people who just want to get shit done, play-json is more type-safe, argonaut is for people who love FP, etc.

Not sure about argonaut, but there is commonality in all of these AST's. Where all of these libraries differ greatly is their parsing/querying, which is not something I want to standardise (but there is good argument for a standard parser)

[kiritsuku]

I don't like that we have this discussion at all.

The problem of being beginner friendly can't be solved with a stdlib because it is a tooling problem. People ask questions like "which library should I use?", "where can I find documentation to this problem?", "which IDE should I use?", "which build tools should I use?", "how is the community organized?", "how do I make my library visible?", "how can I find out if people like my library?", "how many people use my library?" and so on. Solving any of these questions would contribute to making a language more beginner friendly. Adding more content to a stdlib solves the first question for a short time (probably months, maybe even years) but ignores all of the other questions completely. Not an efficient way to solve a problem.

With better tooling support all of the questions above could easily be addressed but Scala - as all other languages before it - has chosen to ignore that option completely. Taking the short route of only solving the immediate problem rarely is the path that should have been taken in the first place. Most people believe that creating a unified interface, an ecosystem for libraries, documentation, compilers, IDEs and other tools is too difficult to solve. But it is not - with the possibilities we have today we no longer have to repeat the mistakes of most other programming languages for forever by always going the short route.

As long as we have discussions about such "trivia" like adding a JSON AST to a stdlib, we miss the time we need to discuss how to solve the ecosystem problem - which is (for me) by far the largest unsolved problem Scala has (and not just Scala but most other programming languages too). Scala spread because it solved problems at the roots instead of fighting the whole day against symptoms. I would appreciate it if we could bring back this spirit.

[fommil]

@sjrd you're thinking purely in terms of browsers. In my world both ends of the pipe are servers and can be ancient. We cannot assume or rely on ordering, of course being ordered by happy accident is a different thing.

[bwmcadams]

-1

... with right to change my opinion presented different arguments/evidence reserved. I was hesitant to wade in on this, and my voice added to this may not be as useful as I think... but here goes. And this is going to be a long one, sorry.

From a technical standpoint I have a few things here:

1. The merits of the argument entirely aside, I don't feel that the proposed SLIP makes any valid, reasoned arguments for a standard AST or JSON Library in Scala. It points out that there are many libraries, and someone once tried to make a standard one (which has been recently resurrected, but see my further point on that below). It presents drawbacks, which is of course something I also see as a requirement. But I see it failing entirely to make any reasoned case for its existence or implementation. This is a flawed proposal on that omission alone, and a strong reasoning for my -1. It seems that we're voting on whether to allow this SLIP be proposed, rather than “are we, as a community, voting to approve it?”. If I'm wrong there, I'd appreciate correction. But my impression is if we feel the flaws like what I've just raised are fixed we are voting whether to merge a proposal–not approve “do we do this“–and argue / vote later if the SLIP should be actually approved.
2. Having thought it through thoroughly, I would also support voting against the approval of the proposed idea of a centralized JSON Library and/or AST. I don't see merit in it, or any advantage being provided to Scala or its users. It seems to come from a place of “there are too many JSON libraries and it is hard to choose”. But this proposal would do nothing to fix that and it is not a problem that needs to, or can be, fixed. Do we really feel Scala can force consensus or compliance of JSON library usage/implementation standardization within the community with meaningful impact or justification?
   • Again, I question what problem is really solved by introducing this? Sure, we give new Scala users a “default” JSON library to choose from. But it doesn't obviate the advantages of other libraries which may be more appropriate to their use case. In contrast, it guides them unnecessarily to use something that is deliberately intended to be more “broadly generalized” in its approach. Do we feel that's a good or productive thing? I am not using this point to stand on a hard “no” – I'm using it to point out that this question needs to be part of our discourse.
   • I find myself – having used many of the different mentioned “competing” JSON libraries in various contexts – swayed by the argument that there's merit in having multiple libraries, each of which can focus on a use case optimized AST instead of a “generalized” one that seems to provide no advantage, and disadvantages which are quite literally outlined in the SLIP. On a counterpoint, it is fairly noted in the SLIP that there has been some recent collaboration to have many of these libraries use a standalone community driven standard AST. Which I'll address thusly...
     • A question we also need to ask: Will the libraries mentioned as part of the fragmentation actually adopt or value an AST that is baked into Scala? If we (well, the JSON lib authors) end up agreeing that “hey, a common AST isn't a bad idea”, that doesn't translate automatically/implicitly/transitively to mean that one should be baked into Scala. Scala moves more slowly than an open source library can. If these libraries standardized on a central AST that means “improvements” are locked to waiting on the next release of Scala – as presumably you're not likely to see a major AST change in a minor release, that could be a long wait.
   • I am hesitant to say it, but “bloat” concerns me. This circles back to the what value do we derive from this?, and are we being consistent in decisions to add/remove things from StdLib? Are we thinking about the justification for adding more to the standard library? XML support is being moved out of the StdLib, IIRC – you'll need (or already need?) a separate dependency for it. Actors did the same thing, though much of that was driven by Akka's existence (again an IIRC). But my point, I hope, stands or at the very least argues for itself from here. I'm not fully in either of the camps arguing for a smaller standard library or a more comprehensive one. My opinion for "smaller standard library" with regards to bloat applies to this case, specifically, based on the other questions I've already posed.

That ends my technical arguments. I'd be remiss without a final statement, however...

A Final Thought on the Poor Behavior Exhibited Throughout This Thread

Or, How I Learned To Stop Arguing And Learned To Prefer Discourse

I'm disappointed in all of us (I include myself as I'm not without blame for doing same in past) on how quickly this thread devolved the way it did. Worsened by the nauseousness of the fact that it devolved at all.

Are we are a community, or a collection of competing tribes? I like to think we're a community - but the kind that doesn't resort to Scarlet Letters, Tarring & Feathering, Stockades, or Good Ol' 'Murican Smear Campaigns. Personally, I'm somewhat mortified that there was assumptions made that a mere -1, or a disagreement without qualification or even fairly assumable faction-driven opinion lumps someone into a targetable group such as "scalaz people".

I, for example, disagree with this proposal. It happens to be that yes, I'm a user of scalaz. I've given conference talks on it. I've helped write a scalaz test library which led me to be involved in Typelevel. I even got involved in cats, contributing some code to it already (but IIRC someone else helped me get it right after my first draft was somewhat wonky, so I feel weird taking credit). But I've also worked for Typesafe. I've been a longtime contributor to–and evangelist for–Akka. I am currently employed by an organization whose primary focus is on services related to Typesafe's product offerings. I'm a strong supporter of all of the above projects and organizations... because there are different tools for different jobs. I admit that there are strong voices here who are capable of swaying my opinion at times; I also try to stop and think about the merit of them before deciding for myself. But hey, now you all have a list of possible tribes to lump me into based on how I vote in the future?

I have spent a significant amount of my life for the past 6+ years advocating and evangelizing for Scala. When large organizations or representatives thereof ask me to justify moving from say, Java to Scala, beyond just the technical advantages... “Strong, Healthy Community” is something I hold up as a badge of pride.

I would feel uncomfortable ever making that argument again, if this is genuinely how we all think we should behave when we disagree. We are introducing unnecessary partisanship, when we should treat each other as equals who have varying opinions that may, at times, clash. How do you think this kind of behavior looks from the outside to someone considering if making the investment (be it time or money or anything else that qualifies)?

Debate & discourse is healthy: but only if it's a proper debate. Without digressing unnecessarily into a lesson in American history, we should look to something like The Lincoln-Douglas Debates in our discourse and conduct. Roughly, what I remember being taught many years ago in school WRT to good discourse...

1. An initial statement should never be deliberately antagonistic, but reasoned with logic & facts. Passion is good, but never ever allow your passion to pass into fervor, lest you be lost forever.
2. The response you give should follow the same rules as above. With the even more important need to avoid your counterpoint being based in any way on an attack on your opponent. They aren't your adversary: they are someone with a valid opinion that happens to clash with yours.

There is a staggering amount of unbelievable minds in this thread, many of whom are the best educated and knowledgeable people I have ever met in my life. Please act like it.

[mdedetrich]

@bwmcadams

Having thought it through thoroughly, I would also support voting against the approval of the proposed idea of a centralized JSON Library and/or AST. I don't see merit in it, or any advantage being provided to Scala or its users. It seems to come from a place of “there are too many JSON libraries and it is hard to choose”. But this proposal would do nothing to fix that and it is not a problem that needs to, or can be, fixed. Do we really feel Scala can force consensus or compliance of JSON library usage/implementation standardization within the community with meaningful impact or justification?

By extension, you can say there is no merit in having a standard String?. Obviously the inclusion of this into stdlib (or a supported module), doesn't automatically solve the problem, it does so indirectly. There is already a lot of support for this library (scalatra/lift/play/spray for now), and if something is including and is official, then it means that over time libraries will mainly target that implementation

I'm disappointed in all of us (I include myself as I'm not without blame for doing same in past) on how quickly this thread devolved the way it did. Worsened by the nauseousness of the fact that it devolved at all.

Agreed. I expected to have a nice discourse about the merits of this SLIP, but instead its turned into some political mudball where people provided non helpful feedback or have standoff'ish comments

[OlivierBlanvillain]

@non

Are there libraries out there who expose a JSON library dependency and are intending to move to json4s-ast?

https://github.com/jto/validation

But the story is not as simple as "move to json4s-ast". Play users still want something that works out the box with the AST returned by the Play parser, Scala.js users want something compatible with the js.Any/js.Dictionary returned by the native js parser and so on.

In this context (wanting to be out of the box Play/Spray/Scala.js/... compatible) I think having a standardised json API could be very useful to circumvent the current "let's be compatible with 7 ASTs" situation that is found in many json related libraries.

[non]

@OlivierBlanvillain Thanks, that's really good to know. I agree that supporting N different ASTs is a bit annoying (I wrote a bunch of code to solve that in Jawn using type classes).

What do you think your migration path will be? Presumably you'll add support for json4s-ast now and then deprecate the others later?

[puffnfresh]

:-1:

My professional life is hard enough as it is because of the current stdlib having many poor performing and broken things. I want less stdlib, not more.

[sjrd]

@fommil

you're thinking purely in terms of browsers. In my world both ends of the pipe are servers and can be ancient. We cannot assume or rely on ordering, of course being ordered by happy accident is a different thing.

I think you misunderstood my comment. I am not saying we should standardize on JSON ASTs that guarantee to preserve ordering. I was only replying to @lihaoyi's concern that, on the JS platform, we should use js.Dictionary to represent objects, but that using those would not be compatible with the fast "requirement" (of the proposal as it currently stands) to preserve order. I was just saying that yes, js.Dictionary does preserve order, and that therefore, should we choose to standardize on something that needs to preserve order, we can use js.Dictionary anyway.

[He-Pin]

@sschaef yes,but slip,dotty,and so on is just a part of the ecosysterm too.and in fact scala ide and idea's plugin both have been improved a lot in the last year.more scala user,more bigger community,then more commercial investment from company.

[fommil]

@sjrd ah sorry, I missed the reference to the JS impl. Yes that sounds sensible.

[mandubian]

@bwmcadams I feel like this discussion isn't so insane except the few usual ones... globally it tells something interesting about the topic...

[shawjef3]

A standard JSON AST that is appropriate for most uses and is official in the same sense as the XML library would help with Scala adoption. If someone wants to do something that uses JSON, they shouldn't have to spend a day reading about and weighing different implementations; that's time wasted that could be used getting something useful done. An advanced user can choose to use a non-official library for whatever reasons.

[mdedetrich]

@puffnfresh

:-1: My professional life is hard enough as it is because of the current stdlib having many poor performing and broken things. I want less stdlib, not more.

If you have issues with the performance/brokenness of this SLIP, then please show them

[mdedetrich]

@non

What do you think your migration path will be? Presumably you'll add support for json4s-ast now and then deprecate the others later?

Thats what I had roughly in mind. More importantly we need the broad community to support json4s-ast (hence why when creating it, I was trying to get everyone onboard in regards to whether or not they would use this AST, and what they need from it)

[adriaanm]

Please be kind and remain on topic XOR refrain from commenting.

If too many cannot abide this basic form of politeness, I will have no choice but to lock this thread, with my apologies to those contributing constructively.

EDIT: We have deleted comments with zero technical contribution and 100% unkindness, and will continue to do so.

[bwmcadams]

@mdedetrich I'm not sure how to respond to equating “primitive types” - that is to say core types that any language needs support for like int, double, String (yes, I realize some languages lack Strings but could we agree String is better than say, char*?), etc to JSON libraries. I believe we need a robust String implementation provided to users as "Standard Library". I don't believe we need larger conceptuals such as JSON, or XML.

I am not a hardcore (actually, not sure I'm even softcore) proponent of slimming the Scala Library down. I am, however, a skeptic when it comes to whether or not we should add more.

[mdedetrich]

@bwmcadams

@mdedetrich I'm not sure how to respond to equating “primitive types” - that is to say core types that any language needs support for like int, double, String (yes, I realize some languages lack Strings but could we agree String is better than say, char*?), etc to JSON libraries. I believe we need a robust String implementation provided to users as "Standard Library". I don't believe we need larger conceptuals such as JSON, or XML.

I suppose you an look at it this way. Any language in created in the before early 2000's would have had very few basic primitive types (and you are correct in this regard), however things have changed since then. As said earlier, any new language (either if its a completely new language, or an extension to a current language), that has some demographic of users in the "web backend space" provide a JSON type and a JSON parser. JSON is the most common interchange format between servers, and possibly even for browser or client.

The results kind of speak to themselves. When I first had a look at Scala in 2007, we had one json type, and that was lift JSON (and thats because it was the first popular framework in Scala). In those 8 years, we now have added another 6 JSON AST's.

I think that speaks for itself, in that there does need to be a primitive AST for JSON