Closed neoeinstein closed 5 years ago
An alpha with these changes has been posted to NuGet as "7.0.0-alpha-170410"
I realize that this is an alpha release, but was wondering if you could help me w/ an issue when I attempt to use this:
Binding session to 'c:\Users\dannygeorge\code\fsharp\extendedevents\ExtendedEventsLib\../build\FParsec.dll'...
System.TypeInitializationException: The type initializer for '<StartupCode$Chiron>.$Chiron' threw an exception. ---> System.IO.FileLoadException: Could not loa
d file or assembly 'FParsec, Version=1.0.0.0, Culture=neutral, PublicKeyToken=40ccfc0a09edbb5d' or one of its dependencies. A strongly-named assembly is requir
ed. (Exception from HRESULT: 0x80131044) ---> System.IO.FileLoadException: A strongly-named assembly is required. (Exception from HRESULT: 0x80131044)
--- End of inner exception stack trace ---
at <StartupCode$Chiron>.$Chiron..cctor()
--- End of inner exception stack trace ---
at Chiron.JsonObjectModule.get_empty()
at FSI_0002.encoder(XEvent x) in c:\Users\dannygeorge\code\fsharp\extendedevents\ExtendedEventsLib\Script2.fsx:line 98
at <StartupCode$FSI_0002>.$FSI_0002.main@() in c:\Users\dannygeorge\code\fsharp\extendedevents\ExtendedEventsLib\Script2.fsx:line 112
The versions that paket is using are: Chiron 7.0.0-alpha-170410
and FParsec 1.0.3-alpha-170404
@neoeinstein I'm reading through the tests now trying to grasp how to actually use the thing now. Any chance you've got something written about it?
This looks quite promising, but is there any plan to move forward ? Or is it currently blocked by some issue ?
Main thing is to get FParsec onto a stable release. Once that is done, I will push the new Chiron as a stable release. If you have any suggestions for changes or improvements to the API, please let me know.
I'm working on a converting the freya example to dotnet core as a learning exercise. So I am using the alpha version of Chiron and am using Aether (v8.2). It seems that the operators API clash: Chiron and Aether.
The example code has this:
let todo_ id =
todos_
>-> Map.value_ id
which is fine for pre-netCore. however, for netCore workaround:
let todo_ id =
todos_
>--> Map.value_ id
which the operator >-->
is marked as obsolete. I'm not sure how common it is to use both Aether and Chiron because I'm just starting, but if it is quite common, this clash might be a little annoying and if Aether drops that operator, projects would break upon upgrade. I just thought I'd point it out since it was something I noticed.
Hi all,
I currently use Chiron, dotnet core alpha version, in one of our projects. I encounter what it seems to me to be a bug. I have the following JSON with two optional fields like this:
{"id":"0e3563cb-FFFB-488f-8b99-ea06cd17ef98", "companyName": "AGILITIC SAS", "civility":"mr", "lastname":"JALOWOI", "firstname":"Sylvain", "phone":"02334345654", "mobile_phone":"", "email":"sja@gm.com", "notes":"", "billing_address":{ "address":"6 rue des peupliers", "city":"La combe-sur-mer", "zipcode":"63150"}, "intervention_address": { "address":"6 rue des peupliers", "city":"La combe-sur-mer", "zipcode":"63150"}}
Actually, one field is missing, "owner", the second optional field was "CompanyName".
On the backend server side (Suave), I have the following code to interpret the JSON:
` type CommonClientInformationForm =
{
id : ClientId
civility : string
lastname : string
firstname : string
phone : string
mobilePhone : string
email : string
notes : string
billingAddr : Address
interventionAddr : Address
}
type CreateClientForm =
| RegularClientForm of bool * CommonClientInformationForm
| CompanyClientForm of string * CommonClientInformationForm
| InvalidClientForm
static member FromJson (_ : CreateClientForm) = jsonDecoder {
let! cid = DI.required "id"
let! e = DI.required "email"
let! f = DI.required "firstname"
let! l = DI.required "lastname"
let! c = DI.required "civility"
let! pn = DI.required "phone"
let! mpn = DI.required "mobile_phone"
let! n = DI.required "notes"
let! baddr = DI.required "billing_address"
let! iaddr = DI.required "intervention_address"
let! cname = DI.optional "companyName"
let! owner = DI.optional "owner"
let commonInfo = { id = cid
civility = c
lastname = l
firstname = f
phone = pn
mobilePhone = mpn
email = e
notes = n
billingAddr = baddr
interventionAddr = iaddr
}
let res = match cname with
| None ->
match owner with
| None -> InvalidClientForm
| Some o -> RegularClientForm(o, commonInfo)
| Some compName -> CompanyClientForm(compName, commonInfo)
return res
}`
At runtime, if there is one missing of the two optional fields, the program stops with a StackOverflow exception. If I remove the code dealing with the two optional fields, the code runs correctly. The behavior I expect when a field is missing in a JSON is that the D.optional "key" function returns None, not an exception. Am I wrong ?
Thanks for the code, the new version is very nice to use
StackOverflow is a very particular exception. Do you have a stack trace? Generally a stack overflow indicates some recursion or deep call stack, so knowing what that call stack was would be helpful in understanding the issue you are observing.
Better yet, if you can create a PR against the chiron-7
branch with an added test that fails, I can work to make that test pass.
Hi Marcus,
thanks for you answer. I cannot reproduce the StackTrace, i think it is a problem into my code. Sorry for the disturbing. However i encounter a problem with using of Optional.
Actually, i have a Json like this :
{ "action": { "interventionCreated": { "clientId": "340078fc-e585-402d-b2e1-96df3b2a2f03", "interventionStatus": { "value": "toschedule" }, "userId": "0e45fedd-2aaf-41c4-b7ea-3554ecebf145", "interventionDeadline": null, "interventionType": { "value": "maintenance" }, "interventionId": "ba56b826-2e7e-49f6-aea4-24019bd5f8f0", "interventionDescr": " " } }, "version": "1.0" }
A field called interventionDeadline has a Null value. So i suppose i need to use DI.optional "interventionDeadline" for getting the field and i will get a None value.
Instead of None value, i get the error message:
Unhandled Exception: System.Exception: Impossible de deserialiser un événement: Found 9 errors: (Choice #1): Failed to find expected property: email (Choice #2): Failed to find expected property: login (Choice #3): Failed to find expected property: userAdded (Choice #4): Failed to find expected property: regularClient (Choice #5): Failed to find expected property: companyClient (Choice #6).interventionCreated.interventionDeadline: Expected to find a string, but instead found null (Choice #7): Failed to find expected property: appointmentAdded (Choice #8): Failed to find expected property: appointmentCancelled (Choice #9): Failed to find expected property: appointmentCompleted du stream: account:041c49f0-eb59-4982-a602-0046d72a6564 - Numéro d'event: 4
How can i read a field like "interventionDeadline" with Chiron 7 that is supposed to be a String field with eventually a Null value ? Thanks for your time, Sylvain
PS: I am in vacation, it is not easy for me to do a PullRequest.
For that line, you can use D.required (D.optionWith D.string) "interventionDeadline"
. That will require that "interventionDeadline" exists and is either null
or a string
. If you want "interventionDeadline" to be optional and also handle null
, then use D.optional (D.optionWith D.string) "interventionDeadline"
.
Part of the API change was to ensure that all three of the optional cases were covered (accept missing, accept null, accept null or missing).
Thanks for your answer, it resolve my problems and it is more clear for me. Best regards Sylvain
Hello,
I have the same problem, but when I use the syntax: D.optional (D.option D.string) "value"
I get the following error:
error FS0001: This expression was expected to have type "Decoder<Json,'a>"
but here has type "JsonResult
I have the same error using the other proposed syntax: D.required (D.option D.string) "value"
In this context, D = Chiron.Serialization.Json.Decode
Do you confirm that this is the correct syntax?
Best regards. Julien.
Decoding objects can be done with a new computation expression: jsonReader or by using the provided operators.
Is there any example about decoding with operators?
@lanarchyste Having a fuller example, including the type you are attempting to deserialize would help me to see if there is anything off in the syntax. I went looking for a test that uses D.optional (D.option _)
and didn't see one, so I will get one added to ensure it is working properly.
@Risord, Take a look at the tests in the PR here: https://github.com/neoeinstein/chiron/blob/chiron-7/tests/Chiron.Tests/Chiron.Tests.fs#L38.
@lanarchyste Well, after just having responded to Risord, that same section of code gives the answer. It's not D.optional (D.option _)
, it's D.optional (D.optionWith _)
. I'll correct my earlier comments.
Hello, I am attempting to update and I am having more success than my last attempt a few months ago. I have the following type
type CommentDatabaseModel =
{
commentId : string
userId : string
message : string
}
and CommentMultiTreeDatabaseModel =
CommentDatabaseModelNode of CommentDatabaseModel * CommentMultiTreeDatabaseModel list
and the following decoders
let commentDatabaseModelDecode =
let inner =
(fun
commentId
userId
message ->
{ commentId = commentId
userId = userId
message = message
})
<!> required "commentId"
<*> required "userId"
<*> required "message"
Json.Decode.jsonObject >=> inner
let rec commentMultiTreeDatabaseModelDecode =
let inner =
(fun
commentDatabaseModel
forest ->
CommentDatabaseModelNode (commentDatabaseModel, forest)
)
<!> D.required commentDatabaseModelDecode "commentDatabaseModel"
<*> D.required (D.listWith (D.lazily commentMultiTreeDatabaseModelDecode)) "forest"
Json.Decode.jsonObject >=> inner
I am attempting to copy Elm decoder
commentTreeDecoder : Decoder (Tree CommentModel)
commentTreeDecoder =
decode MultiwayTree.Tree
|> required "commentModel" commentModelDecoder
|> required "forest" (list (lazy (\_ -> commentTreeDecoder)))
What am I missing? Any help or advice is much appreciated. Thank you
UPDATE
Got it! Use D.delay
I found a bug. Chiron seems to encode decimals with ',' instead of '.'
Chiron.Inference.Json.encode 1.5
|> Chiron.Formatting.Json.format // returns "1,5" expected: "1.5"
Windows 10 with culture fi-FI
I think I cannot create pull request for this because development doesn't exists in any traditional github branch(?). Anyway you should be able to reproduce issue by setting:
CultureInfo.CurrentCulture <- new CultureInfo("fi-FI")
I am currently watching newest commit of this pull request (0b1a12c) and I suggest that all primitive .ToString([potential-format])
calls should be changed to:
.ToString([potential-format], CultureInfo.InvariantCulture)
(Chiron.fs: 996 - 1046)
Culture effects to parsing too so CultureInfo.InvariantCulture
must be passed to .Parse
functions too.
(Chiron.fs: 1527 - 1572)
For example this code fails with fi-FI culture:
let number : float =
Chiron.Inference.Json.deserialize "1.5"
|> function
| JPass n -> n
| _ -> failwith "No can do"
For now workaround is to change current culture at program start:
CultureInfo.CurrentCulture <- CultureInfo.InvariantCulture
@Risord Thanks for the report. I'll make sure to explicitly set the invariant culture in each of these spots. The PR is coming from my own fork of the repo at the moment, so that's likely what is making it difficult to see where to make a PR.
Ok. A new Chiron 7 alpha has been released. The underlying library (FParsec) is now on a stable release, so I am about ready to release Chiron 7 as a stable release. Let me know if there are any additional things that should be tweaked in the API. Thanks for all who have been trialing the new API.
@neoeinstein https://www.nuget.org/packages/Chiron/ seems no new version has been released ?
Hello,
I'm working with the library to deserialize json but I have an issue.
Considering the following types:
type Info =
{ title : string
description : string option
termsOfService : string option
contact : Contact option
license : License option
version : string }
and Contact =
{ name : string option
url : string option
email : string option }
and License =
{ name : string
url : string option }
If I create FromJson methods using the builder jsonDecoder
it compile and the type of the methods is 'a -> Decoder<Json, 'a>
.
But if I use the operators to define FromJson methods it doesn't compile and I get the following error:
'Type mismatch. Expecting a
'Decoder<Json,Contact>'
but given a
'Decoder<JsonObject,Contact>'
The type 'Json' does not match the type 'JsonObject''
What's the difference between Json and JsonObject ? I don't get why when using operators signature of FromJson is 'a -> Decoder<JsonObject, 'a>
and when I use the builder it's 'a -> Decoder<Json, 'a>
.
Did I miss something?
Thanks
open Chiron
open Chiron.Inference
type Contact with
static member FromJson (_ : Contact) =
jsonDecoder {
let! name = Json.Decode.optional "name"
let! url = Json.Decode.optional "url"
let! email = Json.Decode.optional "email"
return { name = name
url = url
email = email }
}
type License with
static member FromJson (_ : License) =
jsonDecoder {
let! name = Json.Decode.required "name"
let! url = Json.Decode.optional "url"
return { name = name
url = url }
}
type Info with
static member FromJson (_ : Info) =
jsonDecoder {
let! title = Json.Decode.required "title"
let! desc = Json.Decode.optional "description"
let! tos = Json.Decode.optional "termsOfService"
let! contact = Json.Decode.optional "contact"
let! license = Json.Decode.optional "license"
let! version = Json.Decode.required "version"
return { title = title
description = desc
termsOfService = tos
contact = contact
license = license
version = version }
}
type Contact with
static member FromJson (_ : Contact) =
fun name url email ->
{ name = name
url = url
email = email }
<!> Json.Decode.optional "name"
<*> Json.Decode.optional "url"
<*> Json.Decode.optional "email"
type License with
static member FromJson (_ : License) =
fun name url ->
{ name = name
url = url }
<!> Json.Decode.required "name"
<*> Json.Decode.optional "url"
type Info with
static member FromJson (_ : Info) =
fun title desc tos contact license version ->
{ title = title
description = desc
termsOfService = tos
contact = contact
license = license
version = version }
<!> Json.Decode.required "title"
<*> Json.Decode.optional "description"
<*> Json.Decode.optional "termsOfService"
<*> Json.Decode.optional "contact"
<*> Json.Decode.optional "license"
<*> Json.Decode.required "version"
Apologies to all. I had published the new version to MyGet. I've corrected that error and have now published a beta
version—making sure that I push to NuGet this time. Please review and let me know if there are any final changes that we should make to the API.
Ok, I've pushed up a new version which contains one more feature that I really wanted. Dealing with recursive structures was a pain with the new encoder/decoders. You would need to write your own delay functions, etc.
Now, you can use a method very similar to FParsec
, wherein you request a ref
from Chiron. You get back a ref
and a usable (en/de)coder. Then, you can use a static initializer (a module do
block) to lazily initialize the ref
.
I've added tests for the new functionality. I want to modify the Schema Def example to use the new mechanism.
Still to consider: would aggressive inlining hints benefit performance here?
There will be some minor clean up, dead code comment deletion, and I will try to get some source documentation added. After that, this new version should be ready to go stable.
@MaxDeg A Json
is a discriminated union of all the possible types JSON can be, that is, a number, a string, a boolean, a null, an array, an object. A JsonObject
is the type used to hold the properties of an object. If you have a Json
, you may be able to decode it to a JsonObject
(if the JSON actually is an object). If you have a JsonObject
, you can always turn it into a Json
by wrapping it in the Json.Object
case.
In your case you may want to use Decode.jsonObject >=> jsonDecoder { … }
. That will try to decode the Json
as a JsonObject
before passing it along to your decoder.
This difference is important as it allows you to create mixins, where you can decode several types out of the same object without needing to unwrap the Json
for each mixin. You do it once with Decode.jsonObject
, and then you can use your Decoder<JsonObject,'a>
to extract your 'a
from there.
@neoeinstein I've made a little script to try out the new beta.
#r "./packages/json/FParsec/lib/net40-client/FParsec.dll"
#r "./packages/json/FParsec/lib/net40-client/FParsecCS.dll"
#r "./packages/json/Chiron/lib/net45/Chiron.dll"
open Chiron
module E = Json.Encode
type Person = {
Name:string
Age:int
City:string
}
type Person with
static member ToJson (x:Person) (jObj) =
jObj
|> E.required E.string "name" x.Name
|> E.required E.int "age" x.Age
|> E.required E.string "city" x.City
{ Name = "Barry"; Age = 22; City = "New York"}
|> E.jsonObjectWith Person.ToJson
|> Json.formatWith JsonFormattingOptions.Pretty
Is this the correct approach to serialize?
I've heard that people are using chiron 7 in production. Perhaps it's time to merge this pull request? Ir there anything left on it?
@neoeinstein is there anything I can do to help move this forward?
My congratulations to Dave (@ninjarobot). I've given you publish rights to the NuGet feed as well as admin collaborator rights on this repository. I'm happy to help out from time to time, but I think that Chiron 7 is in a pretty good state. Dave, feel free to pull it out of alpha and release it to the world as stable.
I hope we see additional improvement of this library! Good luck on maintaining it @ninjarobot ! 😀
I am just moving to the latest available version and I have some questions about performance or rather about the terminology used in the description above.
What are the differences between
Inline_Explicit_ComputationExpression and Inline_Inferred_ComputationExpression
or Inline_Inferred_ComputationExpression and Inline_Inferred_Operators .
Are there any code examples for this available?
My biggest isse with Chiron is indeed performance atm. I have large object graphs that I need to deserialize (and many of those in one go) and it is not uncommon for a end user initiated action to take 5-10 seconds or even 20s. which is much to much. So I need to get the most out of Chiron. Thx
Would it be possible to have an upgrade path from 6 -> 7 that doesn't break the user's code and let one easy into 7 gently?
This is a rather large, breaking rewrite of Chiron 7. The API has been altered and the code optimized.
This is nearly ready to go. It just needs a bit of API cleanup. API documentation should follow shortly after.
What has changed?
serialize
anddeserialize
now refer to the combination ofencode >> format
/parse >> decode
, whileencode
anddecode
take the place of the existingserialize
anddeserialize
.Json.Encode
andJson.Decode
, respectively.Chiron.Inference
. Opening that module will give access to those values.It is a common practice to include a module alias to make it easier to access the encode/decode functions:
This makes it easier to invoke functions such as
E.listWith E.int
JPass
andJFail
Json.readWith
->D.required
Json.readWithOrDefault
when default wasNone
->D.optional
Json.readWithOrDefault decoder key def
->D.optional decoder key >=> D.withDefault def
(with operators) orD.optional decoder key |> Decoder.compose (D.withDefault def)
Json.writeWith
->E.required
Json.writeWithUnlessDefault
when default wasNone
->E.optional
Json.writeWithUnlessDefault
->E.ifNotEqual
Json.read
->DI.required
Json.readOrDefault
when default wasNone
->DI.optional
Json.readOrDefault decoder key def
->DI.optional decoder key >=> D.withDefault def
(with operators) orDI.optional decoder key |> Decoder.compose (D.withDefault def)
Json.write
->EI.required
Json.writeUnlessDefault
when default wasNone
->EI.optional
Json.writeUnlessDefault
->EI.ifNotEqual
jsonReader
or by using the provided operators.Use of struct
Result
The use of a new struct
Result
type was considered. What I found was that operations using a structResult
incurred a performance penalty. While they were nearly allocation free, I found that they could increase the execution time by 100%, resulting in execution times worse than before starting the optimization work. Finding this penalty unacceptable, I chose to use a reference-typeResult
for Chiron.Benchmarks
Benchmarks were run on my local machine.
When parsing:
When formatting:
When encoding:
And when decoding:
I have also tested the new version by implementing a JSON schema parser, and then verifying the amount of time required for each stage: parse, decode, encode, format, parse and decode, encode and format, and a full round-trip. The sample target used was the Swagger 2.0 JSON schema, measuring in at about 39 kiB.