Dzoukr / CosmoStore

F# Event store for Azure Cosmos DB, Table Storage, Postgres, LiteDB & ServiceStack
MIT License
168 stars 21 forks source link

One or more errors occurred. (unordered field Item) - This is probably more a question. #51

Open marcingolenia opened 3 years ago

marcingolenia commented 3 years ago

Hi there! I am looking for an event store which will be friendly to F#, so the one written in F# should be perfect ;D I tried to use it with postgres + marten, but I am failing to read events if they are represented by discriminated union. Let me just give You the test that I run;

module Tests

open System
open System.Threading.Tasks
open Xunit
open CosmoStore.Marten
open CosmoStore
open FsUnit.Xunit

type DummyEvent = { IncrementedBy: int }
type DummyEvent2 = { Date: DateTime }
type Dummies =
    | Dummy1 of DummyEvent
    | Dummy2 of DummyEvent2

[<Fact>]
let ``Saving and reading`` () =
    // Arrange
    let streamId = "MyAmazingStream"

    let config: Configuration =
        { Host = "localhost"
          Username = "postgres"
          Password = "Secret!Passw0rd"
          Database = "cativity" }

    let es = EventStore.getEventStore config

    let eventWrite: EventWrite<Dummies> =
        { Id = Guid.NewGuid()
          CorrelationId = None
          CausationId = None
          Name = nameof DummyEvent
          Data = Dummies.Dummy1{ IncrementedBy = 4 }
          Metadata = None }
    // Act + Assert
    [ eventWrite ]
        |> es.AppendEvents streamId ExpectedVersion.Any
        |> Async.AwaitTask
        |> Async.RunSynchronously
        |> ignore
    let events =
        EventsReadRange.AllEvents
        |> es.GetEvents "MyAmazingStream"
        |> Async.AwaitTask
        |> Async.RunSynchronously
    [ events.Head.Data ] |> should equal [ eventWrite.Data ]

The test breaks on event read.

System.AggregateException
One or more errors occurred. (unordered field Item)

So it is quite mysterious. Probably I am doing something wrong. Without DU it works just OK - so if we will use EventWrite<DummyEvent> Any hints? Thanks guys in advance!

Dzoukr commented 3 years ago

Hi,

I'll ping @kunjee17 as he is more aware of how Marten works.

kunjee17 commented 3 years ago

@marcingolenia Hi, I guess we are not yet supporting that type for serialization. Marten is C# library so we have to manually support Serializing type. That we are doing here https://github.com/Dzoukr/CosmoStore/blob/master/src/CosmoStore.Marten/EventStore.fs#L169 . Best way to check is if things are getting serialized without Marten. Means using serializer in isolation.

Or you might need to add one more serializer to mix. Currently there are three. DU is supported and I guess it is added by @TheAngryByrd . For your specific DU I m not quite sure.

bartelink commented 3 years ago

FsCodec has a UnionConverter which should interoperate with the code Kunjee linked (you tag the type so no registration into the converters list required). (Note FsCodec almost has STJ support but the single thing not merged yet is the port of that very converter - IIRC Marten V4 has STJ support though I assume that's not mandatory)

marcingolenia commented 3 years ago

@kunjee17 I've tried to do the serialization and deserialization using martenSerializer in isolation like this;

    let json = EventStore.martenSerializer.ToJson (Dummies.Dummy1 { IncrementedBy = 4 })
    let o = EventStore.martenSerializer.FromJson(typeof<Dummies>, new StringReader(json))

and it succeeded... what is strange as I can clearly see that it is passed to marten.

marcingolenia commented 3 years ago

I've cloned the repo to investigate further. Did you try to build on Linux? I am running ubuntu, after paket restore I run dotnet build and this is what I have;

error MSB3202: The project file "/home/mgolenia/projects/CosmoStore/tests/CosmoStore.LiteDb.Tests/CosmoStore.LiteDb.Tests.fsproj" was not found. [/home/mgolenia/projects/CosmoStore/CosmoStore.sln]

Ease to fix. After I add existing project I get;

Solution file error MSB5023: Error parsing the nested project section in solution file. A project with the GUID "{058D6307-B865-49E9-BF08-A721B1CE3AA7}" is listed as being nested under project "{B0586636-E0F7-438F-B11A-315940D4A108}", but does not exist in the solution.

After I removed few guids from the sln I've run into next troubles;

could not be found [/home/mgolenia/projects/CosmoStore/src/CosmoStore.LiteDb/CosmoStore.LiteDb.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/Program.fs(19,13): error FS0074: The type referenced through 'CosmoStore.EventStore' is defined in an assembly that is not referenced. You must add a reference to assembly 'CosmoStore'. [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/CosmoStore.TableStorage.Tests.V2.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/Program.fs(33,13): error FS0039: The type 'EventWrite' is not defined. Maybe you want one of the following:   Event   EventArgs   Event`1   Event`2   EventHandler [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/CosmoStore.TableStorage.Tests.V2.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/Program.fs(27,13): error FS0039: The record label 'Id' is not defined. [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/CosmoStore.TableStorage.Tests.V2.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/Program.fs(36,5): error FS0072: Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.TableStorage.Tests.V2/CosmoStore.TableStorage.Tests.V2.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/Program.fs(19,15): error FS0074: The type referenced through 'CosmoStore.EventStore' is defined in an assembly that is not referenced. You must add a reference to assembly 'CosmoStore'. [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/CosmoStore.CosmosDb.Tests.V2.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/Program.fs(34,13): error FS0039: The type 'EventWrite' is not defined. Maybe you want one of the following:   Event   EventArgs   Event`1   Event`2   EventHandler [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/CosmoStore.CosmosDb.Tests.V2.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/Program.fs(28,13): error FS0039: The record label 'Id' is not defined. [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/CosmoStore.CosmosDb.Tests.V2.fsproj]
/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/Program.fs(37,5): error FS0072: Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. [/home/mgolenia/projects/CosmoStore/tests/CosmoStore.CosmosDb.Tests.V2/CosmoStore.CosmosDb.Tests.V2.fsproj]

So I have removed LiteDb implementation + tests, TableStorage.V2.Tests + CosmosStore.CosmosDb.Tests.V2. Then I was able to build the solution. I am looking forward to give some contribution but easy clone and build from master would be sth nice to have from the start.

kunjee17 commented 3 years ago

@marcingolenia It seems to be dotnet issue. All projects are still on v3. And if you are installing latest dotnet core then it would be on 5. I have developed all implementation on Mac. So, OS should not be issue if dot net core is supported there.

marcingolenia commented 3 years ago

Hi, I am back on this. I have added global.json

{
    "sdk": {
        "version": "3.1.404"
    }
}

And the issues are still there. After I did the things I've already mentioned I tried to run fake with TestMarten target and this happens: fake run build.fsx -t TestMarten output:

Stack overflow.
Finished (Failed) 'TestMarten' in 00:00:07.2201011

Can one of the authors look into this? If it is not straightforward can I count on contribution guide? How to build/test at least. Now I cannot do it using CLI or Rider IDE.

kunjee17 commented 3 years ago

@marcingolenia have you check the connection for poatgres ? If that is working?

marcingolenia commented 3 years ago

I did not, I would do it in case of SocketException from NpgsqlConnector, not overflow. Would You accept a PR with;

  1. sln fixes
  2. References fixes in projects
  3. Tests.V2 fixes (to do that I need to ask You what is ExpectedPosition.Any? It seems to be a typo, did you want to write ExpectedVersion.Any)?
  4. Github action with build only (tests with docker: postrges and Azurite may wait) + badge in readme.
  5. Better error messages from the test (at least I will try, I have been using xunit for many years, didn't look into expecto yet).
  6. And probably .net upgarde to 5.0.300 if it will be smooth, in case of troubles I will submit another PR.
kunjee17 commented 3 years ago

cc/ @Dzoukr .

I guess I m ok with improvement. I haven't touch the code since ages. These seems good improvement.

Dzoukr commented 3 years ago

Yup, any improvement is more than welcome. :)

yankun commented 2 years ago

I'm having the exactly same issue. If we can't use discriminated unions for events, thats at last for me a major drawback. How could this be fixed? Googling for "unordered field Item" this is the only hit, yeah.

kunjee17 commented 2 years ago

@yankun it is mostly serialization issue. You can use different strategy to overcome this. Just search of F# union object json serialization. if I remember correctly there is F# specific wrapper around JSON.net and one blog post in fsharpforfunandprofit on this as well.