Add Analytic `Azos.Log.Fact` with arch log encapsulation: {type: atom, dims: {}, data: {}}

[itadapter]

Related:

• 859 BLOB FS

• 864 Json type hints

Big Data analytics project

Schema of analytics event today is:

public Guid EmitAnalyticsEvent(Atom eventType,
                                   object dims,
                                   object metrics,
                                   int source = 0,
                                   Guid rel = default(Guid),
                                   MessageType messageType = MessageType.Info,
                                   DateTime utcTimeStamp = default)

[itadapter]

We would still need HOST, APP, GUID,...why not use Log.Message as-is? What benefit does a new type provide (if any) ? I think it adds bloat and complexity.

This type might be needed in ANALYSIS services, but not here.

[itadapter]

We need to add Fact instead

[itadapter]

Azos.Log.Fact

Use Log.ArchiveConventions to form/extract facts to/from log messages

[itadapter]

Need unit tests for: ArchiveConventions all methods

[itadapter]

Need more Bixon tests

[itadapter]

• Add test for data documents
• Convert DOCs to JsonDataMap using JsonDataMap ToJsonDataMap(JsonWritingOptions options = null), this way we dont need to write doc as Json
• Add perf test JSON vs Bixon

[itadapter]

    2023 May 11 .Net 6 Release
    --------------------------
Complex:
      JSON wrote 32,000 in 1.0 sec at 30,776 ops/sec; 1,590 chars
      JSON read 32,000 in 2.6 sec at 12,181 ops/sec
      BIXON wrote 32,000 in 0.8 sec at 41,327 ops/sec; 1,201 bytes
      BIXON read 32,000 in 0.8 sec at 41,088 ops/sec

Simple:
     JSON wrote 250,000 in 0.8 sec at 315,031 ops/sec; 307 chars
     JSON read 250,000 in 2.3 sec at 108,265 ops/sec
     BIXON wrote 250,000 in 0.8 sec at 329,386 ops/sec; 265 bytes
     BIXON read 250,000 in 0.5 sec at 541,920 ops/sec

[itadapter]

Bixon still needs:

1. Opt out of types maps on deserialization
2. More support for IList(object), IDictionary(kvp) and tests
3. Test case with fields of custom types written as JSON (not data documents)

[itadapter]

2023 May 12 .Net 6 Release
    --------------------------
Complex:
  JSON wrote 32,000 in 1.0 sec at 31,409 ops/sec; 1,590 chars
  JSON read 32,000 in 2.6 sec at 12,312 ops/sec
  BIXON wrote 32,000 in 0.9 sec at 35,354 ops/sec; 1,201 bytes 
  BIXON read 32,000 in 0.7 sec at 42,765 ops/sec

Simple:
  JSON wrote 250,000 in 0.8 sec at 308,273 ops/sec; 307 chars
JSON read 250,000 in 2.3 sec at 110,971 ops/sec
  BIXON wrote 250,000 in 0.9 sec at 272,497 ops/sec; 265 bytes
  BIXON read 250,000 in 0.5 sec at 516,437 ops/sec

[itadapter]

todo:

• add unit test for JsonReader.DocReadOptions.By.BixonDoNotMaterializeDocuments
• check all unit tests manually with printed dumps

[itadapter]

• 867

• 864

• 861

[itadapter]

The JSON serialization for Fact should be akin to that of Message- the EMPTY Rel guids should be excluded from json map:

{
            "FactType": "SnLoad",
            "Id": "5359ff15-dd8d-458e-a22a-d7f736bce777",
            "RelatedId": "00000000-0000-0000-0000-000000000000",//<========================
            "Gdid": "0:2:195833",
            "Channel": "ua-anl",
            "Topic": "ua",
            "Host": "Mac-Studio",
            "App": "uafolio",
            "RecordType": "PerformanceInstrumentation",
            "Source": 0,
            "UtcTimestamp": "2023-05-08T19:09:18.596Z",
            "Dimensions": {
                "scene": "Appt_Dashboard",
                "user": "4309"
            },
            "Metrics": null
        },