Request - Allow inserting JObjects

[cecilphillip]

This is small feature request/recommendation. Sometimes I like working with raw JSON objects. It would be great if the driver enabled inserting raw JObjects into tables. Right now, the following code throws an error.

JObject state = .... ;
 _r.db(DATABASE_NAME).table(EVENTS_TABLE_NAME)
                .insert(state).run(_conn);

[bchavez]

Hi @cecilphillip , this is really good idea. Thanks. I did a preliminary test and I'm getting a response from the server:

"Expected type OBJECT but found ARRAY."

Is that the same error your seeing too?

Give me a few to think about how to pull this off in the driver and I'll get back to you.

[cecilphillip]

Yep, that's the error I got. For now, I'm just using ExpandoObject. So something like this works fine today:

      dynamic expand = new ExpandoObject();
      expand.event_type = "dnm.twitter.post";
      expand.date_create = DateTime.UtcNow;
      expand.properties = new
              {
                  username = "@cecilphillip",
                  message = "140 character is too long"
              };

     _r.db(DATABASE_NAME).table(EVENTS_TABLE_NAME)
                .insert(expand).run(_conn);

This works also, but taking the perf hit with the DLR probably isn't worth it. I haven't measured it.

  dynamic jexpand =  JsonConvert.DeserializeObject<ExpandoObject>(myJObject.ToString(), new ExpandoObjectConverter());

and I'd assume using a dictionary of some sort should work too, but I haven't tired that yet.

I'll think about it too and let you know if I come up with any ideas. Thanks for considering this feature.

[bchavez]

Hi @cecilphillip ,

I think I got your feature working. Here is the test:

public class TheJObject
{
    public string TheString { get; set; }
    public float TheFloat { get; set; }
    public double TheDouble { get; set; }
    public decimal TheDecimal { get; set; }
    public byte[] TheBinary { get; set; }
    public bool TheBoolean { get; set; }
    public DateTime TheDateTime { get; set; }
    public DateTimeOffset TheDateTimeOffset { get; set; }
    public Guid TheGuid { get; set; }
    public TimeSpan TheTimeSpan { get; set; }
    public int TheInt { get; set; }
    public long TheLong { get; set; }
}
[Test]
public void issue_21_allow_JObject_inserts()
{
    var table = r.db(DbName).table(TableName);
    table.delete().run(conn);

    var state = new JObject
        {
            ["TheString"] = "issue 21",
            ["TheFloat"] = 25.2f,
            ["TheDouble"] = 25.3d,
            ["TheDecimal"] = 25.4m,
            ["TheBinary"] = new byte[] {0, 2, 3, 255},
            ["TheBoolean"] = true,
            ["TheDateTime"] = new DateTime(2011, 11, 1, 11, 11, 11, DateTimeKind.Local),
            ["TheDateTimeOffset"] = new DateTimeOffset(2011, 11, 1, 11, 11, 11,11, TimeSpan.FromHours(-8)).ToUniversalTime(),
            ["TheGuid"] = Guid.Empty,
            ["TheTimeSpan"] = TimeSpan.FromHours(3),
            ["TheInt"] = 25,
            ["TheLong"] = 82342342234
        };

    Console.WriteLine(">>> INSERT");
    var result = table.insert(state).runResult(conn);
    var id = result.GeneratedKeys[0];
    result.Dump();

    var check = table.get(id).runAtom<TheJObject>(conn);
    check.Dump();

    check.TheString.Should().Be((string)state["TheString"]);
    check.TheFloat.Should().Be((float)state["TheFloat"]);
    check.TheDouble.Should().Be((double)state["TheDouble"]);
    check.TheDecimal.Should().Be((decimal)state["TheDecimal"]);
    check.TheBinary.Should().BeEquivalentTo((byte[])state["TheBinary"]);
    check.TheBoolean.Should().Be((bool)state["TheBoolean"]);
    check.TheDateTime.Should().Be((DateTime)state["TheDateTime"]);
    check.TheDateTimeOffset.Should().Be((DateTimeOffset)state["TheDateTimeOffset"]);
    check.TheGuid.Should().Be((Guid)state["TheGuid"]);
    check.TheTimeSpan.Should().Be((TimeSpan)state["TheTimeSpan"]);
    check.TheInt.Should().Be((int)state["TheInt"]);
    check.TheLong.Should().Be((long)state["TheLong"]);
}

Please give v2.2.2-beta-1 a try and let me know if it works for you.

It should be available in about 15 to 20 minutes when the CI server uploads it to NuGet. I'll be back later tonight to check back.

Thanks, Brian

[bchavez]

Also, just wanted to make a quick note:

You can use an anonymous type for inserting too:

var state =  new {
     event_type = "dnm.twitter.post",
     date_create = DateTime.UtcNow,
     properties = new
        {
            username = "@cecilphillip",
            message = "140 character is too long"
        };
 }

r.db(DATABASE_NAME).table(EVENTS_TABLE_NAME)
          .insert(state).run(_conn);

[cecilphillip]

I'll give it a try in the morning and let you know

[cecilphillip]

This works:

  string json = @"
            {
                ""Entered"": ""2012 - 08 - 18T13: 26:37.7137482 - 10:00"",
                ""AlbumName"": ""Dirty Deeds Done Dirt Cheap"",
                ""Artist"": ""AC/DC"",
                ""YearReleased"": 1976               
            } ";

   var obj = JObject.Parse(json);             
   var result = _r.db("test").table("dump").insert(obj).runResult(conn);

but a more complex type like this doesn't

string json = @"
            {
                ""Entered"": ""2012 - 08 - 18T13: 26:37.7137482 - 10:00"",
                ""AlbumName"": ""Dirty Deeds Done Dirt Cheap"",
                ""Artist"": ""AC/DC"",
                ""YearReleased"": 1976,
                ""Songs"": [
                {
                    ""SongName"": ""Dirty Deeds Done Dirt Cheap"",
                    ""SongLength"": ""4:11""
                },
                {
                    ""SongName"": ""Love at First Feel"",
                    ""SongLength"": ""3:10""
                }
                ]
            }";

I get an error that says

{"The query response can't be converted to an object of T. The query response is not SUCCESS_ATOM. The response received was COMPILE_ERROR. Use `.run` and inspect the response manually. Ensure that your query result is something that can be converted to an object of T.  Most likely your query returns a STREAM and you should be using `.runCursor`."}

[bchavez]

I see. Hmm... handling this is going to be slightly more complicated than I originally thought.

I probably need to think about this a bit more... I'm a bit tired at the moment, but a quick look leads me to think there's a few places to handle this more thoroughly.

1. Ast.Util.ToReqlAst, instead of upgrading it to a Poco(JObject) convert the JObject to a IDictionary<string, JToken> and convert each K/V recursively similarly how the original code converts an IDictionary.
2. Implement a more high-level JObjectConverter : JsonConverter that basically does the same thing as No.1.

I think, ultimately, the final implementation will only support the serialization of native types that Newtonsoft.Json supports. I _think_ that might be okay. One edge case I'd need to cover is:

var obj = new JObject(){ 
   ["AnObjectNotHandledByNewtonsoft"] = new SomeObject()
 }

Let me sleep on it and I'll take a second shot at it in the morning. =)

[bchavez]

And, a more elegant solution might be to instantly convert JObject to a string and wrap it with r.json(jobject_as_string). This would quite possibly 1) save a lot of headache, 2) avoid all the object conversions in memory, 3) more performance.

[bchavez]

Hi @cecilphillip

I hope I got it this time, please give v2.2.2-beta-2 a try. New version should be on NuGet soon. Let me know how it goes.

Also, the driver should detect DateTime and byte[] when sending the JObject to the server and automatically convert them to RethinkDB pseudo types. As long as Newtonsoft detects these native types the conversion should automatically take place.

I went with the last implementation using r.json. Ultimately, this lead to a more appropriate implementation of ReqlDateTimeConverter; which now directly serializes DateTimes directly to JSON pseudo $reql_time$:TIME instead of trying to cheat by sending the server a Iso8601(string) AST object.

[cecilphillip]

@bchavez I played around with it. This looks good :+1:

[bchavez]

Great! Happy it is working for you. Feel free to reopen the issue if you encounter any issues. Docs are updated with this new feature and all previous unit tests are passing. We should be good.