search

orientechnologies / OrientDB-NET.binary

C#/.NET binary driver for OrientDB
MIT License
93 stars 59 forks source link

Transaction.AddEdge method throwing InvalidCastException. #119

Open sitty opened 8 years ago

sitty commented 8 years ago

I'm creating a graph of people and and their accounts. I have Contact and Company as vertices and an Edge(named Account) from the company to Contact. Contacts have links to more than one Companies. I'm trying to do a bulk load of this so I'm using database.Transaction class. Below is my code. 

using (ODatabase database = new ODatabase(_databaseAlias))
{
    ODocument oDoc = new ODocument();
    oDoc.OClassName = "Firm";
    List<OVertex> _accountDocs = database.Select().From(oDoc).ToList<OVertex>();
    logger.Info("Preparing to write to the database");
    try
    {
        int i = 0;
        foreach (var contact in contacts)
        {
            try
            {
                OVertex vContact = new OVertex();
                vContact.OClassName = "Contact";
                vContact.SetField("FullName", contact.Person.FullName);
                vContact.SetField("Team", contact.Person.Team);
                vContact.SetField("SchoolAttended", contact.Person.SchoolAttended);
                database.Transaction.AddOrUpdate(vContact);
                if (contact.Bio != null)
                {
                    foreach(var acct in contact.Bio)
                    {
                        OVertex firm;
                        OEdge account = new OEdge();
                        account.OClassName = "Account";
                        if (acct.StartYear.HasValue)
                        {
                            account.SetField("From_Date", acct.StartYear.ToString());
                        }
                        if (acct.EndYear.HasValue)
                        {
                            account.SetField("To_Date", acct.EndYear.ToString());
                        }
                        if (!_accounts.TryGetValue(acct.Organization, out firm))
                        {
                            firm = new OVertex();
                            firm.OClassName = "Firm";
                            firm.SetField("name", acct.Organization);
                            database.Transaction.Add(firm);
                            if (!_accounts.ContainsKey(acct.Organization))
                            {
                                _accounts.Add(acct.Organization, firm);
                            }
                        }
                        database.Transaction.AddEdge(account, firm, vContact);
                    }
                }
                i++;
                if( i % 1000 == 0)
                {
                    database.Transaction.Commit();
                }
            }
            catch (Exception ex)
            {
                logger.Error(ex, $"Error saving contact {contact.Person.FullName}");
            }
        }
        database.Transaction.Commit();
    }
    catch (Exception e)
    {
        logger.Error(e);
    }
}

randomly the database.Transaction.AddEdge throws the following InvalidCastException

Unable to cast object of type 'System.Boolean' to type 'System.Collections.IEnumerable'.
   at Orient.Client.ODocument.GetField[T](String fieldPath)
   at Orient.Client.API.OTransaction.appendOridToField(ODocument document, String field, ORID orid)
   at Orient.Client.API.OTransaction.AddEdge(OEdge edge, OVertex from, OVertex to)
   at BioParser.GraphOrientDB.SaveContacts(List`1 contacts) in C:\dev-git\bioparser\app\BioParser\GraphOrientDB.cs:line 117

I debugged the code and what seems to happening is first time I add an Edge it adds a field in_Account to the vContact which is a hashset. The second time I add another edge that in_Account gets set to True not sure how. The third time i try to add and edge for the same contact it throws this invalid cast exception.

GrayDelacluyse commented 8 years ago

Could you turn on verbose binary logging and upload the output you see from the server at that time?

sitty commented 8 years ago

This is what I got in the log files. 2016-10-24 10:44:02:309 FINE {db=TESTDB} onOpen [OLuceneIndexFactory] 2016-10-24 10:44:02:318 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:02:352 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:02:368 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:02:370 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:02:375 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:02:378 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:02:388 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:02:416 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:02:428 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:02:430 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:02:440 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:02:441 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:08:879 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:08:880 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:08:888 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:08:889 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:08:894 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:08:895 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:09:998 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:09:999 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:10:006 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:10:008 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:10:996 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:10:997 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:11:005 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:11:006 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:11:989 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:11:990 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:19:703 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:19:704 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:19:706 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:19:707 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:37:685 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:37:685 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:42:094 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:42:095 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:42:098 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:42:099 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:44:42:782 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:44:42:783 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:00:640 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:00:640 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:00:643 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:00:644 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:00:646 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:00:647 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:32:281 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:32:282 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:32:285 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:32:286 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:32:984 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:32:985 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:32:987 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:32:987 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:33:536 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:33:538 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:34:153 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:34:154 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:34:156 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:34:157 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:34:792 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:34:793 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:34:795 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:34:796 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:34:801 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:34:803 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:35:401 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:35:402 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:35:404 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:35:404 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:35:928 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:35:929 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:35:931 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:35:931 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:36:408 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:36:409 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:36:811 FINE {db=TESTDB} Request id:1 type:41 [ONetworkProtocolBinary] 2016-10-24 10:45:36:812 FINE {db=TESTDB} Read record #0:1 v.101 size=14725 bytes [OLocalPaginatedStorage] 2016-10-24 10:45:40:160 FINE [OClientConnectionManager] found and removed pending closed channel 1 (null) [OClientConnectionManager] 2016-10-24 10:45:40:160 FINE {db=TESTDB} onClose [OLuceneIndexFactory] 2016-10-24 10:45:42:855 WARNI Received signal: SIGINT [OSignalHandler] 2016-10-24 10:45:42:855 INFO OrientDB Server is shutting down... [OServer]

sitty commented 8 years ago

I figured out the issue. The bug in is the OTranscation.appendOridToField method. Is issue in the the following lines.

if (document.HasField(field))
 {
      document.SetField(field, document.GetField<HashSet<ORID>>(field).Add(orid));
 }

document.SetField is setting the return of HashSet.Add method (which is a bool) instead of set the HashSet itself. You could change it as 

 if (document.HasField(field))
{
     var set = document.GetField <HashSet<ORID>>(field);
     set.Add(orid);
    document.SetField(field, set);

}
GrayDelacluyse commented 8 years ago

Great find! We'll make that change.