Saving double loses edges or produces unmarshalling error

[andreyvk]

Hi @Ostico,

I have trouble reproducing this with test classes now, but it happens on the system all the time. I will try to reproduce asap and add a post here later. So the problems are:

I have a class with an attribute of type embeddedmap double. Vertices of that class are connected with edges to other vertices. Whenever I set that attribute value to something like ["value1" => (double) 123123123] (i.e. a large double) it produces on of the following errors or problems:

1. The next time I pull record from the database library throws an error:

com.orientechnologies.orient.core.exception.OSerializationException: Error on unmarshalling field 'freightReceipts' in record #105:44 with value: freightReceipts:{"lastYear":"1000.0","thisYear":2000d,"nextYear":3.09d,"E7d}

1. One or more of edges would disappear and that attribute will look like:

Array
(   
    [value1] => 1.23123123
    [E8d},out_belongs_to] => PhpOrient\\Protocols\\Binary\\Data\\Bag Object
        (
            [base64Content:protected] => AQAAAAIAXgAAAAAAACZ9AF4AAAAAAAAmfgAAAAAAAAAAAAAAAAAAAAAAAAAA
            [binaryContent:protected] => 
            [type:protected] => 
            [size:protected] => 
            [uuid:protected] => 
            [fileId:protected] => 
            [pageIndex:protected] => 
            [pageOffset:protected] =>
            [changeSize:protected] => 
            [ReaderOffset:protected] => 0
            [baseOffset:protected] => 0
            [offset:protected] => 0
            [items:protected] => Array
                (
                )

        )

        ...
)

It looks like large doubles are converted to the scientific notation at some point and it produces problems with marshalling/unmashalling the record! Please take a look at this. This is important as we are not able to go live with this.

Thanks

[andreyvk]

Hi @Ostico ,

I've been able to reproduce a part of the problem:

SCHEMA

create class Test extends V

create property Test.stringList embeddedlist string
create property Test.stringMap embeddedmap string
create property Test.stringValue string

create property Test.doubleList embeddedlist double
create property Test.doubleMap embeddedmap double
create property Test.doubleValue double

create property Test.longList embeddedlist long
create property Test.longMap embeddedmap long
create property Test.longValue long

CODE

$client = new \PhpOrient\PhpOrient( 'localhost', 2424 );
$client->username = 'nsadmin';
$client->password = 'nsadmin';
$client->connect();
$client->dbOpen( 'nsdev', 'nsadmin', 'nsadmin' );

$client->command("delete vertex Test"); //remove old stuff

//create record
$odata= [
    "stringMap" => [
        "attr1" => (double) 1231231232,
        "attr2" => (int) 1231231233,
        "attr3" => "hi there",
    ],

    "doubleValue" => (double) 123123123,

    "stringList" => [
        "aaaa", "bbbb"
    ],

    "doubleMap" => [
        "attr1" => 1231231232,
        "attr2" => (double) 1231231232,
    ],

    "longList" => [
        123123123, (double) 123123123
    ],

    "stringValue" => 123123123,

    "doubleList" => [
        123123123, (double) 123123123
    ],

    "longValue" => 123123123,

    "longMap" => [
        "attr1" => 1231231232,
        "attr2" => (double) 1231231232,
        "attr3" => "1231231232",
    ],

];

$rec = ( new Record() )->setOData( $odata )->setOClass("Test")->setRid( new ID( 100 ) );
$rec = $client->recordCreate( $rec );

//re-load record
$record = $client->recordLoad( $rec->getRid() )[0];

//output
print_r($record->getOData());

The result is only the following:

OUTPUT

Array
(
    [stringMap] => Array
        (
            [attr1] => 1231231232d
            [attr2] => 1231231233
            [attr3] => hi there
        )

    [doubleValue] => 1.23123123
)

A huge portion of data is lost during deserialization process from the server! Also note what doubleValue looks like - 1.23123123. It looks like deserialization of scientific notation screwed things up. If next attribute after it was an edge, then upon the next save that or all edges would be lost.

However, if you query orient directly, then record is intact.

The problem seems to go away, if you comment outdoubleValue, doubleMap and doubleList attributes from the $odata and try again. Apparently there's something very wrong with the deserialization of floating points. Please take a look at this.

[Ostico]

Thank you very much @andreyvk , i will check this week-end

[andreyvk]

@Ostico, thanks. I hope u can resolve this quickly!

[Ostico]

Should be fixed

[andreyvk]

Cheers @Ostico . Are you going to do a release soon or should I use dev-master branch?

[Ostico]

Hi @andreyvk ,

For now, use dev-master. I don't know when i can do a new release

[andreyvk]

Dully noted. Thanks!

[andreyvk]

@Ostico, as I understand you numbers remain a sting in the library. As of this fix, some of my numbers on the screen changed to a scientific notation. This is not an expected behavior and now it's necessary to explicitly cast those numbers to a double, so that problem is gone. Do you think it's possible to do that conversion internally in the libary, when you detect a scientific notation number?

Say, you could convert it to a number and then cast it back into a string. This way the scientific notation would disappear

[Ostico]

@andreyvk i did not understand exactly what you described.

Send me an example of what you obtained and what you expected, i will try to figure out and implement a solution.

[andreyvk]

@Ostico, here's an example. This is how a large number now shows in the form field:

It's not a problem, but I now always have to convert numbers explicitly to a double or float. E.g. `<?= (double) $rec["myLargeNumber"] ?>, which is inconvenient by all means.

So maybe it's more user friendly to convert them to a non-scientific notation in the library?

[Ostico]

Now should be better :)

[andreyvk]

Great! Will try this now