Items disappearing when I restart the world

[ghost]

Each time I reload the world some of my items disappear. I lose my crude hammer every time and lost all the nuggets and Josharias berries I had :(

It's sad because otherwise the game is great.

[Cervator]

Thanks for the report @AmeliaMayIX ! I do think I've seen this under some circumstances. Could you elaborate real quick on the exact details and which game download you used?

Did you lose all your items, and even skills and other things? As if you log back into the game with a brand new character so to say

Are you playing in single player and just reloading from the save, or do you run a server and it happens every time you connect?

Are you using the game launcher or a specific game zip? Where did you download from and do you know the exact version number of your game client?

[ghost]

Thanks for the reply @Cervator ! I downloaded the game from https://terasology.org/ I just clicked the button at the top right that says "Get The Game" and it began downloading. In the game all I did was select "Josharias Survival" when I made a new world.

I thought it was just the crude hammer at first, since it disappeared every time I made a new one, but I later realized items such as the Josharias berries and Nuggets I had were missing as well. I dont know if those items disappear every time or not, because they're a bit harder to get. All my logs disappeared once, but most the time they stay. It only seems to be specific items, since items like dirt and stone stay after a restart. My skills are still the same.

I'm running a single player world. The bug happens every time the world restarts. If I exit to the menu and open the world again my items will be gone. (but again, only specific ones)

I'm opening the Terasology.64x.exe I got from the game download via a shortcut. I couldn't find the version number, but the 'VERSION' file has the Build Number listed as 78.

[askneller]

I've done a bit of investigating and it appears the issue is in the serialization (saving), rather than the de-serialization (loading). It looks like the autosave code has an issue resolving some of the org.terasology.persistence.internal.DelayedEntityRefs.

For example here is my inventory:

Here are the logs from the save (I've added extra logging):

18:25:05.810 [main] INFO  o.t.p.i.ReadWriteStorageManager - Auto Saving - Creating game snapshot
18:25:05.810 [main] INFO  o.t.p.i.ReadWriteStorageManager - Creating save transaction
18:25:05.811 [main] INFO  o.t.p.i.ReadWriteStorageManager - Adding player
18:25:05.811 [Saving-0] INFO  o.t.p.internal.SaveTransaction - Running
18:25:05.811 [main] INFO  o.t.p.i.ReadWriteStorageManager - Auto Saving - Snapshot created: Writing phase starts
18:25:05.825 [Saving-0] INFO  o.t.p.internal.SaveTransaction - Preparing player stores
18:25:05.828 [Saving-0] INFO  o.t.p.serializers.EntitySerializer - Found player EntityRef{id = 1076, netId = 0, prefab = 'engine:player'}
18:25:05.828 [Saving-0] INFO  o.t.p.serializers.EntitySerializer - serializeEntityDelta
18:25:05.828 [Saving-0] INFO  o.t.p.serializers.EntitySerializer - Inventory
InventoryComponent{privateToOwner=true, itemSlots=[EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, org.terasology.persistence.internal.DelayedEntityRef@7000, org.terasology.persistence.internal.DelayedEntityRef@229e, EntityRef{id=0}, org.terasology.persistence.internal.DelayedEntityRef@5a2a, org.terasology.persistence.internal.DelayedEntityRef@6532, EntityRef{id=0}, org.terasology.persistence.internal.DelayedEntityRef@5633, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}]}
18:25:05.828 [Saving-0] INFO  o.t.p.s.ComponentSerializer - Serialise, usingFieldIds false, prefab InventoryComponent, delta InventoryComponent
18:25:05.828 [Saving-0] INFO  o.t.p.s.ComponentSerializer - orig true
delta true
18:25:05.828 [Saving-0] INFO  o.t.p.s.ComponentSerializer - orig [EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}]
delta [EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, org.terasology.persistence.internal.DelayedEntityRef@7000, org.terasology.persistence.internal.DelayedEntityRef@229e, EntityRef{id=0}, org.terasology.persistence.internal.DelayedEntityRef@5a2a, org.terasology.persistence.internal.DelayedEntityRef@6532, EntityRef{id=0}, org.terasology.persistence.internal.DelayedEntityRef@5633, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}, EntityRef{id=0}]
18:25:05.828 [Saving-0] INFO  o.t.p.s.ComponentSerializer - persistedData class ProtobufPersistedData
18:25:05.829 [Saving-0] INFO  o.t.p.s.ComponentSerializer - protobuf data value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
  long: 8831
}
value {
}
value {
}
value {
}
value {
}
value {
  long: 22036
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}
value {
}

18:25:05.830 [Saving-0] INFO  o.t.p.s.ComponentSerializer - 

EntityData.Component to JSON
{
  "itemSlots": [
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    8831,
    null,
    null,
    null,
    null,
    22036,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
  ]
}

18:25:07.234 [Saving-0] INFO  o.t.p.internal.SaveTransaction - Save game finished

The printout of the InventoryComponent before converting to protobuf format is here (formatted for easier viewing):

[
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  org.terasology.persistence.internal.DelayedEntityRef@7000,
  org.terasology.persistence.internal.DelayedEntityRef@229e,
  EntityRef{
    id=0
  },
  org.terasology.persistence.internal.DelayedEntityRef@5a2a,
  org.terasology.persistence.internal.DelayedEntityRef@6532,
  EntityRef{
    id=0
  },
  org.terasology.persistence.internal.DelayedEntityRef@5633,
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  },
  EntityRef{
    id=0
  }
]

You can see that starting from slot 11 there are 5 entries that correspond to the items in the first line of the main inventory.

However you can see in the main log that, once the value has been converted to ProtobufPersistedData in ComponentSerializer (the line that starts with 18:25:05.829 [Saving-0] INFO o.t.p.s.ComponentSerializer - protobuf data value), only 2 of the entries have been saved to the protobuf class (entities with ids 8831 and 22036). This is why I suspect it might be related to the DelayedEntityRef.

Also of note is that when the save on exit is carried out on the main thread, delayed refs aren't used and all items appear to be converted to protobuf data correctly. I think the reason that it doesn't save correctly on exit is that both the off-thread and on-thread saves are run, and it seems to be a race condition as to which actually successfully saves.

[pollend]

I ran git bisect and this is what I narrowed it down to. I'm looking at it and i'm not sure how this would help resolve the serialization problem. I don't see a direct relationship between the two.

17da5d703de990b6ca29c682650963d4136ea03d is the first bad commit
commit 17da5d703de990b6ca29c682650963d4136ea03d
Author: Arav Singhal <singhal.arav@gmail.com>
Date:   Thu Jun 27 15:38:25 2019 +0530

    Remove redundant TypeHandlerLibrary initialization

:040000 040000 5da4a4eeb933543cf4fb4868c6b9d9fd78787154 23c885392aa49a4b2b93edf53494387bc7ae1c13 M      engine
maybe that isn't right

[Cervator]

This should now be resolved via https://github.com/MovingBlocks/Terasology/pull/3903 !!

Thanks @askneller for the initial research, gave some good pointers. @pollend was able to track down the related commit, and @eviltak explained the reason behind the problem. All fixed now, and other bugs too that turned out to be related to this! Big win :-)

@AmeliaMayIX this is fixed in the latest source version, but it may take a few days or till the weekend to get a new game release out with the fix. But this issue will be a thing of the past in short order now!