Json Queries: No Objects found for guids / duplicates in indices

[muren400]

Hi,

sometimes during the checkin process, the BIMserver needs to do a rollback and reimport the data. If this happens, json queries for a list of guids wont find any objects. (There do exist objects with these guids)

The first comment in StreamingCheckinDatabaseAction.rollback() suggests, that it might be neccesary to remove the indices during rollback. // TODO do we need to remove indices too?

I wasnt quite sure, how to do this, so I investigated QueryGuidsAndTypesStackFrame.getOidOfGuidAlternative() instead and replaced

byte[] firstDuplicate = databaseInterface.get(indexTableName, valueBuffer.array());
if (firstDuplicate != null) {
    ByteBuffer buffer = ByteBuffer.wrap(firstDuplicate);
    buffer.getInt(); // pid
    long oid = buffer.getLong();

    return new ObjectIdentifier(oid, (short)oid);
}

with

List<byte[]> duplicates = databaseInterface.getDuplicates(indexTableName, valueBuffer.array());
if (duplicates != null && duplicates.isEmpty() == false) {              
    byte[] lastDuplicate = duplicates.get(duplicates.size()-1);

    ByteBuffer buffer = ByteBuffer.wrap(lastDuplicate);
    buffer.getInt(); // pid
    long oid = buffer.getLong();

    return new ObjectIdentifier(oid, (short)oid);
}

which did the trick. So it seems, there are actually old and invalid duplicates left in the index.

Not sure if this is the best solution though, since its probably best to actually remove the indexed entries (as suggested in the comment) and not have duplicates at all. But I needed this to also work for already imported files.

Is there a way I can reset these indices externally through the BIMserver API? Because we have BIMserver Instances in production and I'd rather not fiddle around and replace any jars in those installations.

Thanks

[hlg]

There is no way to clean this up from the API, as it should be done automatically during rollback if necessary, not upon a user decision. So you won't get around replacing jars if there is something to be fixed.

I am not familiar with BIMserver's database layer and will first have to better understand your issue and dig around the code base. Can you point me to how to reproduce your issue? I tried with aborted checkins due to erroneous IFC files, where referenced entities are missing, but these don't seem to cause any trouble when querying for GUIDs afterwards.

[muren400]

The rollback is triggered by a BimserverConcurrentModificationDatabaseException during DatabaseSession.executeAndCommitAction(). I don't really know, how to reproduce this either. It seems to happen very sporadically. Only way I could think of, would be to programatically throw the Exception once during the checkin process on purpose.