search

iTwin / imodel-transformer

API for exporting an iModel's parts and also importing them into another iModel
MIT License
3 stars 2 forks source link

Add pending reference resolution when referenced element is not exported #80

Closed mindaugasdirg closed 1 year ago

mindaugasdirg commented 1 year ago

If element is not exported all pending references are left in memory until transformation completes. This causes memory issues when a lot of elements are not exported. To reduce memory usage transformer can resolve pending references as soon as it knows that element won't be exported.

CLAassistant commented 1 year ago

CLA assistant check
All committers have signed the CLA.

MichaelBelousov commented 1 year ago

is this still WIP as marked in the title? If so can you mark it as a draft?

mindaugasdirg commented 1 year ago

@MichaelBelousov, sorry forgot to remove WIP from the title. PR is ready for review.

mindaugasdirg commented 1 year ago

I've also updated pipeline since tests were facing SegFault issue mentioned here: https://github.com/iTwin/itwinjs-core/issues/5663#issuecomment-1600764489

MichaelBelousov commented 1 year ago

thinking about this some more, instead of keeping the pending references in memory and only updating them, can we just keep the id of the incomplete element, and after all have been inserted, re-transform all incomplete elements from scratch? So same as what we do now, but rather than update elements as soon as we find their references, wait until the end when everything is inserted, and batch update them.