Evaluate / List all places we use nativeDb or platform and any other 'deprecated' APIs.

[nick4598]

This will have implications when it comes time to consume itwinjs 5.0. NativeDb will no longer be accessible at all due to the private symbol approach. We need public API replacements for everything we currently MUST have from NativeDb in the transformer.

We can consider getting rid of some if we deem them no longer necessary, but we definitely can't get rid of them all. Another alternate approach is to somehow give this repo special privileges, but it seems impossible with the private symbol approach. I think this repo is one of the few that was considered acceptable to use internal APIs? But I may be misremembering that. I think its a moot point since itwinjs-core is moving in the direction of private symbols.

Failing PR run that lists all the ones currently in imodel-transformer This doesn't mention all the other packages. ctrl + f for IModelHost.platform and nativeDb to find the rest.

relatedPR that disabled all them for now

LIST GOES HERE:

### Internal Usage
- [ ] ~~nativeDb.extractChangedInstanceIdsFromChangeSets~~
- [ ] ~~nativeDb.schemaToXmlString~~
- [ ] ~~IModelHost.platform.RevisionUtility.recompressRevision~~
- [ ] ~~IModelHost.platform.RevisionUtility.disassembleRevision~~
- [ ] ~~IModelHost.platform.RevisionUtility.assembleRevision~~
- [ ] ~~IModelHost.platform.RevisionUtility.normalizeLzmaParams~~
- [ ] ~~IModelHost.platform.RevisionUtility.computeStatistics~~
- [ ] ~~IModelHost.platform.RevisionUtility.getUncompressSize~~

### Need of @preapproved
- [ ] nativeDb.setITwinId
- [ ] nativeDb.deleteAllTxns
- [ ] nativeDb.openIModel
- [ ] nativeDb.resetBriefcaseId
- [ ] nativeDb.saveLocalValue
- [ ] nativeDb.saveChanges
- [ ] nativeDb.closeFile
- [ ] nativeDb.getChangeTrackingMemoryUsed
- [ ] nativeDb.completeCreateChangeset
- [ ] nativeDb.startCreateChangeset
- [ ] nativeDb.applyChangeset
- [ ] nativeDb.startProfiler
- [ ] nativeDb.stopProfiler
- [ ] IModelHost.platform.DgnDb

### Public
- [ ] nativeDb.exportSchema
- [ ] nativeDb.inlineGeometryPartReferences
- [ ] nativeDb.simplifyElementGeometry
- [ ] nativeDb.insertElement
- [ ] nativeDb.getLastError

• Changes
  • 10/10/2024
    • Removed nativeDb.extractChangedInstanceIdsFromChangeSets and replaced with ChangedInstanceIds.initialize - https://github.com/iTwin/imodel-transformer/pull/213
  • 10/14/2024
    • Removed IModelHost.platform.RevisionUtility because they are not being used by any of our tests - https://github.com/iTwin/imodel-transformer/pull/215
  • 10/15/2024
    • Removed nativeDb.schemaToXmlString and replaced with SchemaTestUtils.schemaToXmlString - https://github.com/iTwin/imodel-transformer/pull/217
  • 10/16/2024
    • Moved nativeDb.getChangeTrackingMemoryUsed to preapproved tasklist - still useful in test-app
    • Moved nativeDb.completeCreateChangeset to preapproved tasklist - still useful in performance-test
    • Moved nativeDb.startCreateChangeset to preapproved tasklist - still useful in performance-test
    • Moved nativeDb.applyChangeset to preapproved tasklist - still useful in performance-test
    • Moved nativeDb.startProfiler to preapproved tasklist - still useful in performance-script
    • Moved nativeDb.stopProfiler to preapproved tasklist - still useful in performance-script
    • Moved IModelHost.platform.DgnDb to preapproved tasklist - still useful in performance-test, test-app, other tests

[derbynn]

MUST HAVE REPLACEMENT:

• exportSchema
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/main/packages/transformer/src/IModelTransformer.ts#L2749
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/cases/prepare-fork.ts#L58
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/cases/identity-transformer.ts#L53
• inlineGeometryPartReferences *
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/IModelImporter.ts#L764
• simplifyElementGeometry *
  • Not found in itwinjs-core
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/IModelImporter.ts#L308
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/IModelImporter.ts#L346
• insertElement * (not sure if we use nativeDb's insertElement on purpose. forceUseId: this.options.preserveELementIdsForFiltering seems important and only available on nativeDb.)
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/main/packages/transformer/src/IModelImporter.ts#L297
• getLastError * (branchprovenanceinitializer which uses attach database, so maybe it needs getLastError)
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/BranchProvenanceInitializer.ts#L77
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/BranchProvenanceInitializer.ts#L89
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/BranchProvenanceInitializer.ts#L115
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/BranchProvenanceInitializer.ts#L121
• setToStandalone seems important for tests. Standalone iModels are read/write files that are not associated with an iTwin or managed by iModelHub
• setITwinId (setToStandalone)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol approach
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/iModelUtils.ts#L37
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Main.ts#L503
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L900
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L996
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L1113
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L1265
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L2112
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L3382
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2169
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/BranchProvenanceInitializer.test.ts#L428
• deleteAllTxns (setToStandalone)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol approach
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/iModelUtils.ts#L39
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Main.ts#L505
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/TestUtils/IModelTestUtils.ts#L898
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2171
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/BranchProvenanceInitializer.test.ts#L430
• openIModel (setToStandalone)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol approach
    • Links to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/iModelUtils.ts#L36
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Main.ts#L502
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2168
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/BranchProvenanceInitializer.test.ts#L427
• resetBriefcaseId (setToStandalone)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/iModelUtils.ts#L40
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Main.ts#L506
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2172
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/BranchProvenanceInitializer.test.ts#L431
• saveLocalValue (setToStandalone)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol
    • Links to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/iModelUtils.ts#L41
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Main.ts#L507
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2173
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/BranchProvenanceInitializer.test.ts#L432
• saveChanges (setToStandalone)
• closeFile (setToStandalone)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol
    • Links to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/iModelUtils.ts#L43
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Main.ts#L512
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2175
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2207
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/BranchProvenanceInitializer.test.ts#L434

MIGHT NEED REPLACEMENT:

• getChangeTrackingMemoryUsed (test-app)
  • Not found in itwinjs-core
  • Link to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Transformer.ts#L428
• completeCreateChangeset (performance-tests)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol
  • Link to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/rawInserts.ts#L199
• startCreateChangeset (performance-tests)
  • Found in IModelNative.platform.DgnDb() which is internal and will soon be inaccessible due to private symbol
  • Link to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/rawInserts.ts#L190
• applyChangeset (performance-tests)
  • Not found in itwinjs-core
  • Link to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/rawInserts.ts#L87
• startProfiler * (performance-scripts, runWithSqliteProfiler)
  • Not found in itwinjs-core
  • Link to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-scripts/runWithSqliteProfiler.ts#L25
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-scripts/runWithSqliteProfiler.ts#L28
• stopProfiler * (performance-scripts, runWithSqliteProfiler)
  • Not found in itwinjs-core
    • Link to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-scripts/runWithSqliteProfiler.ts#L31

MIGHT BE ABLE TO REMOVE:

• extractChangedInstanceIdsFromChangeSets (We could probably remove usage on this, use our own ChangedInstanceIds class)
  • Not found in itwinjs-core
  • Links to usage:
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L1559
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L1638
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L1772
    • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformerHub.test.ts#L2573

Yet to be sorted:

• nativeDb Functions

  • hasPendingTxns

    • Public

  • hasUnsavedChanges

    • Public

  • schemaToXmlString (there could be a workaround here... not sure. the one in core uses DOMParser)

    • Not found. Only found example function in itwinjs-core/example-code/snippets/src/pcf/DynamicSchema.ts
    • Links to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L3584

• IModelHost.platform

  • DgnDb

    • moved to iModelNative which is internal and will soon be inaccessible due to private symbol approach
    • Links to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/performance-tests/test/iModelUtils.ts#L35
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/test-app/src/Main.ts#L501
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/IModelTransformer.test.ts#L2202
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/standalone/BranchProvenanceInitializer.test.ts#L426

  • RevisionUtility.recompressRevision

    • moved to iModelNative which is internal and will soon be inaccessible due to private symbol approach
    • Link to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/TestUtils/RevisionUtility.ts#L60

  • RevisionUtility.disassembleRevision

    • moved to iModelNative which is internal and will soon be inaccessible due to private symbol approach
    • Link to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/TestUtils/RevisionUtility.ts#L72

  • RevisionUtility.assembleRevision

    • moved to iModelNative which is internal and will soon be inaccessible due to private symbol approach
    • Link to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/TestUtils/RevisionUtility.ts#L87

  • RevisionUtility.normalizeLzmaParams

    • moved to iModelNative which is internal and will soon be inaccessible due to private symbol approach
    • Link to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/TestUtils/RevisionUtility.ts#L96

  • RevisionUtility.computeStatistics

    • moved to iModelNative which is internal and will soon be inaccessible due to private symbol approach
    • Link to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/TestUtils/RevisionUtility.ts#L108

  • RevisionUtility.getUncompressSize

    • moved to iModelNative which is internal and will soon be inaccessible due to private symbol approach
    • Link to usage:
      • https://github.com/iTwin/imodel-transformer/blob/7131a96da7146b1c9b414ffc3a3db507f6fa6fea/packages/transformer/src/test/TestUtils/RevisionUtility.ts#L118

*Denotes that the function is not being used exclusively by tests

[aruniverse]

fyi @pmconne @wgoehrig to keep you guys in the loop

[pmconne]

Would it be possible to provide a list of the APIs you are using that currently do not have an alternative public API, with link(s) to your usage of each?

[nick4598]

Thanks for the suggestion, we will work on gathering that information

[nick4598]

Daniel has updated the list with links to their usage. I also tried to give a rough categorization of each one, "must have replacement", "might need replacement".