Improve performance of data tuples import

[geofranzi]

• [x] check where feature permissions are checked (every single row insert?) ..it appears a lot in the DB log (but could be also related to migration process) -> not relevant
• [x] change count to any (https://github.com/BEXIS2/Core/blob/master/Components/DLM/BExIS.Dlm.Services/Data/DatasetManager.cs#L2861)
• [x] Check order in materialized views creation

in the future:

• [x] check if data tuples are sent row by row to DB -> in the log it looks like that. This could be one reason for the long import times during migration/import ~ 1,5ms per Insert during BExIS migration
• [x] check if a bulk import is possible

  • its already sent as block

[geofranzi]

rewrite mat view query: saves 1/3 of the time:

CASE (xpath('/Content/Item[Property[@Name="VariableId" and @value="1"]][1]/Property[@Name="Value"]/@value', t.xmlvariablevalues)::text) WHEN '{""}'::text THEN NULL WHEN '{_null_null}'::text THEN NULL ELSE cast((xpath('/Content/Item[Property[@Name="VariableId" and @value="1"]][1]/Property[@Name="Value"]/@value', t.xmlvariablevalues)::character varying[])[1] AS float8) END AS var1,

WHERE v.datasetref = 1 AND v.status in (0,2)

L160 .AppendLine(string.Format("WHERE v.datasetref = {0} AND v.status in (0,2)", datasetId))

L322 string fieldDef = $"CASE {accessPath}::text WHEN '{{\"\"}}'::text THEN NULL WHEN'{{_null_null}}'::text THEN NULL ELSE cast(({accessPath}::character varying[])[1] {fieldType}) END AS {this.BuildColumnName(Id).ToLower()}";

[geofranzi]

https://github.com/BEXIS2/Core/commit/fe14f578d3ef6733f85d3d2862640dee196a5671

[DavidBlaa]

can we close this issue? the generation of the view seems to work