RealEstateCore / rec

RealEstateCore ontologies.
http://realestatecore.io
BSD 3-Clause "New" or "Revised" License
84 stars 37 forks source link

Difference between uploading DTDL models from ADT Explorer and Import Jobs #237

Closed ogiel closed 1 year ago

ogiel commented 1 year ago

Hi all,

This issue might be related to my just posted issue about model behavior.

I have transformed the RealEstateCore.DTDLv2.jsonld file in this repo into a jsonld import jobs file for the new ADT import jobs API. The strange thing is, when we try to upload these models through the ADT Explorer it results in a lot of errors (actual erros below) and only a small subset of models being uploaded.

If we run an import job with the same models it works fine and it adds a all the models to our ADT instance. These are the exact same models.

We even verified this behavior by uploading models through import jobs, then download all these models from the ADT instance and then reupload the downloaded files. This results in the same error as doing it by hand from the jsonld (as json) file from this repo.

RestError: None of the models in this request could be created due to a problem with one or more models: DtmiResolverAsync failed to resolve requisite references to element(s): dtmi:org:brickschema:schema:Brick:Direction_Status;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:System_Status;1 (referenced in 3 places), dtmi:org:brickschema:schema:Brick:Enable_Status;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Mode_Status;1 (referenced in 2 places), dtmi:org:brickschema:schema:Brick:Speed_Status;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Status;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Overridden_Status;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Off_Status;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Filter_Status;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Pressure_Status;1 (referenced in 2 places), dtmi:org:brickschema:schema:Brick:CO2_Setpoint;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Deadband_Setpoint;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Speed_Setpoint;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Humidity_Setpoint;1 (referenced in 11 places), dtmi:org:brickschema:schema:Brick:Pressure_Setpoint;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Differential_Setpoint;1 (referenced in 2 places), dtmi:org:brickschema:schema:Brick:Time_Setpoint;1 (referenced in 2 places), dtmi:org:brickschema:schema:Brick:Demand_Setpoint;1 (referenced in 3 places), dtmi:org:brickschema:schema:Brick:Temperature_Setpoint;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Load_Parameter;1 (referenced in 2 places), dtmi:org:brickschema:schema:Brick:Delay_Parameter;1 (referenced in 1 place), dtmi:org:brickschema:schema:Brick:Alarm_Sensitivity_Parameter;1 (referenced in 2 places), dtmi:org:brickschema:schema:Brick:Limit;1 (referenced in 3 places), dtmi:org:brickschema:schema:Brick:Tolerance_Parameter;1 (referenced in 2 places), dtmi:org:brickschema:schema:Brick:Humidity_Parameter;1 (referenced in 3 places), dtmi:org:brickschema:schema:Brick:Temperature_Parameter;1 (referenced in 4 places), dtmi:org:brickschema:schema:Brick:Usage_Sensor;1 (referenced in 2 places). See model documentation(https://aka.ms/ADTv2Models) for supported format. at new t (https://explorer.digitaltwins.azure.net/static/js/2.c69de75d.chunk.js:2:14584145) at https://explorer.digitaltwins.azure.net/static/js/2.c69de75d.chunk.js:2:14589431 at https://explorer.digitaltwins.azure.net/static/js/2.c69de75d.chunk.js:2:14590116

cbupp commented 1 year ago

@ogiel - I believe this is do the ADT Model API limitations.

I run into a similar problem when running the UploadModels tool from the Azure team.

I'm forced to use the -b (batch) option to limit the number of synchronous model uploads to prevent dependent models from being uploaded in the same batch.

My suspicion is that batched models are getting split into async threads and a dependent model is uploaded before it's dependency.

ogiel commented 1 year ago

If anyone might reference this issue in the future; Upload the models with the tools and use the batch option -b as explained by @cbupp or transform REC into an ndjson file and use the import jobs API for uploading models.

Refer to #236 for additional information.