mrcfschr
commented
2 years ago For anyone still looking for something like this, Microsoft research's platform for situated intelligence(PSI) does this for you.
https://github.com/microsoft/psi.
I summarized some documentation here (https://github.com/immersLAB/psi) on how to visualize the HL2 sensor streams using psi.
Running the HoloLensCapture as described there creates a calibration folder in the documents folder of your HL with files containing intrinsic parameters and distortion coefficients etc for the different streams.
Using azure Kinect, one can obtain a calibration.json file from a recording (mkv format) that looks like this:
calibration.json
{"CalibrationInformation":{"Cameras":[{"Intrinsics":{"ModelParameterCount":14,"ModelParameters":[0.50949156284332275,0.50468581914901733,0.49236810207366943,0.49235457181930542,2.6508913040161133,1.8468171358108521,0.1003890261054039,2.9833972454071045,2.7026257514953613,0.51992595195770264,0,0,2.5446879590163007E-5,4.0959363104775548E-5],"ModelType":"CALIBRATION_LensDistortionModelBrownConrady"},"Location":"CALIBRATION_CameraLocationD0","Purpose":"CALIBRATION_CameraPurposeDepth","MetricRadius":1.7399997711181641,"Rt":{"Rotation":[1,0,0,0,1,0,0,0,1],"Translation":[0,0,0]},"SensorHeight":1024,"SensorWidth":1024,"Shutter":"CALIBRATION_ShutterTypeUndefined","ThermalAdjustmentParams":{"Params":[0,0,0,0,0,0,0,0,0,0,0,0]}},{"Intrinsics":{"ModelParameterCount":14,"ModelParameters":[0.49882322549819946,0.50667357444763184,0.47840622067451477,0.63765299320220947,0.61587780714035034,-2.6588020324707031,1.4567103385925293,0.49323645234107971,-2.4888460636138916,1.3905943632125854,0,0,-0.00057023687986657023,0.00099901377689093351],"ModelType":"CALIBRATION_LensDistortionModelBrownConrady"},"Location":"CALIBRATION_CameraLocationPV0","Purpose":"CALIBRATION_CameraPurposePhotoVideo","MetricRadius":0,"Rt":{"Rotation":[0.99999845027923584,-0.0016666973242536187,-0.00056906969984993339,0.0017137537943199277,0.99535059928894043,0.096302621066570282,0.00040591656579636037,-0.096303448081016541,0.99535191059112549],"Translation":[-0.032005276530981064,-0.0020297770388424397,0.00405590096488595]},"SensorHeight":3072,"SensorWidth":4096,"Shutter":"CALIBRATION_ShutterTypeUndefined","ThermalAdjustmentParams":{"Params":[0,0,0,0,0,0,0,0,0,0,0,0]}}],"InertialSensors":[{"BiasTemperatureModel":[-0.0095095150172710419,0,0,0,0.021830782294273376,0,0,0,0.010339461266994476,0,0,0],"BiasUncertainty":[9.9999997473787516E-5,9.9999997473787516E-5,9.9999997473787516E-5],"Id":"CALIBRATION_InertialSensorId_LSM6DSM","MixingMatrixTemperatureModel":[0.98848807811737061,0,0,0,-0.0012132143601775169,0,0,0,0.008040926419198513,0,0,0,-0.001189512200653553,0,0,0,1.0081484317779541,0,0,0,0.0054368264973163605,0,0,0,0.0080261304974555969,0,0,0,0.0055349566973745823,0,0,0,0.99033874273300171,0,0,0],"ModelTypeMask":16,"Noise":[0.00095000001601874828,0.00095000001601874828,0.00095000001601874828,0,0,0],"Rt":{"Rotation":[-0.0016056329477578402,0.11567381024360657,-0.99328595399856567,-0.99999845027923584,0.000562333210837096,0.0016819704324007034,0.00075311760883778334,0.99328708648681641,0.11567272245883942],"Translation":[0,0,0]},"SecondOrderScaling":[0,0,0,0,0,0,0,0,0],"SensorType":"CALIBRATION_InertialSensorType_Gyro","TemperatureBounds":[5,60],"TemperatureC":0},{"BiasTemperatureModel":[-0.075560465455055237,0,0,0,-0.16636647284030914,0,0,0,0.047918349504470825,0,0,0],"BiasUncertainty":[0.0099999997764825821,0.0099999997764825821,0.0099999997764825821],"Id":"CALIBRATION_InertialSensorId_LSM6DSM","MixingMatrixTemperatureModel":[0.99965709447860718,0,0,0,0.00010260543058393523,0,0,0,-0.0040383865125477314,0,0,0,0.00010435239528305829,0,0,0,0.98290562629699707,0,0,0,-0.00033697500475682318,0,0,0,-0.0040774843655526638,0,0,0,-0.00033454151707701385,0,0,0,0.99007177352905273,0,0,0],"ModelTypeMask":56,"Noise":[0.010700000450015068,0.010700000450015068,0.010700000450015068,0,0,0],"Rt":{"Rotation":[-0.00342939468100667,0.1078546941280365,-0.99416077136993408,-0.999993085861206,-0.0017843873938545585,0.0032559288665652275,-0.001422800705768168,0.99416506290435791,0.10786006599664688],"Translation":[-0.0508907213807106,0.0038949707522988319,0.0017785278614610434]},"SecondOrderScaling":[0,0,0,0,0,0,0,0,0],"SensorType":"CALIBRATION_InertialSensorType_Accelerometer","TemperatureBounds":[5,60],"TemperatureC":0}],"Metadata":{"SerialId":"000659100112","FactoryCalDate":"1/1/2020 9:45:29 AM GMT","Version":{"Major":1,"Minor":2},"DeviceName":"AzureKinect-PV","Notes":"PV0_max_radius_invalid"}}}
Is there a way to obtain a calibration file like this from HoloLens 2? If not I think it would be a cool feature to add in the future? Thank you! :)