tomthecarrot / arcore-for-all

Google ARCore (dev preview 1) for "unsupported" Android devices
GNU General Public License v3.0
1.04k stars 145 forks source link

What would google have to do to create a device calibration.. for example for Samsung S8+ so that it too could be supported ? #83

Open janerivi opened 6 years ago

janerivi commented 6 years ago

I would assume there isnt a huge difference in device specs between Samsung S8 and S8+

I bought an S8+ because it had a more powerful processor and someone wrote in an article that ARCore was supported on S8+ (wich was wrong).

I assume it would not take that many hours for an employee at google to setup the device calibrations for S8+ when there already is one for S8. I assume it has something to do with stuff like lens distortion parameters for the lens in S8+ wich may differ slightly from the S8.

Best regards..

hong19860320 commented 6 years ago

Maybe you can try it as following steps: 1) create a configuration file /sdcard/CAD.xml, you can use S8's configuration i dumped from libtango_service_library.so of arcore-preview.apk

<!-- SoC based calibration from 47 Qualcomm S8 Phones-->
<rig>
  <device_id>0</device_id>
  <camera>
    <camera_model type="calibu_fu_fv_u0_v0_k1_k2_k3" index="0">
      <rolling_shutter_readout_time>0.0326218</rolling_shutter_readout_time>
      <rolling_shutter_direction>vertical</rolling_shutter_direction>
      <height>480</height>
      <width>640</width>
      <params> [495.733883807343;  495.6676553008456;  315.4700278866147;  232.2707439173451;  0.1489241495201168;  
-0.4417427288891008;  0.4067056789938272] </params>
    </camera_model>
  </camera>
  <camera>
    <camera_model type="calibu_fu_fv_u0_v0_k1_k2_k3" index="1">
      <rolling_shutter_readout_time>0.0326218</rolling_shutter_readout_time>
      <rolling_shutter_direction>vertical</rolling_shutter_direction>
      <height>1080</height>
      <width>1920</width>
      <params> [1487.199;  1487.001;  946.410;  516.813;   0.1489241495201168;  -0.4417427288891008;  0.4067056789938272] </params>
    </camera_model>
  </camera>
  <camera>
    <camera_model type="calibu_fu_fv_u0_v0_k1_k2_k3" index="3">
      <rolling_shutter_readout_time>0.0326218</rolling_shutter_readout_time>
      <rolling_shutter_direction>vertical</rolling_shutter_direction>
      <height>1080</height>
      <width>1920</width>
      <params> [1487.199;  1487.001;  946.410;  516.813;   0.1489241495201168;  -0.4417427288891008;  0.4067056789938272] </params>
    </camera_model>
  </camera>
  <intrinsic_imu_calibration name="" imu_id="100" type="linear_imu_sg_mg_sa_ma_gqa">
    <b_w_b_a> [0.002358463772260724;  0.001809926290445352;  0.001591088205025581;  -0.02611307672906278;  -0.1105770734858414;  0.1079185441646273] </b_w_b_a>
    <intrinsics> [1.005704361068297;  1.026189925411079;  1.002831637595448;  0.004578144716975105;  -0.001848050570252286;  -0.00296183013855017;  1.005412424479319;  1.005207929570434;  1.003552846073736;  -0.001463114802064685;  0.003828624926299869;  0.006517712743266594;  0.003935646301693438;  -0.0006195812757256409;  0.00100949438688012;  0.9999915538284879] </intrinsics>
    <gyro_noise_sigma>0.00053088444</gyro_noise_sigma>
    <gyro_bias_sigma>0.00014125375</gyro_bias_sigma>
    <accel_noise_sigma>0.0048836490000000003</accel_noise_sigma>
    <accel_bias_sigma>0.012589253999999999</accel_bias_sigma>
  </intrinsic_imu_calibration>
  <extrinsic_calibration frame_A_id="100" frame_B_id="0">
    <A_q_B> [0.9999847053718686;  0.001229914633586842;  -0.00329806225450645;  0.004266042392761234] </A_q_B>
    <A_p_B> [-0.005508152151446929;  0.007537306913866897;  -0.008580830707013921] </A_p_B>
  </extrinsic_calibration>
  <extrinsic_calibration frame_A_id="100" frame_B_id="1">
    <A_q_B> [0.9999847053718686;  0.001229914633586842;  -0.00329806225450645;  0.004266042392761234] </A_q_B>
    <A_p_B> [-0.005508152151446929;  0.007537306913866897;  -0.008580830707013921] </A_p_B>
  </extrinsic_calibration>
  <extrinsic_calibration frame_A_id="100" frame_B_id="3">
    <A_q_B> [0.9999847053718686;  0.001229914633586842;  -0.00329806225450645;  0.004266042392761234] </A_q_B>
    <A_p_B> [-0.005508152151446929;  0.007537306913866897;  -0.008580830707013921] </A_p_B>
  </extrinsic_calibration>
  <extrinsic_calibration name="Device_T_OpenGl" index="0" frame_A_id="40" frame_B_id="42">
    <A_T_B> [0, 1, 0, 0;  -1, 0, 0, 0;  0, 0, 1, 0] </A_T_B>
  </extrinsic_calibration>
  <extrinsic_calibration name="Imu_T_Device" index="0" frame_A_id="100" frame_B_id="40">
    <A_T_B> [0, -1, 0, 0;  1, 0, 0, 0;  0, 0, 1, 0] </A_T_B>
  </extrinsic_calibration>
</rig>

2) set path to android property "ro.config.calibration_cad" after root your device, sothat libtango_service_library.so can find and load it: # setprop ro.config.calibration_cad /sdcard/CAD.xml 3) run ARCore sample code My huawei P9 can run, but can not find any surface. Maybe S8's configuration is not suitable for my device, GOOD LUCK!

kevin-dong-ros commented 6 years ago

Hi, @hong19860320, I have tested you method, but the tango core is still stopped after running ARcore sample. My phone is XIAOMI MI6. Is this problem not happened in your phone?

hong19860320 commented 6 years ago

@kevin-dong-ros Hi, can you provide your log file, I need more details. BTW, for huawei P9, tango core will crash because it can't find the calibration file, just like this:

10-16 13:58:20.056 3919-3950/? W/tango: helpers.cc:418 Defaulting to persistent calibration file.
10-16 13:58:20.056 3919-3950/? W/tango: helpers.cc:395 Property calibration_cad is not defined.
10-16 13:58:20.056 3919-3950/? I/tango: calibration_provider.cc:486 Instantiating calibration provider with sources:
                                            CAD: 
                                            Factory: 
                                            Online: /data/data/com.google.tango/files/config/online-calibration.xml
                                            ActiveFile: kOnlineCalibrationFile
                                            Hardcoded CAD model fallback: Enabled
10-16 13:58:20.058 3919-3919/? A/tango: calibration_provider.cc:589 Check failed: parse_cad_error_code == LoadCalibrationXmlErrorCode::kSuccess 
10-16 13:58:20.058 3919-3950/? E/tango: device_provider.cc:336 Could not identify device type from build fingerprint 'HUAWEI/EVA-AL10/HWEVA:7.0/HUAWEIEVA-AL10/C00B386:user/release-keys'.
10-16 13:58:20.058 3919-3950/? E/tango: helpers.cc:288 Hardcoded CAD model unavailable for device kUnknownDevice
10-16 13:58:20.058 3919-3950/? W/tango: calibration_provider.cc:379 Requested parsing of Online calibration, but failed to parse it and the following fallbacks: 
                                        - Online calibration: File does not exist: /data/data/com.google.tango/files/config/online-calibration.xml.
                                        - Factory calibration: File not set.
                                        - CAD calibration: File not set.
                                        - Hardcoded CAD model: Could not load hardcoded CAD model. It is only available when:
                                          - Explicitly setting the active file to kHardcodedCadModel.
                                          - On device, as a fallback when reading the CAD calibration file fails.
10-16 13:58:20.058 3919-3950/? W/tango: calibration_provider.cc:379 Requested parsing of CAD calibration, but failed to parse it and the following fallbacks: 
                                        - CAD calibration: File not set.
                                        - Hardcoded CAD model: Could not load hardcoded CAD model. It is only available when:
                                          - Explicitly setting the active file to kHardcodedCadModel.
                                          - On device, as a fallback when reading the CAD calibration file fails.

                                        --------- beginning of crash
10-16 13:58:20.058 3919-3919/? A/libc: Fatal signal 6 (SIGABRT), code -6 in tid 3919 (om.google.tango)

                                       [ 10-16 13:58:20.059   364:  364 W/         ]
                                       debuggerd: handling request: pid=3919 uid=10117 gid=10117 tid=3919
10-16 13:58:20.064 1782-2327/? I/PGServer: checkPkgType. calling pkg: com.huawei.iaware
10-16 13:58:20.124 3951-3951/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
10-16 13:58:20.124 3951-3951/? A/DEBUG: Build fingerprint: 'HUAWEI/EVA-AL10/HWEVA:7.0/HUAWEIEVA-AL10/C00B386:user/release-keys'
10-16 13:58:20.124 3951-3951/? A/DEBUG: Revision: '0'
10-16 13:58:20.124 3951-3951/? A/DEBUG: ABI: 'arm64'
10-16 13:58:20.125 3951-3951/? A/DEBUG: pid: 3919, tid: 3919, name: om.google.tango  >>> com.google.tango <<<
10-16 13:58:20.125 3951-3951/? A/DEBUG: signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
10-16 13:58:20.127 3951-3951/? A/DEBUG: Abort message: 'calibration_provider.cc:589 Check failed: parse_cad_error_code == LoadCalibrationXmlErrorCode::kSuccess '
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x0   0000000000000000  x1   0000000000000f4f  x2   0000000000000006  x3   0000000000000008
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x4   0000000000000000  x5   0000000000808080  x6   0000007de6257000  x7   0000000000000000
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x8   0000000000000083  x9   ffffffffffffffdf  x10  0000000000000000  x11  0000000000000001
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x12  ffffffffffffffff  x13  0000000000000000  x14  0000000000000000  x15  0000e048428e72d9
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x16  0000007de56ffec8  x17  0000007de56a9624  x18  0000000000000000  x19  0000007de630ab40
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x20  0000000000000006  x21  0000007de630aa98  x22  0000000000000002  x23  0000007fd0092978
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x24  0000000000000068  x25  0000007de234c198  x26  0000007dd71e3680  x27  0000007de22b6a20
10-16 13:58:20.127 3951-3951/? A/DEBUG:     x28  0000007de22b69f8  x29  0000007fd0092350  x30  0000007de56a6acc
10-16 13:58:20.127 3951-3951/? A/DEBUG:     sp   0000007fd0092330  pc   0000007de56a962c  pstate 0000000060000000
10-16 13:58:20.137 3951-3951/? A/DEBUG: backtrace:
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #00 pc 000000000006b62c  /system/lib64/libc.so (tgkill+8)
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #01 pc 0000000000068ac8  /system/lib64/libc.so (pthread_kill+64)
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #02 pc 0000000000023de8  /system/lib64/libc.so (raise+24)
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #03 pc 000000000001c86c  /system/lib64/libc.so (abort+52)
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #04 pc 0000000000005bf8  /vendor/lib64/liblog.so (__android_log_assert+232)
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #05 pc 0000000000100718  /data/app/com.google.tango-1/lib/arm64/libtango_service_library.so (_ZN13MessageLoggerD2Ev+4120)
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #06 pc 0000000000c8433c  /data/app/com.google.tango-1/lib/arm64/libtango_service_library.so
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #07 pc 0000000000c85be8  /data/app/com.google.tango-1/lib/arm64/libtango_service_library.so
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #08 pc 0000000000129150  /data/app/com.google.tango-1/lib/arm64/libtango_service_library.so (_ZN13tango_service12TangoContextC1ESt10shared_ptrINS_15JniTangoServiceEE+512)
10-16 13:58:20.137 3951-3951/? A/DEBUG:     #09 pc 000000000013c80c  /data/app/com.google.tango-1/lib/arm64/libtango_service_library.so (_ZN12tango_binder7ServiceC2ESt10shared_ptrIN13tango_service15JniTangoServiceEE+228)
10-16 13:58:20.138 3951-3951/? A/DEBUG:     #10 pc 000000000013a238  /data/app/com.google.tango-1/lib/arm64/libtango_service_library.so (Java_com_google_atap_tango_TangoServiceJNINative_CreateNativeService+972)
10-16 13:58:20.138 3951-3951/? A/DEBUG:     #11 pc 00000000007f5eb8  /data/app/com.google.tango-1/oat/arm64/base.odex (offset 0x7a7000)
10-16 13:58:20.210 420-434/? I/gralloc: alloc_device_alloc:564: Alloc handle(0x7d23247300): interfmt=200000001, stride=1088, size=8486912, usage=0xb00
10-16 13:58:20.214 420-3783/? I/gralloc: alloc_device_alloc:564: Alloc handle(0x7d23247700): interfmt=200000001, stride=1088, size=8486912, usage=0xb00
10-16 13:58:20.217 420-3783/? I/gralloc: alloc_device_alloc:564: Alloc handle(0x7d2320f300): interfmt=200000001, stride=1088, size=8486912, usage=0xb00
10-16 13:58:20.220 420-3783/? I/gralloc: alloc_device_alloc:564: Alloc handle(0x7d23293000): interfmt=200000001, stride=1088, size=8486912, usage=0xb00
10-16 13:58:20.223 420-434/? I/gralloc: alloc_device_alloc:564: Alloc handle(0x7d2320f600): interfmt=200000001, stride=1088, size=8486912, usage=0xb00
10-16 13:58:20.235 2449-2449/? W/IInputConnectionWrapper: reportFullscreenMode on inexistent InputConnection
kevin-dong-ros commented 6 years ago

Hi, @hong19860320, log information is provided as follow: screenshot from 2017-10-16 16-00-12

And I have done

setprop ro.config.calibration_cad /storage/emulated/0/CAD.xml

hong19860320 commented 6 years ago

@kevin-dong-ros try this: CAD.zip

kevin-dong-ros commented 6 years ago

@hong19860320, I have linked this CAD.xml file to ro.config.calibration_cad and tango-core still crashed. Can you share the log from your phone which tango-core is not crashed?

hong19860320 commented 6 years ago

@kevin-dong-ros check tangcore's permissions, and make sure it can access your calibration file. huaweip9_load_cad_file_ok.zip

kevin-dong-ros commented 6 years ago

@hong19860320, my sample app can access calibration file after giving storage permissions. But it is already crashed because of no valid camera index being detected. screenshot from 2017-10-17 13-31-27 I think it is happened because of the camera index and property is not similar between google pixel/HUAWEI P9 to XIAOMI MI6.