search

labpano / pilot-open-api

Open API for Pilot cameras
MIT License
11 stars 3 forks source link

readme

中文

Introduction

Pilot OS is the operating system of Pilot series of panoramic cameras. It is customized based on the Android system (currently Android 7.0 version). The way of developing applications on Pilot OS is almost the same as that of general Android application development.

This program mainly provides the functions of the Camera part, and also provides a streaming library. It is integrated into the App running on Pilot camera and can be used for lens preview, taking photos, recording videos, live streaming, etc.

Integration

Initialization (Load Preview)

Initialization will also display the preview into the view container:

// Create PilotSDK and specify the parent container to place the preview view
mPilotSDK = PilotSDK(findViewById(R.id.vg_preview), object : PanoSDKListener {
            override fun onPanoCreate() {
                // After the creation is successful, you can further adjust the preview parameters
            }

            override fun onPanoRelease() {
            }

            override fun onChangePreviewMode(mode: Int) {
            }

            override fun onSingleTap() {
            }

            override fun onEncodeFrame(count: Int) {
            }
        })

After the initialization is successful, you can set the preview resolution&fps, preview effect, etc. Such as setting the resolution&fps：

// Use PilotSDK internal method adjustment
PilotSDK.changeCameraResolution(
        PilotSDK.CAMERA_PREVIEW_4048_2530_15 // the resolution you need, it includes the frame rate
        , object : ChangeResolutionListener() {
            override fun onChangeResolution(width: Int, height: Int) {
                // resolution change
            }
        })

Currently supported preview resolution&fps include:

Take Photos

After the preview is loaded, you can take a picture.

Photo resolution and preview used resolution&fps ：

Reference sample photoSample

Record videos

After the preview is loaded, you can record.

Video resolution and preview used resolution&fps:

Reference sample videoSample

Live Streaming

After the preview is loaded, the previewed data can be obtained for live broadcast.
We provide 4K and 8K RTMP streaming libraries for live streaming.

4K Streaming

Most of the media servers currently support 4K, and support the RTMP protocol.
The provided 4K streaming library can push streaming at resolutions of 4K and below, and can be received by common platforms such as Facebook and YouTube.

  1. Create Pusher and set parameters

    mPusher = PusherFactory.createPusher()
// Set video parameters, including resolution, fps, bit rate
mPusher.addVideoTrackInfo(
IPusher.ENCODER_ID_H264, 3840, 1920, 24, 12000000
)
// Set audio parameters
mPusher.addAudioTrackInfo(
IPusher.AUDIO_SOURCE_ID_MIC, IPusher.ENCODER_ID_AAC,
48000, 2, 16, 128000
)
// Set push address
mPusher.addOutput(
IPusher.OUTPUT_TYPE_RTMP, pushURL, 5000 //5 seconds timeout
)

  2. Binding Suerface

    mPusher.setEncodeVideoListerner(object : IPusherEncodeVideoListerner {
      var init = false;
      override fun updateSurface(streamName: String, surface: Surface): Int {
          if (!init) {
             // Set the created Suerface to PilotSDK, and the preview screen will be drawn on this Surface at the same time. This step is very important
              PilotSDK.setEncodeInputSurfaceForLive(surface)
              init = true
          }
          return 0
      }
  })

  3. Start streaming

    mPusher.start(object : IPusherStateListerner {
        override fun notifyPusherState(
            state: IPusherStateListerner.PusherState, errorCode: Int, message: String
        ) {
          // Status information during the push
        }
    })

  4. Stop streaming

    // Cancel the surface drawn in the preview screen live
PilotSDK.setEncodeInputSurfaceForLive(null)
// Stop
mPusher.stop()

4K live streaming resolution and preview used resolution&fps:

Reference sample liveSample

PilotLive 8K Streaming

  1. Create StreamSender

    mStreamSender = StreamSender()

  2. Start streaming

    mStreamSender.start(
        PilotSDK.getCameras(), // Open Camera, the number is 4
        false, // Can save bandwidth when h265 is used
        2, // channel number
        3648,
        2280,
        24,
        15000000, // bitrate
        pushURL,
        null,
        0f,
        ""
    )

  3. Stop streaming

    mStreamSender.stop(false)

PilotLive 8K preview resolution&fps used for streaming:

Reference sample live8kSample

PRO Adjustment

After the preview is loaded, the preview effect can be further adjusted.

The following values can be adjusted when taking pictures: exposure time, ISO, EV (when exposure time is manual), WB, stitching distance. The following values can be adjusted when recording: ISO, EV, stitching distance.

Play Media File

Add com.pi.pano.MediaPlayerSurfaceView view, and use
com.pi.pano.MediaPlayerSurfaceView#open(String filename)
Open local file for playback.

Other

  1. AutoPowerOff service pause/resume (Only available on Pilot OS v5.14.0+)

    • pause： com.pi.ext.PiOtherHelper#pauseAutoPowerOffService(Context context)
    • resume: com.pi.ext.PiOtherHelper#resumeAutoPowerOffService(Context context)

  2. The com.pi.pano.helper package provides a relatively simple help class for reference.

Troubleshooting Q&A

  1. The preview is all black.
    When improper use of camera equipment or improper use environment may cause abnormal equipment, the camera device can be restarted.

  2. The definition of live broadcast is too low.
    Live broadcasting has high requirements on the network. Confirm whether the clarity and bit rate of the current network bandwidth used for live broadcasting is feasible. Avoid low bandwidth or long delay.

  3. Using the provided 4K streaming library to push h.265 media server is not supported.
    At present, RTMP live only supports H.264, and the 4K and 8K streaming libraries provided can push h.265, but it needs media server and player for processing.

  4. The preview temperature is too high when the device is turned on for a long time.
    Normal phenomenon, use preview device to consume performance heat, the device contains fan and other cooling components, the fan can be automatically turned on in the setting.

  5. Can the integrated SDK be used on non Pilot OS systems?
    Don't do that.

License

License

Samples

Sample Description
photoSample download Apk Reference for take a photo process
videoSample download Apk Reference for record video
liveSample download Apk Reference for RTMP live broadcast 4K and below resolution
live8kSample download Apk Reference for RTMP live broadcast PilotLive 8K resolution