google / filament

Filament is a real-time physically based rendering engine for Android, iOS, Windows, Linux, macOS, and WebGL2
https://google.github.io/filament/
Apache License 2.0
17.79k stars 1.89k forks source link

Sample app demonstrating Vuforia/Filament integration #2821

Closed qmz13579 closed 4 years ago

qmz13579 commented 4 years ago

A basic sample application that demonstrates the direct integration of Vuforia/Filament

I think of a way to integrate three projects into a new project ImageTargets(Vuforia\Samples) filament-main(filament\android\samples) sceneform-android(sceneform\samples)

How to solve Vuforia camera data output to Filament background

Thank you. It took three days to solve this problem

if (mSurfaceView != null &&  mSurfaceView.getWidth() > 0 && mSurfaceView.getHeight() > 0) {
  if (!cameraStream.isTextureInitialized()) {
    cameraStream.initializeTexture(null);
  }
  if (m_SenderHandler == null) {
    m_SenderThread = new HandlerThread("ArSendThread");
    m_SenderThread.start();
    m_SenderHandler = new Handler(m_SenderThread.getLooper());
  }
  final Stream stream = cameraStream.getCameraTexture().getFilamentStream();
  stream.setAcquiredImage(mSurfaceView, new Handler(), null);
  final Bitmap outBitmap = Bitmap.createBitmap(mSurfaceView.getWidth(), mSurfaceView.getHeight(), Bitmap.Config.ARGB_8888);
  if (stream != null && outBitmap != null) {
    PixelCopy.request(mSurfaceView, outBitmap, new PixelCopy.OnPixelCopyFinishedListener() {
      @Override
      public void onPixelCopyFinished(int copyResult) {
        if (copyResult == PixelCopy.SUCCESS) {
          int size = outBitmap.getRowBytes() * outBitmap.getHeight();
          ByteBuffer byteBuffer = ByteBuffer.allocate(size);
          outBitmap.copyPixelsToBuffer(byteBuffer);
          byte[] data = byteBuffer.array();
          stream.setAcquiredImage(data, new Handler(), null);
        }
      }
    }, m_SenderHandler);
  }
}

The background of Filament is still black, I don’t know which is wrong, I don’t know about Filament’s Stream

Vuforia's SDK camera output is GLSurfaceView(OpenGL ES) Above Vuforia, below Filament (black background)

3a4c1ed563686b789ad112dd46c161b

romainguy commented 4 years ago

setAcquiredImage is supposed to receive a HardwareBuffer, not a byte[]. What you are doing is incredibly inefficient, you should instead have Vuforia render into a SurfaceTexture that you could share directly with Filament (see Stream.Builder.stream()).