Closed sourabhgupta811 closed 4 years ago
This is how TouchAndDragRenderer
is expected to work.
To change this behavior, call Geometry3D.getBoundingSphere()
then use Intersector.intersectRaySphere()
to get the hitPoint
of the touch and add that offset into the mNewObjPos
GLU.gluUnProject(x, renderer.getViewportHeight() - y, 0, mViewMatrix.getDoubleValues(), 0,
mProjectionMatrix.getDoubleValues(), 0, mViewport, 0, mNearPos4, 0);
GLU.gluUnProject(x, renderer.getViewportHeight() - y, 1.f, mViewMatrix.getDoubleValues(), 0,
mProjectionMatrix.getDoubleValues(), 0, mViewport, 0, mFarPos4, 0);
mNearPos.setAll(mNearPos4[0] / mNearPos4[3], mNearPos4[1]
/ mNearPos4[3], mNearPos4[2] / mNearPos4[3]);
mFarPos.setAll(mFarPos4[0] / mFarPos4[3],
mFarPos4[1] / mFarPos4[3], mFarPos4[2] / mFarPos4[3]);
double factor = (Math.abs(mSelectedObject.getZ()) + mNearPos.z)
/ (renderer.getCurrentCamera().getFarPlane() - renderer.getCurrentCamera()
.getNearPlane());
BoundingSphere boundingSphere = mSelectedObject.getGeometry().getBoundingSphere();
Vector3 mHitPoint = new Vector3();
Intersector.intersectRaySphere(mNearPos, mFarPos, boundingSphere.getPosition(), boundingSphere.getRadius(), mHitPoint);
mNewObjPos.setAll(mFarPos);
mNewObjPos.subtract(mNearPos);
mNewObjPos.multiply(factor);
mNewObjPos.add(mNearPos);
mNewObjPos.add(mHitPoint);
mSelectedObject.setX(mNewObjPos.x);
mSelectedObject.setY(mNewObjPos.y);
@contriteobserver This is what i am doing now. But it is not working. Also, I want to know if the RayPicker is now working.
Hi @contriteobserver Thanks for the help. I got it working by directly computing the offset. But there is an issue that ObjectColorPicker is not working with PointShell primitive.
Not sure this is an actual issue with PointShell
, as any object using the GLES20.GL_POINTS
drawing mode does not draw the face colors that ObjectColorPicker
depends upon.
Please clarify what your needs are and we can try to get you there.
I think it might be better to show you, you see how i am picking and interacting with objects in first part of gif. I need to do the same with PointShells. I have changed the PointsOrbitalPlugin to make translation happen which is not perfect as of now. But more importantly, since ObjectColorPicker won't work as you said, i am having a hard time figuring out how to detect if i am touching a PointShell. In the gif, I have hardcoded the PointShell that will be moved when i swipe. But it does not work if i want to have multiple interactive PointShells.
Two options, in both cases we parent a PointShell
to an invisible object:
parent the PointShell
to a physics object
Rajawali is a rendering engine, for physics interactions between objects, (objects colliding and bouncing off each other) a physics engine like JBox2D
is also required, see our JBox2D dynamics tutorial.
Also uploaded a proof of concept repo showing how to integrate this rendering engine with the Dyn4J physics engine.
parent the PointShell
to an invisible object
Material noMaterial = new Material();
noMaterial.setColor(Color.TRANSPARENT);
Object3D obj = new Sphere(1.5f,6,3);
obj.setPickingColor(Color.YELLOW);
obj.setMaterial(noMaterial);
obj.setTransparent(true);
picker.registerObject(obj);
getCurrentScene().addChild(obj);
ptMaterial = new Material();
ptMaterial.enableTime(true);
ptMaterial.useVertexColors(true);
ptMaterial.addPlugin(new PointApertureMaterialPlugin(8));
Object3D points = new PointShell(1024,2);
points.setTransparent(true);
points.setBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE);
points.setDrawingMode(GLES20.GL_POINTS);
points.setMaterial(ptMaterial);
obj.addChild(points);
I parented the PointShell with an invisible object but the objectPicker is still not picking it up.
points1 = new PointShell(10000,0.2f);
points1.setTransparent(true);
points1.setBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE);
points1.setDrawingMode(GLES20.GL_POINTS);
points1.setMaterial(agMaterial);
Material no1Material = new Material();
no1Material.setColor(Color.TRANSPARENT);
point1Prent = new Sphere(1.5f,6,3);
point1Prent.setPickingColor(Color.YELLOW);
point1Prent.setMaterial(no1Material);
point1Prent.setTransparent(true);
renderer.getCurrentScene().addChild(point1Prent);
The following seems to be working, I must be missing something, please clarify:
@Override
protected void initScene() {
mViewport = new int[] { 0, 0, getViewportWidth(), getViewportHeight() };
mNearPos4 = new double[4];
mFarPos4 = new double[4];
mNearPos = new Vector3();
mFarPos = new Vector3();
mNewObjPos = new Vector3();
mViewMatrix = getCurrentCamera().getViewMatrix();
mProjectionMatrix = getCurrentCamera().getProjectionMatrix();
mPicker = new ObjectColorPicker(this);
mPicker.setOnObjectPickedListener(this);
try {
getCurrentScene().setBackgroundColor(Color.BLUE);
DirectionalLight light= new DirectionalLight(-1, 0, -1);
light.setPower(1.5f);
getCurrentScene().addLight(light);
getCurrentCamera().setPosition(0, 0, 4);
Material noMaterial = new Material();
noMaterial.setColor(Color.TRANSPARENT);
Material ptMaterial = new Material();
ptMaterial.enableTime(true);
ptMaterial.useVertexColors(true);
ptMaterial.addPlugin(new PointApertureMaterialPlugin(8));
for (int i = 0; i < 7; i++) {
PointShell shell = new PointShell(1024,1);
shell.setTransparent(true);
shell.setBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE);
shell.setDrawingMode(GLES20.GL_POINTS);
shell.setMaterial(ptMaterial);
Sphere sphere = new Sphere(1f, 12, 12);
sphere.setTransparent(true);
sphere.addChild(shell);
sphere.setMaterial(noMaterial);
sphere.setPickingColor(Color.YELLOW);
sphere.setX(-4 + (Math.random() * 8));
sphere.setY(-4 + (Math.random() * 8));
sphere.setZ(-2 + (Math.random() * -6));
sphere.setDrawingMode(GLES20.GL_TRIANGLES);
mPicker.registerObject(sphere);
getCurrentScene().addChild(sphere);
}
} catch (Exception e) {
e.printStackTrace();
}
}
public void onRenderSurfaceSizeChanged(GL10 gl, int width, int height) {
super.onRenderSurfaceSizeChanged(gl, width, height);
mViewport[2] = getViewportWidth();
mViewport[3] = getViewportHeight();
mViewMatrix = getCurrentCamera().getViewMatrix();
mProjectionMatrix = getCurrentCamera().getProjectionMatrix();
}
@Override
public void onOffsetsChanged(float xOffset, float yOffset, float xOffsetStep, float yOffsetStep, int xPixelOffset, int yPixelOffset) {
}
@Override
public void onTouchEvent(MotionEvent event) {
switch (event.getAction()) {
case MotionEvent.ACTION_DOWN:
getObjectAt(event.getX(), event.getY());
break;
case MotionEvent.ACTION_MOVE:
moveSelectedObject(event.getX(), event.getY());
break;
case MotionEvent.ACTION_UP:
stopMovingSelectedObject();
break;
}
}
public void getObjectAt(float x, float y) {
mPicker.getObjectAt(x, y);
}
public void onObjectPicked(@NonNull Object3D object) {
mSelectedObject = object;
}
@Override
public void onNoObjectPicked() {
Log.w(getLocalClassName(), "No object picked!");
}
public void moveSelectedObject(float x, float y) {
if (mSelectedObject == null)
return;
//
// -- unproject the screen coordinate (2D) to the camera's near plane
//
GLU.gluUnProject(x, getViewportHeight() - y, 0, mViewMatrix.getDoubleValues(), 0,
mProjectionMatrix.getDoubleValues(), 0, mViewport, 0, mNearPos4, 0);
//
// -- unproject the screen coordinate (2D) to the camera's far plane
//
GLU.gluUnProject(x, getViewportHeight() - y, 1.f, mViewMatrix.getDoubleValues(), 0,
mProjectionMatrix.getDoubleValues(), 0, mViewport, 0, mFarPos4, 0);
//
// -- transform 4D coordinates (x, y, z, w) to 3D (x, y, z) by dividing
// each coordinate (x, y, z) by w.
//
mNearPos.setAll(mNearPos4[0] / mNearPos4[3], mNearPos4[1]
/ mNearPos4[3], mNearPos4[2] / mNearPos4[3]);
mFarPos.setAll(mFarPos4[0] / mFarPos4[3],
mFarPos4[1] / mFarPos4[3], mFarPos4[2] / mFarPos4[3]);
//
// -- now get the coordinates for the selected object
//
double factor = (Math.abs(mSelectedObject.getZ()) + mNearPos.z)
/ (getCurrentCamera().getFarPlane() - getCurrentCamera()
.getNearPlane());
mNewObjPos.setAll(mFarPos);
mNewObjPos.subtract(mNearPos);
mNewObjPos.multiply(factor);
mNewObjPos.add(mNearPos);
mSelectedObject.setX(mNewObjPos.x);
mSelectedObject.setY(mNewObjPos.y);
}
public void stopMovingSelectedObject() {
mSelectedObject = null;
}
}
predicated on the assumption that View.OnTouchListener().onTouch
requests follow-on 'MotionEvent' by returning true
:
view.setOnTouchListener(new View.OnTouchListener() {
@Override
public boolean onTouch(View view, MotionEvent motionEvent) {
renderer.onTouchEvent(motionEvent);
return true;
}
});
I copy pasted the exact same code and it looks like this
And the touch is not working. I am running it on Redmi Note 8 Pro device. This is the exact code that i used-
surfaceView.setOnTouchListener(new View.OnTouchListener() {
@Override
public boolean onTouch(View v, MotionEvent event) {
touchRenderer.onTouchEvent(event);
return true;
}
});
private class TouchRenderer extends Renderer implements OnObjectPickedListener {
Object3D mSelectedObject;
ObjectColorPicker mPicker = null;
int[] mViewport;
double[] mNearPos4 = null;
double[] mFarPos4 = null;
Vector3 mNearPos = null;
Vector3 mFarPos = null;
Vector3 mNewObjPos = null;
Matrix4 mViewMatrix = null;
Matrix4 mProjectionMatrix = null;
public TouchRenderer(Context context) {
super(context);
}
@Override
protected void initScene() {
mViewport = new int[] { 0, 0, getViewportWidth(), getViewportHeight() };
mNearPos4 = new double[4];
mFarPos4 = new double[4];
mNearPos = new Vector3();
mFarPos = new Vector3();
mNewObjPos = new Vector3();
mViewMatrix = getCurrentCamera().getViewMatrix();
mProjectionMatrix = getCurrentCamera().getProjectionMatrix();
mPicker = new ObjectColorPicker(this);
mPicker.setOnObjectPickedListener(this);
try {
getCurrentScene().setBackgroundColor(Color.BLUE);
DirectionalLight light= new DirectionalLight(-1, 0, -1);
light.setPower(1.5f);
getCurrentScene().addLight(light);
getCurrentCamera().setPosition(0, 0, 4);
Material noMaterial = new Material();
noMaterial.setColor(Color.TRANSPARENT);
Material ptMaterial = new Material();
ptMaterial.enableTime(true);
ptMaterial.useVertexColors(true);
ptMaterial.addPlugin(new PointApertureMaterialPlugin(8));
for (int i = 0; i < 7; i++) {
PointShell shell = new PointShell(1024,1);
shell.setTransparent(true);
shell.setBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE);
shell.setDrawingMode(GLES20.GL_POINTS);
shell.setMaterial(ptMaterial);
Sphere sphere = new Sphere(1f, 12, 12);
sphere.addChild(shell);
sphere.setMaterial(noMaterial);
sphere.setPickingColor(Color.YELLOW);
sphere.setX(-4 + (Math.random() * 8));
sphere.setY(-4 + (Math.random() * 8));
sphere.setZ(-2 + (Math.random() * -6));
sphere.setDrawingMode(GLES20.GL_TRIANGLES);
mPicker.registerObject(sphere);
getCurrentScene().addChild(sphere);
}
} catch (Exception e) {
e.printStackTrace();
}
}
public void onRenderSurfaceSizeChanged(GL10 gl, int width, int height) {
super.onRenderSurfaceSizeChanged(gl, width, height);
mViewport[2] = getViewportWidth();
mViewport[3] = getViewportHeight();
mViewMatrix = getCurrentCamera().getViewMatrix();
mProjectionMatrix = getCurrentCamera().getProjectionMatrix();
}
@Override
public void onOffsetsChanged(float xOffset, float yOffset, float xOffsetStep, float yOffsetStep, int xPixelOffset, int yPixelOffset) {
}
@Override
public void onTouchEvent(MotionEvent event) {
switch (event.getAction()) {
case MotionEvent.ACTION_DOWN:
getObjectAt(event.getX(), event.getY());
break;
case MotionEvent.ACTION_MOVE:
moveSelectedObject(event.getX(), event.getY());
break;
case MotionEvent.ACTION_UP:
stopMovingSelectedObject();
break;
}
}
public void getObjectAt(float x, float y) {
mPicker.getObjectAt(x, y);
}
public void onObjectPicked(@NonNull Object3D object) {
mSelectedObject = object;
}
@Override
public void onNoObjectPicked() {
Log.w(getLocalClassName(), "No object picked!");
}
public void moveSelectedObject(float x, float y) {
if (mSelectedObject == null)
return;
//
// -- unproject the screen coordinate (2D) to the camera's near plane
//
GLU.gluUnProject(x, getViewportHeight() - y, 0, mViewMatrix.getDoubleValues(), 0,
mProjectionMatrix.getDoubleValues(), 0, mViewport, 0, mNearPos4, 0);
//
// -- unproject the screen coordinate (2D) to the camera's far plane
//
GLU.gluUnProject(x, getViewportHeight() - y, 1.f, mViewMatrix.getDoubleValues(), 0,
mProjectionMatrix.getDoubleValues(), 0, mViewport, 0, mFarPos4, 0);
//
// -- transform 4D coordinates (x, y, z, w) to 3D (x, y, z) by dividing
// each coordinate (x, y, z) by w.
//
mNearPos.setAll(mNearPos4[0] / mNearPos4[3], mNearPos4[1]
/ mNearPos4[3], mNearPos4[2] / mNearPos4[3]);
mFarPos.setAll(mFarPos4[0] / mFarPos4[3],
mFarPos4[1] / mFarPos4[3], mFarPos4[2] / mFarPos4[3]);
//
// -- now get the coordinates for the selected object
//
double factor = (Math.abs(mSelectedObject.getZ()) + mNearPos.z)
/ (getCurrentCamera().getFarPlane() - getCurrentCamera()
.getNearPlane());
mNewObjPos.setAll(mFarPos);
mNewObjPos.subtract(mNearPos);
mNewObjPos.multiply(factor);
mNewObjPos.add(mNearPos);
mSelectedObject.setX(mNewObjPos.x);
mSelectedObject.setY(mNewObjPos.y);
}
public void stopMovingSelectedObject() {
mSelectedObject = null;
}
}
}
sorry, I did somehow miss a line in the cut-and-paste, now fixed in the example:
sphere.setTransparent(true);
But otherwise, the 'TouchRenderer' code works fine on my devices, unfortunately do not have a Redmi Note 8 Pro
handy, this might be a device specific thing, could you try on another device to confirm?
I tested it on Redmi note 4, Redmi note 5, OnePlus 7T. It is not working on any of these.
try perhaps making the parent and child slightly different sizes, say 0.999f
and 1.001f
.
It still does not work this way. I am trying everything but it keeps giving an rgba value of 255,255,255,255 in ObjectColorPicker and thus no object is picked. Edit- I just implemented it via RayPicker class from the Project. And it is working fine that way. You have mentioned in other issues that RayPicker is broken currently. For me, I am not finding any issue with it. Can you please tell me if it is still broken or not.
RayPicker
fixes in master, but these changes are not in a release yet.Okay Thanks a lot @contriteobserver I tested RayPicker on 3d models, PointShells and 2d Planes and it is working good. Thanks for helping.
Hi Sorry for bothering you on an old Thread @contriteobserver but I've been struggling with something related to this for several days now, I'm trying to get the world position of the hitPoint where the Touch occurred over the object, I managed to gat RayPicker working and it detects the touch, but I think there is something wrong with my Intersector.intersectRaySphere
, how I calculate mNearPos
, mFarPos
, or something in that method because I have a Radius 1.0 Sphere and I'm getting either 0,0,0 or big strange values as a result, I also tried to supply GLU.gluUnProject
with x
y
values in -1 to 1 range after calling screenToCartesian(x, y)
but it still doesn't work, could you maybe take a look and help me find the issue?
class TestRenderer3(context: Context?) : Renderer(context), OnObjectPickedListener {
var mSelectedObject: Object3D? = null
var mPicker: RayPicker? = null
lateinit var mViewport : IntArray
var mNearPos4: DoubleArray? = null
var mFarPos4: DoubleArray? = null
var mNearPos: Vector3? = null
var mFarPos: Vector3? = null
var mNewObjPos: Vector3? = null
var mViewMatrix: Matrix4? = null
var mProjectionMatrix: Matrix4? = null
private var mGestureListener: View.OnTouchListener? = null
//object to be placed at hit position over the BigSphere
var mSphere = Sphere(0.2f, 8, 8)
var mBigSphere = Sphere(1.0f, 12, 12)
override fun initScene() {
mViewport = intArrayOf(0, 0, viewportWidth, viewportHeight)
mNearPos4 = DoubleArray(4)
mFarPos4 = DoubleArray(4)
mNearPos = Vector3()
mFarPos = Vector3()
mNewObjPos = Vector3()
mViewMatrix = currentCamera.viewMatrix
mProjectionMatrix = currentCamera.projectionMatrix
mPicker = RayPicker(this)
mPicker!!.setOnObjectPickedListener(this)
try {
currentScene.backgroundColor = Color.BLACK
val light = DirectionalLight(-1.0, 0.0, -1.0)
light.power = 1.5f
currentScene.addLight(light)
currentCamera.setPosition(0.0, 0.0, 4.0)
val noMaterial = Material()
noMaterial.color = Color.TRANSPARENT
val ptMaterial = Material()
ptMaterial.enableTime(true)
ptMaterial.useVertexColors(true)
val v2Material = Material()
v2Material.color = Color.RED
v2Material.enableLighting(true)
v2Material.diffuseMethod = Lambert()
val vMaterial = Material()
vMaterial.color = Color.BLUE
vMaterial.enableLighting(true)
vMaterial.diffuseMethod = Lambert()
mSphere.material = vMaterial
mSphere.position = Vector3(1.0, 1.0, 0.0)
mSphere.drawingMode = GLES20.GL_TRIANGLES
currentScene.addChild(mSphere)
mBigSphere = Sphere(1f, 12, 12)
mBigSphere.material = v2Material
//sphere.setPickingColor(Color.YELLOW)
mBigSphere.position = Vector3(0.0, 0.0, 0.0);
mBigSphere.drawingMode = GLES20.GL_TRIANGLES
//mPicker.registerObject(sphere);
currentScene.addChild(mBigSphere)
mBigSphere.geometry.boundingSphere
} catch (e: Exception) {
e.printStackTrace()
}
}
override fun onRenderSurfaceSizeChanged(gl: GL10, width: Int, height: Int) {
super.onRenderSurfaceSizeChanged(gl, width, height)
mViewport[2] = viewportWidth
mViewport[3] = viewportHeight
mViewMatrix = currentCamera.viewMatrix
mProjectionMatrix = currentCamera.projectionMatrix
}
override fun onOffsetsChanged(
xOffset: Float,
yOffset: Float,
xOffsetStep: Float,
yOffsetStep: Float,
xPixelOffset: Int,
yPixelOffset: Int
) {
}
fun addTouchListener(view: FrameLayout) {
Log.v("TestRenderer3", "Adding listeners")
mGestureListener = View.OnTouchListener { _, event ->
onTouchEvent(event)
true
}
view.setOnTouchListener(mGestureListener)
Log.v("TestRenderer3", "Listener added")
}
override fun onTouchEvent(event: MotionEvent) {
when (event.action) {
MotionEvent.ACTION_DOWN -> getObjectAt(event.x, event.y)
MotionEvent.ACTION_MOVE -> moveSelectedObject(event.x, event.y)
MotionEvent.ACTION_UP -> stopMovingSelectedObject()
}
}
fun getObjectAt(x: Float, y: Float) {
mPicker!!.getObjectAt(x, y)
}
override fun onObjectPicked(`object`: Object3D) {
mSelectedObject = `object`
}
override fun onNoObjectPicked() {
Log.e("sarasa", "No object picked!")
}
fun moveSelectedObject(x: Float, y: Float) {
if (mSelectedObject == null) return
//
// -- unproject the screen coordinate (2D) to the camera's near plane
//
GLU.gluUnProject(
x.toDouble(), (viewportHeight - y).toDouble(), 0.0, mViewMatrix!!.doubleValues, 0,
mProjectionMatrix!!.doubleValues, 0, mViewport, 0, mNearPos4, 0
)
//
// -- unproject the screen coordinate (2D) to the camera's far plane
//
GLU.gluUnProject(
x.toDouble(), (viewportHeight - y).toDouble(), 1.0, mViewMatrix!!.doubleValues, 0,
mProjectionMatrix!!.doubleValues, 0, mViewport, 0, mFarPos4, 0
)
//
// -- transform 4D coordinates (x, y, z, w) to 3D (x, y, z) by dividing
// each coordinate (x, y, z) by w.
//
mNearPos!!.setAll(
mNearPos4!![0] / mNearPos4!![3], mNearPos4!![1]
/ mNearPos4!![3], mNearPos4!![2] / mNearPos4!![3]
)
mFarPos!!.setAll(
mFarPos4!![0] / mFarPos4!![3],
mFarPos4!![1] / mFarPos4!![3], mFarPos4!![2] / mFarPos4!![3]
)
//
// -- now get the coordinates for hit point
//
val hitPoint = Vector3(0.0, 0.0, 0.0)
Intersector.intersectRaySphere(
mNearPos,
mFarPos,
mBigSphere!!.geometry.boundingSphere.position,
mBigSphere!!.geometry.boundingSphere.radius,
hitPoint
)
moveSphere(hitPoint)
}
private fun moveSphere(hitPoint: Vector3) {
Log.d("sarasa", "hitPoint=$hitPoint")
// val factor = ((Math.abs(hitPoint.z) + mNearPos!!.z)
// / (currentCamera.farPlane - currentCamera
// .nearPlane))
// mNewObjPos!!.setAll(mFarPos!!)
// mNewObjPos!!.subtract(mNearPos!!)
// mNewObjPos!!.multiply(factor)
// mNewObjPos!!.add(mNearPos!!)
// mSelectedObject!!.x = mNewObjPos!!.x
// mSelectedObject!!.y = mNewObjPos!!.y
mSphere.position = hitPoint
}
fun stopMovingSelectedObject() {
mSelectedObject = null
}
}
Rajawali Version or Branch
Master
Device and Android Version
Redmi note 8 pro
Summary
I am implementing multitouch Gesture on objects. For translation i am using the method from TouchAndDragRenderer. But when i move the object. it takes a sudden jump and object centre translates to finger position. See the gif below.
Steps to Reproduce
Click on the edge of object and move it. It will take a jump.