Open himmetozcan opened 2 years ago
@himmetozcan , thanks for showing interest in my project. Are you trying to align the point clouds from ARCore and COLMAP ?
Thanks for replying. The end goal is merging multiple point clouds of an object using ArCore. After some research, I think I do not know enough information about ArCore's capabilities. Currently, I save the depth image, RGB image, and camera parameters and generate a point cloud from them. The depth image and RGB image are already pixel aligned. I take pictures of the same item from different views, let's say it is an object such as a chair, and try to merge the individual point clouds so that I can have a full 3D view of the object. First I thought using COLMAP would give me the camera poses and locations and so I can use this information to merge the point clouds from ArCore. Now I see that maybe I can already get multiple point cloud frames from ArCore, that are merged already? Like using this [function ](https://developers.google.com/ar/reference/java/com/google/ar/core/PointCloud#getIds()) from ArCore. What do u think is the right way to do it?
ARCore exposes the pointcloud (https://developers.google.com/ar/reference/java/com/google/ar/core/PointCloud), and you can save that. Check my code here: https://github.com/alexs7/arcore-android-sdk/blob/dc6df300ea17f6862d2a984f263f1cd93b2d89fa/samples/hello_ar_java/app/src/main/java/com/google/ar/core/examples/java/helloar/ClientWrapper.java#L46
Have you looked into ICP methods ? https://en.wikipedia.org/wiki/Iterative_closest_point
If your images are overlapping then COLMAP might be the way to go if you are looking to offload the point cloud creation. I am curious too see what you can do with ARCore only
Very nice code, I see that you are sending the data to a local server. Any chance if you can share the code for listening to the send data (maybe a small flask project?).
I have tried ICP on ArCore point cloud, without using any information about camera poses. It is not working that good, and it is very slow (taking 10-15 seconds). I am using the ICP algorithm in open3d.
Hello, the server is a Node.js server that run also a visual AR debugger. The code is here:
https://github.com/alexs7/Mobile-Pose-Estimation-Pipeline-Prototype/tree/demo_version/threejs_viewer
If you cd
in that dir and run npm start
it should start a GUI with a server listening to localhost:3000/
Let me know if you need any help
Hi again. I have managed to run the android app, but couldn't run the threejs_server. npm start
isn't working since there is no package.json file. I ve created one with npm init
but since there is no start script I ve just run node main.js
and getting this
error:
app.on('ready', createWindow);
^
TypeError: Cannot read property 'on' of undefined
at Object.<anonymous> (J:\ArCore\Mobile-Pose-Estimation-Pipeline-Prototype-demo_version\threejs_viewer\main.js:36:5)
I am not familiar with node/npm
so I am stuck here. Any help is appreciated.
Apologies for the late reply, here is the package.json
file, try this and let me know if it works.
"Start" will start electron.
{
"name": "arcore_debugger",
"version": "1.0.0",
"description": "",
"main": "main.js",
"scripts": {
"start": "electron ."
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"electron": "^8.0.1",
"electron-packager": "^14.2.1"
},
"dependencies": {
"body-parser": "^1.19.0",
"bootstrap": "^4.4.1",
"express": "^4.17.1",
"jquery": "^3.4.1",
"pngjs": "^3.3.2",
"popper": "^1.0.1",
"quaternion": "^1.1.0",
"sleep": "^5.1.1",
"three": "^0.91.0"
}
}
Hi there. In one of my project, I am trying to merge ARCore point cloud output, and COLMAP outputs. Particularly, I am trying to use the camera poses from COLMAP, to merge the multiple view ARCore point clouds.
I see that you are converting the COLMAP point cloud to ARCore world, maybe vice versa. Can you point me to the right direction.? For example, can u at least provide a few data from ARCore so that I can understand how the code is working.