search

BlenderHQ / camera_projection_painter

Allows you to paint over the texture of the reconstructed object with a clone brush from the original photo.
https://docs.camera-painter.com/
GNU General Public License v3.0
18 stars 2 forks source link

Agisoft Metashape Support #1

Open tbenazzi opened 6 months ago

tbenazzi commented 6 months ago

Your Work is a Gem Dude !!! We can't wait to see an Agisoft Metashape support to try this tool and help you to develop further.

Actually, metashape can export images in those formats :

image

Ask us if you need some project cameras export. we have plenty dataset to share if necessary.

ivan-perevala commented 6 months ago

I already ivestigated Metashape's "Agisoft XML (*.xml)" file format, so its possiblly would be landed in next release.

Of course, there is more questions about exporting (for same purpose as with Reality Capture's *.xmp files - transform the entire scene and import back to RC), cause there is a lot of data which would be lost (region, meta, reference, even components keys are hard to recalculate from Blender's scene data). So, I think for start I would just skip export part.

Anyway, thank you for request and support!

ssh4net commented 6 months ago

@ivan-perevala I think most of users only expect an import from photogrammetry apps not an export back.

But using solid XML file open an another question on how handle arbitrary camera load from such files. When with RC XMPs we can load only selected cams.

ivan-perevala commented 6 months ago

@ssh4net Metashape exports only active component into "Agisoft XML" file. That means that for importing all cameras in your dataset, you actually need export each component as separate file and than - just import them by an addon as multiple files, exactly same way as with RC XMP's.

And I dont see any issue in importing all the cameras, tbh. In Blender you always can just enter object mode and delete unused cameras, or do the same from outliner, even in texture paint mode. Or just do not use them for painting :)

Of course, that is possible, in cost of import file browser performance (import just camera names from file, create UI list of them and give an option for each/all names to be imported or ignored)

Maybe I dont got the point, but if you want in some way select cameras for import during importing from a single file (which MS *.xml is), what is the goal of this process? Why does it needed?

I think most of users only expect an import from photogrammetry apps not an export back.

I agree. For sure, it would be nice to see some kind of "consistency" in addon's IO - each file format can be not only imported, but also exported back. But I think this should be disscussed for each third-party software file format. For example, as I mentioned, export makes sense for Reality Capture's XMP. So, the question is: does it makes any sense for exporting Metashape's XML? @tbenazzi , @ssh4net , what do you guys think?

tbenazzi commented 6 months ago

@ivan-perevala thanks for your answer.

regarding the export back to the third party software, I don't have a specific use case in mind where it could be useful.

Once images are aligned, controlled and fixed in position, rotation and distorsion, we don't need to modify anything anymore on them. I maybe missing a use case where it could be useful but in our case it's not really necessary.

ivan-perevala commented 6 months ago

@tbenazzi , well, than for now it would be only import.

Ask us if you need some project cameras export. we have plenty dataset to share if necessary.

I'm using the same dataset (ring in Warszaw) which I prepared on a bunch of different cameras with dirrerent resolutions and orientations, and its pretty enough for dev purposes. @ssh4net also gave me more comlex datasets (with 1500+ images:)), so no need to worry about)

ssh4net commented 6 months ago

@ivan-perevala Oh, I know one use case. I used to export Agisoft XML from RealityCapture (using reports) to debug issue with texturing using 32 bit float (signed) images. Also some might feel that other app (with a greener grass) can texture better. So that discussion remember me that I want to have the way to convert cameras and camera intrinsics across apps. And probably one of the reasons why we have this XMP export, in Camera Painter :D

ivan-perevala commented 6 months ago

I commited some basic implementation b757b286e1a08f6bbb779f6318c5f3f11f1e62e0. For now I see a few obvious issues:

So for now I'm not creating a new release, this should all be fixed image Actually, I added:

tbenazzi commented 3 months ago

Hello @ivan-perevala ,

I put some time to analyse your work, Actually I did not manage to find a way to import my Agisoft AT :/

Last week I had the "Agisoft metashape" choice into the Software menu, i reinstalled the addon and it's not the case anymore with the latest release. Am I doing something wrong regarding the installation ?

image

second thing: most of the time our projects are georeferenced into a specific cartographic system. Is it a problem for your add-on to understand those big coordinates ? I know that blender does not manage to import double precision float. Could be possible to truncate a constant shift value in order to keep à cm or mm accuracy on coordinates of camera positions ? Metashape offer this feature when we export model but not when we export cameras... it's maybe already the case on your work.

ivan-perevala commented 3 months ago

Evening, @tbenazzi !

I haven't done a release yet with the last published commits. Therefore, yes, it is necessary to download not the release, but the last published commit.

Currently I'm working on import/export optimization, so I plan to finalize the new addon release by the next Blender LTS (4.2) release, already with Metashape IO implementation. In any case, I won't change the UX too much in this regard, I think currently importing *.xml, *.txt is enough to understand what needs to be added/corrected in the future.

Regarding the problem with Blender's single precision, I think, for a start, you can try to import such a scene as *.fbx, *.obj, *.dae (any standard importer) The limit of the floating point number is 1.17549435e-38 - 3.40282347e+38, so here you need to take into account the entire context of the task. In any case, the further from the center of the coordinates, the lower the accuracy of the data (and the highest - from -0.5 to 0.5).

The add-on, unlike Blender itself, can import, save, and export data in double-precision format, but again, only for cameras (i.e. mesh vertices coordinates are not affected).

The restriction also applies to cameras. For example, the imported camera position is stored as a double-precision vector, but if you change the actual position of the camera object in the scene, this data will be updated (from the actual single-precision coordinates). The data update takes into account the difference in the float -> double conversion - thanks to this, the data is not updated by itself (because the difference between the actual and imported value, although very small, is still there). To summarize, the imported data will remain double-precision data until the actual data is changed. Another limitation follows from this - if you import values with double precision that will go beyond the limits of a number with single precision, then they will be "truncated" - that is, the actual import will take place, but not correctly.

In any case, I am talking about extremely large numbers that have not yet been tested in practice with this add-on.

If there is any copyleft scene for testing, I would ask that you share. I think the question can be solved using the "Global Scale" parameter of the importer (in the import process, it directly affects not only the scale, but also the accuracy of the data)