Possibility to load classic vector 3D data files

[pcaillou]

As GAMA is improving a lot concerning the management of 3D data, I think it is a good
time to add the possibility to load classic 3D vector data files.

I am quite motivated to try to implement this feature, but I do not know the classic
file formats: 3ds ? Autocad files ? 

Original issue reported on code.google.com by patrick.taillandier on 2013-12-22 18:43:08

[pcaillou]

I think SVG files would be nice as it is a quite popular format (and also a little bit
like shape files regarding interchange between applications). 

Original issue reported on code.google.com by alexis.drogoul on 2013-12-23 08:36:41

[pcaillou]

I must correct myself: SVG is not a 3D format (only 2D in the official specifications),
and the most popular 3D interchange format is ".obj". However, importing SVG graphics
could be interesting anyway to create complex 2D vector representations. And there
are already readers for .obj and .3ds files in JOGL, so if it is only for visualization
purposes, there is almost nothing more to do (just accepting these new formats as we
accept .png or .jpg in "draw", and programming their rendering in JOGL). If it is for
modeling purposes, it will be a bit more complicated as you need to translate back
the 3D geometries created by JOGL into JTS geometries (not too complicated, I assume,
but still), and we need to make sure that the 3D information is not lost in the different
operations we make in models.

Original issue reported on code.google.com by alexis.drogoul on 2013-12-23 18:20:09

[pcaillou]

There is also standard file like x3D (http://www.web3d.org/x3d/) or collada (.dae)(https://collada.org/)

For sure it will be a good idea to implement it I already spent some time on that and
never find a clean jogl reader/loader to do it. As soon as we have a loader that can
behave like the JTS one by giving the x,y,z of each vertex of the file it "should"
be easy to implement them in opengl.

I am also quite motivated to work on that (since a long time actually)

Original issue reported on code.google.com by agrignard on 2013-12-24 16:19:46

[pcaillou]

Hi,

Before going any further, there is a need to modularize a bit the management of files.
Right now, they are more or less centralized in msi.gama.core, but I have begun implementing
an extension point allowing subclasses of IGamaFile to be declared in plugins and recognized
by the core. So that in the jogl plugin, we can take advantage of the existing reader
to declare GamaObjFile, for instance. The same for the OSM plugin and GamaOsmFile.
Give me 2 days to finalize it!

Original issue reported on code.google.com by alexis.drogoul on 2013-12-25 06:56:14

• Labels added: Milestone-1.6.1

[pcaillou]

OK, the addition for enabling loading files declared in plugins has been committed.
It's now possible to declare file types in the JOGL plugin, using the OpenGL library
to load .obj or .3ds files and have them recognized by msi.gama.core (along with specialized
operators like is_obj, obj_file, is_3ds, 3ds_file created automatically). 

You can work now :)

Cheers
Alexis

Original issue reported on code.google.com by alexis.drogoul on 2013-12-30 06:03:37

[pcaillou]

I've begun taking a look at the loading of .obj files, and I wonder if we do not have
to add information to GamaGeometry in order to render 3D objects correctly. First,
do we need to store the vertex normals ? It seems to be used mainly for lighting, but
there are perhaps other uses. Second, the notion of faces (defined by the list of vertex
that constitues a "face" polygon) seems to be important (otherwise, all the 3D objects
will be transparent).

So my first move would be to :

1) Load each vertex in an array (transformed into GamaPoints)
2) Load each face (defined by their indexes of vertices)
3) Create a MultiPolygon in JTS
4) Add to this geometry the faces (each being a Polygon in JTS)

Do you think it is correct ? 3D geometries, in that case, will not be stored as "simple"
geometries, but as "multi-"geometries, each of the geometries defining a face. It would
require, then, to draw them correctly in JOGL (dont know if it is the case yet).

This change would also require taking a look at and changing the existing "3D" geometries
(cube, etc.) to store them using the same scheme. 

Tell me what you think. I will commit (as soon as possible) the first attempt, so that
you can judge by yourself.

Cheers
Alexis

Original issue reported on code.google.com by alexis.drogoul on 2013-12-30 08:25:02

[pcaillou]

This has been committed. Look at GamaObjFile in the jogl plugin. Nothing has been tested
yet. The buffer of this file is simply the list of polygons that compose the 3D geometry.

Original issue reported on code.google.com by alexis.drogoul on 2013-12-30 09:14:08

[pcaillou]

As a side note, I have just committed GamaSVGFile (with two builders: svg_file(string),
svg_file(string, point), the first one using the size defined in the SVG file, the
second one forcing the envelope of the resulting shape). SVG files are supposed to
support loading one and only one geometry (which can be complex, though).
An example of use can be found in the Ant Sorting toy model. Complex 2D shapes can
now be loaded from something else than shape files ! 

Original issue reported on code.google.com by alexis.drogoul on 2013-12-30 16:19:27

[pcaillou]

I have committed a first attempt at reading 3DS files (Gama3DSFile). In a fashion similar
 to GamaObjFile, it only reads vertices and faces and stores the faces as polygons
(triangles, actually) in a multi-geometry GamaShape. Multiple objects can be read at
once (resulting in multiple GamaShapes). To access them, list(file("xxx.3ds)) is enough.
Or 'loop shp over file("xxx.3ds")'. It has NOT been tested so far.

Original issue reported on code.google.com by alexis.drogoul on 2013-12-30 21:46:30

[pcaillou]

Here is a toy model to test svg, .obj and .3ds teapot rendering

I commited it in models/Features/3D Visualization/Toy Models/ComplexObjectLoading

I was expecting to be able to write: something like:

geometry objfile <- geometry (obj_file("./includes/teapot.obj"));
geometry threedsfile <- geometry (threeds_file("./includes/teapot.3ds"));

but it seems forbidden as it says:
The extension obj is not recognized for this type of file
    at msi.gama.runtime.exceptions.GamaRuntimeException.error(GamaRuntimeException.java:57)
    at msi.gama.util.file.GamaFile.checkValidity(GamaFile.java:63)

What Am i missing?

Original issue reported on code.google.com by agrignard on 2014-01-02 16:58:11

---

• Attachment: ComplexObjectLoading.zip

[pcaillou]

I accidentally added the "." in the extensions of the annotation in GamaObjFile. 

The correct annotation is : @file(name = "obj", extensions = "obj") // and not ".obj"

Maybe I should try to detect this, by the way, as it is not too complicated to do…
 Anyway, if you change it and rebuild the projects, it should work. 

Original issue reported on code.google.com by alexis.drogoul on 2014-01-02 17:04:44

[pcaillou]

Ok this is working now (I ve commited the chnage) but I get the following error when
loading a .obj file.

I just downloaded a teapot.obj so maybe the shape is too complex or at least not "valid"
in the sense of the jts

Caused by: java.lang.NullPointerException
    at com.vividsolutions.jts.operation.valid.ConnectedInteriorTester.visitInteriorRing(ConnectedInteriorTester.java:178)
    at com.vividsolutions.jts.operation.valid.ConnectedInteriorTester.visitShellInteriors(ConnectedInteriorTester.java:155)
    at com.vividsolutions.jts.operation.valid.ConnectedInteriorTester.isInteriorsConnected(ConnectedInteriorTester.java:100)
    at com.vividsolutions.jts.operation.valid.IsValidOp.checkConnectedInteriors(IsValidOp.java:612)
    at com.vividsolutions.jts.operation.valid.IsValidOp.checkValid(IsValidOp.java:259)
    at com.vividsolutions.jts.operation.valid.IsValidOp.checkValid(IsValidOp.java:183)
    at com.vividsolutions.jts.operation.valid.IsValidOp.isValid(IsValidOp.java:153)
    at com.vividsolutions.jts.operation.valid.IsValidOp.isValid(IsValidOp.java:60)
    at com.vividsolutions.jts.geom.Geometry.isValid(Geometry.java:433)
    at msi.gaml.types.GamaGeometryType.buildPolygon(GamaGeometryType.java:124)
    at msi.gama.jogl.files.GamaObjFile.fillBuffer(GamaObjFile.java:86)

Original issue reported on code.google.com by agrignard on 2014-01-02 17:20:10

[pcaillou]

As I wrote, I did not test the two functions (it was mainly as a first trial in order
to establish the foundations). So there may be a lot of errors, and I would suggest
to begin with very simple shapes (a cube, maybe ?). The main problem lies in my opinion
in the way the OpenGL display can handle these complex geometries (where each "face"
is a polygon in a multi-polygon geometry). 

Original issue reported on code.google.com by alexis.drogoul on 2014-01-02 17:24:38

[pcaillou]

OK I'll dig into it.

In the big line the solution of the multi-geometry made of faces sounds good.

Original issue reported on code.google.com by agrignard on 2014-01-02 17:30:35

[pcaillou]

(comment of Patrick on the gama-dev list)
This bug mention by Arnaud is due to the test of validity of the buildPolygon function
(GamaGeometryType):
if ( p.isValid() ) { return new GamaShape(p.buffer(0.0)); } // Why buffer (0.0) ???

For a reason that I do not understand, the isValid() seems to return a null pointer
exception (whereas the geometry is perfectly valid). 
 I have tested to create by hand several geometries (geometry test <- polygon([{2,2},{2,10},{10,6},{5,8}]);)
and I have always the same bug. 

If we remove the test of validity, the geometries are correctly created.

Original issue reported on code.google.com by alexis.drogoul on 2014-01-02 17:51:37

[pcaillou]

To answer Comment #15, I can reproduce the NPE, but impossible to know where it comes
from and how to fix it. I've tried to find a report on the JTS mailing list, but nothing
emerged. I have then removed the test, although it makes me a bit nervous… 

Original issue reported on code.google.com by alexis.drogoul on 2014-01-03 06:21:33

[pcaillou]

I am also starting with the .3ds file (I've just  commited a short modif in Gama3DSfile.java)

3ds loading.

I am trying to load a teapot.3ds (I can open this file in sketchup). In gama I get
a NPE at msi.gama.metamodel.shape.GamaShape.copy(GamaShape.java:450)

public GamaShape copy(final IScope scope) {
...
g = new GamaShape((Geometry) geometry.clone());
...
}

the geometry is null

The constructor of the GamaShape is called in the fillBuffer when doing

for ( Obj obj : objects ) {
  Geometry g = GeometryUtils.factory.buildGeometry(obj.faces);
  ((GamaList) buffer).add(new GamaShape(g));
}

and objects are created in processNextChunk() if currentChunk.id = OBJECT in my case
(with teapot.3ds) I never enter in this case, so the objects is never fill.

Maybe we could split this issue in one for the .obj and one for the .3ds now that the
loading of the file seems ok.

Original issue reported on code.google.com by agrignard on 2014-01-03 09:23:04

[pcaillou]

==========Obj loading=========
First version of a .obj loading but in 2D...

I just changed one line in the code in the reading of the faces in fillBuffer (r 8389)
points.add(vertices.get(v[i]))
has been changed by
points.add(vertices.get(v[i]-1)) to avoid index out of bound.

The loading seems ok. But I stiil get a problem in opengl the renderer is frozen and
nothing is drawn.

Original issue reported on code.google.com by agrignard on 2014-01-03 10:32:25

---

• Attachment: Screen Shot 2014-01-03 at 11.28.10.png
  

[pcaillou]

==========Obj loading=========
Done!!

Original issue reported on code.google.com by agrignard on 2014-01-03 12:03:17

---

• Attachment: Screen Shot 2014-01-03 at 12.57.55.png
  

[pcaillou]

Congratulations ! 

I'm happy to see that it works. I'll take a look at the parsing of the 3DS files, as
I may have been a bit quick...

Original issue reported on code.google.com by alexis.drogoul on 2014-01-03 12:17:10

[pcaillou]

I wanted to add (beautiful) 3D cars in my models, but, contrary to our teapot, my 3D
cars do not want to be displayed. I attached the obj file. 

Original issue reported on code.google.com by patrick.taillandier on 2014-01-03 13:19:08

---

• _Attachment: car_riviera.obj_

[pcaillou]

There is more information in this file  than in the teapot to support the texture

This is not taken in account yet but it should at least display the car without the
texture.

Original issue reported on code.google.com by agrignard on 2014-01-03 14:02:33

[pcaillou]

i want some humanity, too, but my model didnt work. It contained no texture, i think.
http://static.tf3dm.com/models/dropbox/dropbox/sq/Skeleton.rar

Original issue reported on code.google.com by hqnghi88 on 2014-01-03 14:07:11

[pcaillou]

@Patrick For the riviera car we need at least the mtl file mtllib car_riviera.mtl that
contain the textures.

@Nghi the .obj also include an mtl file (mtllib skeleton.mtl).

Now it seems that it only works for non texture .obj and if the faces are define only
with one value (f 1 2 3) and not (f 1/1 2/2 3/3)

Original issue reported on code.google.com by agrignard on 2014-01-03 14:19:05

[pcaillou]

Here, all the files for the riviera car .

Original issue reported on code.google.com by patrick.taillandier on 2014-01-03 14:22:03

---

• _Attachment: car_riviera.jpg_
• _Attachment: car_riviera.mtl_

[pcaillou]

Yes, for the moment, as stated in the comments of the two classes, it only works for
loading geometries. However, parsing the textures, etc. should not be too difficult.
The difficulty, afterwards, is to know how to store them in the geometry. 

regarding f 1/1 2/2 3/3, etc. the parser should be ok to deal with them. But maybe
there is an error in it ? 

Original issue reported on code.google.com by alexis.drogoul on 2014-01-03 14:25:15

[pcaillou]

I'll work on that (have textured object) but I have no idea how long it can take but
(I hope it will work soon!!)

Original issue reported on code.google.com by agrignard on 2014-01-03 14:25:49

[pcaillou]

Concerning the texture I think we can easily store it in the geometry as I did for the
procedural city. A GeomtryObject already has a list of texture in its attributes.

Original issue reported on code.google.com by agrignard on 2014-01-03 14:28:00

[pcaillou]

Finally what we need is a MTLLoader.

Original issue reported on code.google.com by agrignard on 2014-01-03 14:42:16

[pcaillou]

Hi, for 3ds loader, i have a working class with stand-alone jogl. I didnt success to
integrate it in threeds_file. I share it if it'll be helpful

Original issue reported on code.google.com by hqnghi88 on 2014-01-03 14:48:36

---

• Attachment: JD3dsModel.java

[pcaillou]

Another lib (one for obj, another for 3ds): https://code.google.com/p/victamin/

Original issue reported on code.google.com by patrick.taillandier on 2014-01-03 14:49:58

[pcaillou]

To make it as clean as possible maybe we should follow the "mood" of what have been
done in the fillBuffer. ANd not rewrite a parser form scratch.

And add something like:

//LOADS MATERIALS
if ( newline.startsWith("mltlib ") ){
}

//LOADS NORMALS
if ( newline.startsWith("vn ") ){
}

//LOADS TEXTURES
if ( newline.startsWith("vt ") ){
}

Is the previous code has been inspired by an existing one?

Original issue reported on code.google.com by agrignard on 2014-01-03 15:26:41

[pcaillou]

Actually, there is no problem to use other libraries / readers — I've just implemented
the two parsers to start the trend and provide a common framework. fillBuffer() can
perfectly call another library if needed (it is the case in the SVG file implementation,
although it does not seem to work for every file — the isValid() bug is still here
!). The only thing is to build and return either a multi-geometry or a list of geometries.

Original issue reported on code.google.com by alexis.drogoul on 2014-01-07 05:26:15

[pcaillou]

I know, the discussion here is very technical, but I thot to share that I imported GML
file of a city center. attached is a picture in gama.

just adding .GML to your list :)

Original issue reported on code.google.com by b.srirama on 2014-01-24 13:40:47

---

• Attachment: Capture.JPG

[pcaillou]

Hi Srirama,

this is a good news!! 

So does it means that you wrote the parser in GAMA? 

Will you commit it? If yes Is it possible to have the model and the .gml to test it?

Original issue reported on code.google.com by agrignard on 2014-01-24 14:16:45

[pcaillou]

No Sir, I am not programmer of that caliber (yet) :):). I needed something quick for
a presentation with the municipality for my phd case study. So I took the long route
but quick, I change .gml to shapefile. then load it. But i see GML file is a normal
xml file, it is very easy to identify a building, its ID and link its ID to its elevation.
There are many gml files online, it is open format. If you not find it, I can send
it.
 they are usually very large, like 30/40 mb. here is only 10mb allowed.

Original issue reported on code.google.com by b.srirama on 2014-01-24 14:29:37

[pcaillou]

As Geotools (that we use to load shapefile) allows to read GML files, it should not
be too complex to add this feature (at least, in theory ;) ).

Original issue reported on code.google.com by patrick.taillandier on 2014-01-24 17:15:10

[pcaillou]

Hi,

Considering the shapefiles can be imported ? Did anyone try importing multipatch shapefiles?
May be already exists in Geotools? I attach a document to this line. If multipatch
shapefile can be imported, a lot of .dae files can be important as there exist lot
of dae to multipatch shapefile converters.

Original issue reported on code.google.com by b.srirama on 2014-02-03 11:33:26

---

• Attachment: [6.3-1.5 THE USE OF MULTIPATCH FOR THE OBJECTS IN.doc](https://storage.googleapis.com/google-code-attachments/gama-platform/issue-760/comment-38/6.3-1.5 THE USE OF MULTIPATCH FOR THE OBJECTS IN.doc)

[pcaillou]

important* = imported

Original issue reported on code.google.com by b.srirama on 2014-02-03 11:35:04

[pcaillou]

(No text was entered with this change)

Original issue reported on code.google.com by patrick.taillandier on 2014-02-17 15:37:17

• Labels added: Milestone-1.7
• Labels removed: Milestone-1.6.1

[pcaillou]

Hi,

here I am trying to bring in a 3d shape (a multipatch feature from arcgis)

1. created in trimble sketchup exported to .dae
2. imported into arcgis as multipatch geodatabase
3. exported to multipatch shapefile

could not load into gama. It is a very simple shape, you can see attached picture.

I wanted to display some pavement deterioration. 

thanks
srirama

Original issue reported on code.google.com by b.srirama on 2014-04-12 07:46:52

---

• _Attachment: gama_road3d.zip_
• _Attachment: gama_road3d.tiff_

[pcaillou]

Issue 1067 has been merged into this issue.

Original issue reported on code.google.com by alexis.drogoul on 2014-09-09 01:19:40