search

Mindwerks / worldengine

World generator using simulation of plates, rain shadow, erosion, etc.
MIT License
987 stars 128 forks source link

Support HDF5 #91

Closed ftomassetti closed 8 years ago

ftomassetti commented 9 years ago

We should be able to read and write worlds in the HDF5 format

ftomassetti commented 9 years ago

Steal it from WorldSynth

psi29a commented 9 years ago

yeah, liberate my own work :P

psi29a commented 9 years ago

We support the HDF4/5 format in export, but I think we want to use HDF5 (pytables) instead of protobuf.

ftomassetti commented 9 years ago

Exactly what I wrote on #65. In export I guess we consider just the elevation map, right? I am thinking about all the data

psi29a commented 8 years ago

We can now export to HDF4/5 (also same for ther formats, so this effect multiple issues) Import will likely take a bit of work. ;)

ftomassetti commented 8 years ago

oh, it is already implemented? We do too many things I missed this :) However are we supporting through GDAL exporting just the elevation map? I was working on creating an HDF5 file with all the data we currently export to protobuf or pickle

psi29a commented 8 years ago

Honestly, I'm steering towards protobuf if it does indeed offer python2/3 compatibility.

ftomassetti commented 8 years ago

Right but right now there is an issue that cause me to not be able to load from Java protobuf worlds generated from WorldEngine (#151). As far as I can understand it should be solved when 3.0.0 is properly released. In my initial experiments with HDF5 it seems quite easy to support it, so perhaps we could have support ALSO for HDF5 and that would be particularly useful until protobuf is finalized.

Do you think it is not worthy the trouble having more formats? I admit it could be confusing for the user to support several formats, especially if we use the same extension. Perhaps we could use something like foo.hdf5.we and bar.pb.we

psi29a commented 8 years ago

If you want it, got for it. :) HDF5 is great for storing data-sets... but no so much for meta-data. How far along are you?

ftomassetti commented 8 years ago

well, writing the serialization was easy. Writing the unserialization and a test took a bit but I am about to send the PR now