Closed ftomassetti closed 6 years ago
Was there something special that had to happen for an OS X (Mac) build/release? I know you guys had me test something at one point.
I am not sure what we need to do for OS X but we could use some help I guess :)
I have time to do more testing, but I'd need a little bit of instruction, like last time.
I'm not really aware of the potential pitfalls - especially how my MacPorts install of python might mask lack of package installs that might happen on a computer running the stock OS X version of python. In other words, I've installed a lot of extra python modules, and it seems like the worst case we need to test against is a user who is running stock python on OS X. I don't know how to make sure that only the stock python is called, to verify that all the other necessary modules are included when installing WorldEngine.
I use brew in my OSX VirtualBox instance.
Build procedures are in the readme.
And now release 0.19.0 has been tagged!
So basically the release is done? :) I guess one of you then has the honor of creating all the binary packages.^^
Is it a bad idea to commit code in the meantime? I tried hard not to start anything new, but I wasn't entirely successful.
The code for release is tagged, we will now create the binary packages but any code committed in the meantime will not affect this. The work on release 0.20.0 or 1.0.0 is officially started :D
So, I've had a few spare cycles to try it on my Mac...
I didn't want to sudo pip install worldengine
, because I didn't want it to mess with my MacPorts python install, which I use for lots of other things. So I went through the things that pip tried to install (without sudo), and made sure everything was installed. Then, went to my clone of the worldengine repo and tried python setup.py build
, which didn't complain. Moving to the build/ directory and then trying to get it to run, I keep getting failures on some of the protobuf stuff. I've verified a couple of times that I've got protobuf installed.
>python2.7 worldengine -n myworld -g out
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/Users/russell/projects/worldengine/build/lib/worldengine/__main__.py", line 5, in <module>
from cli import main
File "worldengine/cli/main.py", line 5, in <module>
import worldengine.generation as geo
File "/Users/russell/projects/worldengine/build/lib/worldengine/../worldengine/generation.py", line 2, in <module>
from worldengine.world import Step
File "/Users/russell/projects/worldengine/build/lib/worldengine/../worldengine/world.py", line 14, in <module>
import worldengine.protobuf.World_pb2 as Protobuf
File "/Users/russell/projects/worldengine/build/lib/worldengine/../worldengine/protobuf/World_pb2.py", line 20, in <module>
DoubleMatrix\x12+\n\x08rivermap\x18# \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12)\n\x06icecap\x18$ \x01(\x0b\x32\x19.World.World.DoubleMatrix\x1a\x1a\n\tDoubleRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x01\x1a\x1b\n\nBooleanRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x08\x1a\x1b\n\nIntegerRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x05\x1a\x18\n\x07\x42yteRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x05\x1a\x34\n\x0c\x44oubleMatrix\x12$\n\x04rows\x18\x01 \x03(\x0b\x32\x16.World.World.DoubleRow\x1a\x36\n\rBooleanMatrix\x12%\n\x04rows\x18\x01 \x03(\x0b\x32\x17.World.World.BooleanRow\x1a\x36\n\rIntegerMatrix\x12%\n\x04rows\x18\x01 \x03(\x0b\x32\x17.World.World.IntegerRow\x1a,\n\x0e\x44oubleQuantile\x12\x0b\n\x03key\x18\x01 \x02(\x05\x12\r\n\x05value\x18\x02 \x02(\x01\x1aq\n\x19\x44oubleMatrixWithQuantiles\x12.\n\tquantiles\x18\x01 \x03(\x0b\x32\x1b.World.World.DoubleQuantile\x12$\n\x04rows\x18\x02 \x03(\x0b\x32\x16.World.World.DoubleRow\x1aS\n\x0eGenerationData\x12\x0c\n\x04seed\x18\x01 \x01(\x05\x12\x10\n\x08n_plates\x18\x02 \x01(\x05\x12\x13\n\x0bocean_level\x18\x03 \x01(\x02\x12\x0c\n\x04step\x18\x04 \x01(\t'
TypeError: __init__() got an unexpected keyword argument 'syntax'
Here is the code its referencing:
Generated by the protocol buffer compiler. DO NOT EDIT!
# source: World.proto
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# from google.protobuf import descriptor_pb2
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor.FileDescriptor(
name='World.proto',
package='World',
syntax='proto2',
serialized_pb=b'\n\x0bWorld.proto\x12\x05World\"\xf6\r\n\x05World\x12\x17\n\x0fworldengine_tag\x18\x01 \x02(\x05\x12\x1b\n\x13worldengine_version\x18\x02 \x02(\x05\x12\x0c\n\x04name\x18\x03 \x02(\t\x12\r\n\x05width\x18\x04 \x02(\x05\x12\x0e\n\x06height\x18\x05 \x02(\x05\x12\x30\n\rheightMapData\x18\x06 \x02(\x0b\x32\x19.World.World.DoubleMatrix\x12\x17\n\x0fheightMapTh_sea\x18\x07 \x02(\x01\x12\x19\n\x11heightMapTh_plain\x18\x08 \x02(\x01\x12\x18\n\x10heightMapTh_hill\x18\t \x02(\x01\x12*\n\x06plates\x18\n \x02(\x0b\x32\x1a.World.World.IntegerMatrix\x12)\n\x05ocean\x18\x0b \x02(\x0b\x32\x1a.World.World.BooleanMatrix\x12,\n\tsea_depth\x18\x0c \x02(\x0b\x32\x19.World.World.DoubleMatrix\x12)\n\x05\x62iome\x18\r \x01(\x0b\x32\x1a.World.World.IntegerMatrix\x12\x38\n\x08humidity\x18\x0e \x01(\x0b\x32&.World.World.DoubleMatrixWithQuantiles\x12-\n\nirrigation\x18\x0f \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12\x33\n\x10permeabilityData\x18\x10 \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12\x18\n\x10permeability_low\x18\x11 \x01(\x01\x12\x18\n\x10permeability_med\x18\x12 \x01(\x01\x12/\n\x0cwatermapData\x18\x13 \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12\x16\n\x0ewatermap_creek\x18\x14 \x01(\x01\x12\x16\n\x0ewatermap_river\x18\x15 \x01(\x01\x12\x1a\n\x12watermap_mainriver\x18\x16 \x01(\x01\x12\x34\n\x11precipitationData\x18\x17 \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12\x19\n\x11precipitation_low\x18\x18 \x01(\x01\x12\x19\n\x11precipitation_med\x18\x19 \x01(\x01\x12\x32\n\x0ftemperatureData\x18\x1a \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12\x19\n\x11temperature_polar\x18\x1b \x01(\x01\x12\x1a\n\x12temperature_alpine\x18\x1c \x01(\x01\x12\x1a\n\x12temperature_boreal\x18\x1d \x01(\x01\x12\x18\n\x10temperature_cool\x18\x1e \x01(\x01\x12\x18\n\x10temperature_warm\x18\x1f \x01(\x01\x12\x1f\n\x17temperature_subtropical\x18 \x01(\x01\x12\x33\n\x0egenerationData\x18! \x01(\x0b\x32\x1b.World.World.GenerationData\x12*\n\x07lakemap\x18\" \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12+\n\x08rivermap\x18# \x01(\x0b\x32\x19.World.World.DoubleMatrix\x12)\n\x06icecap\x18$ \x01(\x0b\x32\x19.World.World.DoubleMatrix\x1a\x1a\n\tDoubleRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x01\x1a\x1b\n\nBooleanRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x08\x1a\x1b\n\nIntegerRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x05\x1a\x18\n\x07\x42yteRow\x12\r\n\x05\x63\x65lls\x18\x01 \x03(\x05\x1a\x34\n\x0c\x44oubleMatrix\x12$\n\x04rows\x18\x01 \x03(\x0b\x32\x16.World.World.DoubleRow\x1a\x36\n\rBooleanMatrix\x12%\n\x04rows\x18\x01 \x03(\x0b\x32\x17.World.World.BooleanRow\x1a\x36\n\rIntegerMatrix\x12%\n\x04rows\x18\x01 \x03(\x0b\x32\x17.World.World.IntegerRow\x1a,\n\x0e\x44oubleQuantile\x12\x0b\n\x03key\x18\x01 \x02(\x05\x12\r\n\x05value\x18\x02 \x02(\x01\x1aq\n\x19\x44oubleMatrixWithQuantiles\x12.\n\tquantiles\x18\x01 \x03(\x0b\x32\x1b.World.World.DoubleQuantile\x12$\n\x04rows\x18\x02 \x03(\x0b\x32\x16.World.World.DoubleRow\x1aS\n\x0eGenerationData\x12\x0c\n\x04seed\x18\x01 \x01(\x05\x12\x10\n\x08n_plates\x18\x02 \x01(\x05\x12\x13\n\x0bocean_level\x18\x03 \x01(\x02\x12\x0c\n\x04step\x18\x04 \x01(\t'
)
_sym_db.RegisterFileDescriptor(DESCRIPTOR)
Any ideas for what to look at next? I'm sure I'm screwing up the install somehow.
Is there no virtualenv
on OSX? It took me an eternity to find out that installing WorldEngine under Linux only worked inside a virtual environment.
The error message hints at you using Protobuf 2. The syntax-keyword was introduced in Protobuf 3, I believe.
Thanks much for the tips!
https://github.com/Mindwerks/worldengine#osx <-- using brew worked for me but there were problems with protobuf and I had to manually copy files around.
How is the release going? Is anybody creating binary packages? I have Linux and a Windows inside a virtual machine, with some instructions I might be able to help.
No packages are created until we are tagged for release.
We have the tag for release and the source packages on Pypi but no binary packages
The tag is: v0.19.0
yay, a release! @ftomassetti I thought you wanted me to review (and possibly merge) your PR. We'll save that for 0.20 ?
Sorry for the misunderstanding, I did not want to sneak a release behind your back: I thought the plan was to tagging the release as soon as all the stuff marked as 0.19.0 was in.
Yes, I think it is better to merge such big PRs at the beginning of a new development cycle.
Roger that, I'll try to get around to making binary packages when I have time. It may or may not be with GDAL. If someone else wants to follow my guide and get everything included, by all means, please do.
I don't want to hold up the release because of real-life getting in the way on my side. :)
Well, the release has been tagged, binary packages can arrive when real-life permits that :)
OpenMW is also prepping for a big release (last release was in June... major switch from Ogre3D to OSG. ;)
I assume moving the tag to the current state isn't possible? We would be including the latest manual that way.
@psi29a What guide is there to follow for a release?
The manual is available online: if we start moving the tag we will keep moving it :)
But we can release v0.19.1 anytime you want. Release early, release often!
Created linux binary package for x86_64 and uploaded it here: https://github.com/Mindwerks/worldengine/releases/tag/untagged-3bd72a11fa787656f531
this is a draft, it is possible that some people will not be able to see this.
Ok, I am one of those people.
OK, here is it on my dropbox... not for public consumption: https://dl.dropboxusercontent.com/u/396161/worldengine-0.19.0-Linux_x86_64.tar.bz2
When we're ready, we'll use github for releases that can be posted all over reddit and etc.
It is just a dirtectory that can be extracted, in that directory is the worldengine binary and all the supporting libs. I managed to cram gdal and hdf5 in.
Ah ah I just put it on my dropbox: https://www.dropbox.com/s/as4p4x9i3wzxals/worldengine-0.19.0-Linux_x86_64.tar.bz2?dl=0
One is enough, but thanks.^^ A first run on my machine worked fine. I guess I'll try to create an ancient map from a saved world, too - other than that should I do any specific testing? Or is it enough that nothing crashes?
The release has grown quite a bit, 0.18 was at ~18 MB. libicudata.so.55
and libopenblas.so.0
are huge files - what do they do?
Loading the world and creating an ancient map worked fine, too.
likely from gdal ;) gdal is HUGE
Added OSX as well, please test: https://dl.dropboxusercontent.com/u/396161/worldengine-0.19.0-OSX-x86_64.tar.bz2
Also attached to this release, so please do not share. I'll delete the dropbox files when we publish the release.
Who can even test this? You might want to use @...
@ftomassetti apparently as a Yosemite laptop. I have Mountain Lion in a virtualbox. The more than can test this the better of course. ;)
I just tested, apparently GDAL is included. I'm surprised at the size, it came in at 19.7 MiB as apposed to the Linux build. I suspect that the Linux build has more GDAL plugins (and the kitchen sink). ;)
I could generate a world correctly but I got an error running ./worldengine -x 100 -y 100 --hdf5 --sat
Traceback (most recent call last):
File "<string>", line 619, in <module>
File "<string>", line 550, in main
File "<string>", line 92, in draw_satellite_map
File "/Users/psi29a/Workspace/WorldEngine/worldengine/worldengine/draw.py", line 816, in draw_satellite_on_file
File "/Users/psi29a/Workspace/WorldEngine/worldengine/worldengine/draw.py", line 375, in draw_satellite
File "/Users/psi29a/Workspace/WorldEngine/worldengine/worldengine/draw.py", line 250, in get_normalized_elevation_array
TypeError: astype() takes no keyword arguments
main returned -1
It can be replicated with:
./worldengine -x 100 -y 100 --sat -s 28753
Same command run fine on linux. I am not sure it is due to the way we packaged the project on Mac.
Running on mac from master it runs fine. That is strange
That's here, and astype()
takes a keyword argument since numpy version 1.7. 1.6 didn't have that argument, though. Maybe the bundled numpy is outdated?
chapeau @tlcd, good catch!
Nice catch, I'll take a look to see what version is installed.
Removing the dtype
-keyword from astype()
would be an option, too. Although if there really is an old numpy-version in use, other problems could crop up later.
worldengine\draw.py
Line 250: c = numpy.rint(c).astype(dtype=numpy.int32) # proper rounding
Line 538: colors = numpy.rint(colors).astype(dtype=numpy.int32) # proper rounding
Line 588: colors = numpy.rint(colors).astype(dtype=numpy.int32) # proper rounding
worldengine\image_io.py
Line 130: _array = numpy.rint(_array).astype(dtype=PNGWriter.get_dtype(channel_bitdepth)) # proper rounding
OK... according to pip freeze on my OSX instance, the version is 1.9.2
however, running what @ftomassetti above does indeed trigger the error. Perhaps this bug is in the OSX version of numpy.
I ran the command on the mac using master (not the binary package) and it seems to run fine
ok, something is up... fired up ipython: from numpy import version; version.full_version 1.6.1
looks like something is being used in the place...
Fixed (I hope): https://dl.dropboxusercontent.com/u/396161/worldengine-0.19.0-OSX-x86_64.tar.bz2 This is a 30MiB file... bigger than the 19MiB from the one above. I think more plugins are included this time around. ;)
Any testers?
Run fine for me!
Super! The only think left is try our hand at creating windows packages.
0.19 still isn't out yet. What's necessary to create a Windows package?
Just follow my guidelines and you should be good. ;)
I just need 1) time and 2) find a windows VM to do this in.
What guidelines?
Version number should be updated:
Am I missing something?