Added new scatter plot

[esampson]

Added a new command line option; --scatter.

This will draw a scatter plot of all land elements with the normalized temperature along the x axis and the normalized humidity along the y axis. This will be useful for fine tuning humidity and temperature routines.

[esampson]

I tried adding --scatter to the full smoke test but for some reason that generated the error:

IndexError: image index out of range

[tcld]

It would be really, really nice if https://github.com/Mindwerks/worldengine/pull/146 could go in first. Since I am touching a lot of code at the moment, I keep having to go back over it to make it fit changes made by others. In this case generation.py is changed by both of us.

[esampson]

It can certainly wait and go in after your changes, however I'm guessing it won't be too hard for them to cherry pick and merge both sets of changes together (and if I'm wrong then I am happy to update generation.py after #146 has gone in and make a new commit)

[ftomassetti]

Amazing idea drawing charts. As a world-generation fanatic I am going to love this :) We should have more, many more charts!

[esampson]

Scatter plot for seed 1.

[ftomassetti]

that is impressive! I do not understand it very well but for sure it looks nice

[tcld]

Pretty.^^ Thanks for the example. What do you think about preserving the aspect ratio that is used by the map itself? That way it would be easier to see which point corresponds to what area on the map. This way, of course, it would be easier to compare maps of different sizes. So both approaches have their advantages.

[esampson]

X axis is temperature and Y axis is humidity. As X goes to the right it becomes hotter and as Y goes down there is more water. The lines represent the various thresholds (semi-arid, arid, etc.).

What it is currently telling me is that the routines are doing a fair job of mimicking the correspondence between 'rainfall' (which is our humidity) and temperature. Our polar regions (far left) doesn't have to much water and our hottest regions are getting the maximum water.

However I can also see room for improvement. While the bottom line of the plot is good the top is sloping down too much. We don't have any arid regions in the tropical zones, which is wrong. I need to figure a way to 'stretch' the results up as they go to the right, if that makes any sense.

Copying the aspect ratio of the map wouldn't do anything to help you see what areas correspond to what areas on the map. It's a data plot so a point in the lower right means there's a spot on the map that has both high temperature and high humidity, but it could be absolutely anywhere on the map. There's just no correspondence to location.

[tcld]

Sorry, I didn't look too far into your code. Yes, it does make sense and my request was bullshit. :)

On a sidenote: Is there any way to include the axes and/or some kind of legend? Is an image the best way to output this kind of data? There is probably some simple plotting library out there that is a little more suited for the job. Maybe we can add that as an optional library later.

[esampson]

Yeah. I've been thinking about labeling the axis (and possibly a few other tricks), but right now I have to look up how to place text onto an image with the current libraries (or else find a new library and add that).

An image like this is probably a pretty good way to express the data, actually. As an example in the real world astronomers make extensive use of Hertzsprung-Russell diagrams.

Once you understand what you are looking at (and I'll admit, its taken me more than a little bit to do that, but essentially there should be a triangle that fills about half the graph with the hypotenuse running from upper left to lower right) it gives you a pretty clear and concise way of understanding if there are issues. There might be some variations in design such as whether or not you normalize the columns and rows, but you are still talking about conveying the overall information as an image.

[esampson]

BTW, if you guys don't want to integrate this that's actually not a huge deal for me (in fact it makes life easier since I won't feel obligated to expand the routine and label axis and things). I wrote this because I had a feeling that our temperature and humidity routines were spotty and I wanted to see what needed fixing. I wasn't originally planning on submitting it, but I decided I might as well since ultimately we may give people more control over temperature and humidity and it would be good to give them a tool that lets them see if they are producing results that are way off base.

There's stuff that would make the chart nicer but right now the next thing on my agenda will probably be to work on the temperature, precipitation, and humidity routines to try and get us more 'Earth normal' results (because of the slant to the right we don't end up with very much dry terrain in the tropical regions, thus we almost never produce anything like the Sahara) unless people say they would really like something else done first.

Of course this will screw up nearly all of our blessed images since it will lead to different precipitation maps which will lead to different biome maps and most likely different river maps and erosion patterns (I haven't looked at those routines but I assume messing with rain will mess with their results).

[tcld]

I would prefer if things could be Earth-like but don't always have to be. So while you are working on further things, maybe you could put in a parameter or two that give a little control over this. Just in case. :)

I don't think that changing the blessed images is too big a deal, as long as the results look good/interesting. Whenever anything about one of the generation-steps is changed, one or several of the images will change - that's just how it is; and it will happen quite often.

About including this code: I think it would be a great addition, but it would be good to use some kind of proper plotting library so things are more self-explanatory. A decent implementation would maybe open up ways for later addition of even more plots. (But even as is, this would be a good starting point.)

[esampson]

Yeah, it's my preference as well that the code ultimately evolve to a point where it produces Earth like planets by default but that it is relatively easy for someone to produce a desert world or an ice world or whatever it is that they may want.

That said, I would like the say a few things in the interest of full disclosure.

My full name is Evan Sampson and I have been involved with online gaming for about the past 15 years. I have worked on Everquest, Everquest 2 and Planetside 2. I'm disclosing this because I don't want someone to come along, somehow trace me down, and then think 'Holy crap! He's doing this so he can steal our work!'.

My involvement with this project initially came about because I was looking for a better way to generate planets for Traveller, hence part of the reason I would like to be able to generate non-Earth like planets. It was unrelated to any project I was working on professionally.

That said, I would like, with everyone's approval, to perhaps show this project to various people in the future. I have long had the idea that the tools currently being used to generate content for most MMO's are really crappy. When a city needs to be generated someone has to sit there and place every individual building and assign every individual NPC. This results in 'cities' being incredibly tiny with populations of a few hundred people and often unnatural layouts. Maps are small because artists have to generate every hillock and place every tree. Quests are repetitive (meaning that thousands of heroes have each individually helped recover the lost crown of Angblat) because they are written by people and each quest results in hours of effort on the part of the content developers.

Assuming people are fine with this I will probably show this project to various people in the future, not as a place to steal code but simply as an example of how procedurally based content could work. I don't plan to take any credit beyond that of a 'contributor'. This is your project and you're just letting me help out.

I hope to show them that you can 'randomly' generate an entire planet in a matter of hours that has more depth and detail than a content designer could create in several months. This isn't saying that I want to eliminate the content designers, because I think that they still have some very important things that they can do, but to make it so that they aren't having to create all their content ex nihilo. This would free them up so that they can focus on making a better polished product.

Wow, that's all a bit of a mouthful, so let me summarize. Yes, I work in the game industry. No, I'm not working on this because I want to incorporate the code into some finished project, but Yes, I may show this project to people as an example of something that I think should be done that is not currently being done by the industry (assuming people are ok with that).

[esampson]

Ok. First, I updated the scatter plot code just a little bit. The results now color 'bad' squares grey.

I'm almost kicking myself for not thinking to create this code before and make a better check of things. Now that I've got this tool I've sat down and played with the precipitation and temperature generation routines and I've been able to adjust the curves to much better results (I'll submit those modifications on a separate pull request). Those let me generate the following new plot for seed 1 Thanks to the much better coverage of temperature and humidity zones I created the following world using seed_2. Way better biome placement, I think. Nice steppes (green textured areas), chaparral (light brown textured area) and savanna (yellowish texture) now that I wasn't seeing before. A couple of moderately good sized hot deserts as well (some of the dune textures, although cool deserts use those as well).

[esampson]

No idea why my latest submit would be failing. The error is with the ancient_map blessed images, but none of my submits have touched that (and it was passing earlier).

[tcld]

This PR https://github.com/Mindwerks/worldengine/pull/146 required an update of the ancient maps. Since I uploaded them (and until https://github.com/Mindwerks/worldengine/pull/146 is merged) the ancient map-tests will fail. Sorry!

You should start looking into a proper library so you don't have to redo too much stuff later.

[ftomassetti]

I really believe in an incremental approach: I think in the long run we may want to use some plotting libraries but this code delivers value right now, so we could merge it and later refactor it. Perhaps we will add a few more plots before understanding what is the best plotting library we should use. Or perhaps users will hate the idea of having plots and force us to remove it :)

About showing the work or use it in even commercial applications: the license permits you to do so. Even people who never contributed would have the right to do so. If anything I would be immensely honored if you would even consider using WorldEngine.

If you want to show this project around that would be amazing and if you want to give us suggestions about things which would make the project more useful (according to your experience), we would be thrilled to hear about that.

I am currently a freelance so I can find some spare time to work on it and for me it could be a nice way to promote myself. World generation is something I find very interesting even if I have more experience working on other things (I got a PhD working on DSL, MDD, mixing languages and stuff like that, later I worked on large web applications at TripAdvisor and Groupon).

About content generation, I had another project, civs, which was about creating civilizations living on worlds generated by WorldEngine. I wanted to end up having a sort of "history generator", so that games could benefit from a rich background and realistic worlds in a broad sense (realistic ways for population to spread, for languages to mix, for countries to grow and die). I plan to go back working on it in the feature.

[ftomassetti]

unless @psi29a has something against it I would go on and merge it

[esampson]

I know that the license allows me to show the project but I'm funny about that kind of thing. Even though I'm covered legally I like to make sure that other people know if I'm going to do something like that.

[ftomassetti]

I appreciate that, thank you. As I said I would see only advantages for me if this project could get any attention :)

[tcld]

I finally took a close look at this. It looks like it will function well and like it would be a very interesting addition to worldengine :) - but I had to criticize the lacking use of numpy at some points. I really don't want to fight an uphill battle and have to add a PR of my own to every PR, especially since the numpy replacements here seem to be fairly simple. If you need any help adding some numpy-code, I will happily try to give it. It is really not that hard to get into - and once it is done it will be an extremely useful tool (most likely for every Python-project ever^^).

[esampson]

Now with comments to explain what's going on and replacing lots of loops with numpy.

[ftomassetti]

ok, it will need to merge master into this branch or rebase this brach upon master. Do you need help with that? Do you want me to do that for you?

[tcld]

I assume you know how to rebase this and solve the merge-conflicts?

[esampson]

Working on that now.

[esampson]

Merged in the changes for the updated Master.

[tcld]

I didn't go over every line again but it looks good to me. Thanks for investing the effort of dealing with all my comments. :)

[esampson]

No problem at all. Thanks for taking the time to review it and find those things I missed.

After this push I will work on adding command line options for adjusting the temperature/precipitation curve and when I do that I will see what I can do about making those other variables optional.

After that I think I will be out of the code for a bit, for the most part. I may look at some new icons for the ancient map but otherwise I think I will be mostly documenting.