Add data collection

[jakobaxelsson]

Support is needed for data collection and visualization. A rudimentary first version has been implemented. The approach taken is as follows:

• Declare the state variables to be available for data collection using annotations, e.g. Annotate[int, "State"] in the subclasses of core.Entity.
• Create a mesa data collector by finding all such classes, and setting them up for data collection.
• A configuration parameter is provided for turning data collection on or off, both for batch and interactive execution.
• It is possible to view the collected data in the user interface through a menu choice.

The current implementation has several drawbacks or limitations:

• [x] It collects the same attributes for all agent types. However, this is not appropriate since not all agent types have all the attributes, and therefore there are a lot of None values in the resulting tables. Instead, there should be a data collection for each class, and it should be possible to select in the user interface which class to show. It is not obvious how to do this in a good way using mesa's standard datacollector, so it might be necessary to create a custom one.
• [x] There is a bug in the user interface. When the data is viewed, and then going back to one of the other views, it is not displayed correctly.
• [x] Functionality is needed for saving the data. Both as a menu choice in the user interface, and by specifying a file in the batch mode. Alternatively, there could be an option to instead specify a zip file, to which both the configuration and the data is written. In that way, it becomes clear what the data was and how it was created.
• [ ] Make it possible to select what data to save. Primarily, this would be a user interface functionality, since it might be too complex to specify on the command line. However, an option is to add a configuration file parameter for the batch mode. In that way, it could be possible to create the configuration in the UI while selecting the desired data, and save it to use for batch simulations. Alternatively, the JSON configuration file could be edited manually.

[jakobaxelsson]

UI bug fixed. Problem was that the simulation view had initially style attribute "display: grid". When switching to data view, this became "display: none", and when switching back, it became "display: block". Solution was to add an extra div level above these, and doing the show/hide on that level.

[jakobaxelsson]

Improved data collection to use a separate table for each agent class. Updated user interface so that user interface can choose which table to see.

[jakobaxelsson]

With a data collector that has data in several tables, it is less obvious how to save it to a file for easy processing. Mesa provides functionality for getting each table as a data frame, and it would be possible to save that as a csv file. Then, one could make a zip file of all the tables (and maybe also add the configuration to that file, for reference).

However, to make this work both in browser and the normal file system, it might be necessary to create the zip file in memory first like this:

import io
import zipfile

zip_buffer = io.BytesIO()

with zipfile.ZipFile(zip_buffer, "a",
                     zipfile.ZIP_DEFLATED, False) as zip_file:
    for file_name, data in [('1.txt', io.BytesIO(b'111')),
                            ('2.txt', io.BytesIO(b'222'))]:
        zip_file.writestr(file_name, data.getvalue())

content = zip_buffer.getvalue()

The content can then be passed to the UI save_file_as function or to a normal file write in batch mode.

[jakobaxelsson]

File save and load is now re-implemented. The main file format is a zip file which contains as a minimum a manifest file with metainformation and a configuration file. If any data has been collected, this is also added with one separate csv file for each class. If profiling was enabled (in batch mode) the output of that is added.

Reading and writing files has been implemented both in batch (through command line flags) and interactive mode. Currently, the purpose of reading a file is to reuse a configuration. This accepts either a zip archive that contains the configuration file, or just the configuration file on its own.