A long term maintenance goal.

[m312z]

This would require some form of decay in the system, such as buildings becoming damaged (or simply disappearing).

Relevant for multiagent systems and goal reasoning systems.

[m312z]

I converted @Roedhip's suggestion into an issue for discussion.

The important point is what differentiates maintenance goals from online goals. That is, new building goals that come in over time with associated deadlines and rewards. The agents must choose which to construct, and whether to abandon current projects for easier or more rewarding ones.

[Roedhip]

Maintenance goals and online goals are very similar, but a few differences I can think of are:

• Maintenance goals can lead to subgoals which cannot be ignored, while typical online goals are just a matter of optimising growth.
• In this growth based domain, typical online goals lead to growth in the agents' abilities. This would not be the case in a maintenance scenario (if agents cannot build new buildings and must maintain an initial set of buildings, this domain could model a maintenance scenario without growth).
• Similarly, access to resources and planning ahead with limited resources is more relevant in a maintenance scenario than in a scenario based on constant growth.

[m312z]

In this growth based domain, typical online goals lead to growth in the agents' abilities. This would not be the case in a maintenance scenario (if agents cannot build new buildings and must maintain an initial set of buildings, this domain could model a maintenance scenario without growth).

I was discussing with Ludwig in the design document - I think that while some special buildings provide a passive benefit to the team, those buildings requested by online goals would typically be randomised ingredient buildings with no effect other than increasing score. It makes for a clear decision point between the ultimate goal (score) and improving abilities.

[m312z]

Maintenance goals can lead to subgoals which cannot be ignored, while typical online goals are just a matter of optimising growth.

Given that the online goals don't necessarily modify the agent's abilities, if some could be tagged as "critical" goals that must be completed or the final score would be zero - would that model a maintenance goal?

I get the feeling that there is still another difference - perhaps in that the online goals cannot be predicted?

[Roedhip]

If the domain can be limited so that the agents can't improve their own abilities while working to meet online goals, and we're able to grant weights to different online goals, that seems like it does cover most of the key aspects of maintenance. Though one aspect of maintenance which I hadn't thought of earlier is the process of inspection leading the identification of new goals.

An ideal maintenance scenario would probably have online goals which are hidden and some process by which the agents can look to discover these goals. This does add a whole new action type with quite a bit if supporting framework, plus it would require a good control of which online goals are likely to appear so that some understanding of the possible online goals is passed to the solver. This might be too much to add.

[Roedhip]

Order of importance for maintenance goals (note, getting maintenance-like goals isn't the most important extension of the model):

• Online goals
• Online goals with deadlines and penalties for failure, penalty may be complete failure
• No exponential growth (toggle off buildings that give bonuses)
• No growth (no ability to spend your spare time mining and stockpiling resources. This could be achieved by turning off mines and adding starting resources. It could also be achieved by limiting access to mines. It could also be achieved by turning off mines and adding mechanics for resources to spawn as goals appear).
• Hidden goals achieved by partial observability (the world is partially observable, including goals)
• Hidden goals without partially observable world (the only partially observable part of the world is the goals)

[m312z]

An update on Mark's comments.

Online goals Online goals with deadlines and penalties for failure, penalty may be complete failure

This is done.

No exponential growth (toggle off buildings that give bonuses)

The growth can be set to 0. Agents can ignore the special building recipes if the scenario does not warrant it - I think most of the time the online goals is interesting enough.

No growth (no ability to spend your spare time mining and stockpiling resources. This could be achieved by turning off mines and adding starting resources. It could also be achieved by limiting access to mines. It could also be achieved by turning off mines and adding mechanics for resources to spawn as goals appear).

A set of starting resources would be good. This should be a separate card in the projects.

Hidden goals achieved by partial observability (the world is partially observable, including goals) Hidden goals without partially observable world (the only partially observable part of the world is the goals)

Partial observability is what Ludwig works on now. I think what you describe could be configured.

[Oxbeefbabe]

In regards to starting resources and mines: In the world_gen_modifiers initialisation file, you can set a number of mines and resources of each colour. What you are asking for here can be achieved by using this. However, with the green resource, this might not work well, since they will quickly decay and there is no other way to gather more. Adjusting mining so that no more than x number of actors can mine at it can be added pretty easily. The same goes for limits on how many resources can be on the floor. However, with this, there must still be at least one space available on the floor since mines will drop resources on the floor, instead of the actor's inventory.

I might go through and collect a bunch of small "tweaks" similar to those described above and add them into the program in one go, instead of pushing them with larger adjustments.

The way I'm working on partial observability, each separate type of entity, excluding actors, (tasks are also considered entities) can have their observability toggled independently from other entities. What you can described can be done.

[m312z]

Ah, I missed that you could set the starting resources.

@Oxbeefbabe Is this on the floor or in inventory (either is fine)?

Also, can the resource properties be switched off? If so, you could create a very simple version of the simulation.

[Oxbeefbabe]

For now it is just on the floor. If it was in actor inventory the initialisation files would become mess (unless each actor had an equal amount of starting resources)

Resource properties cannot be switched off per say but you can change the parameters to essentially have no effect. (Red: cycle length = 100, collection intervals = 0, 100; blue: blue extra effort modifier = 1, orange num of actors to mine = 1, black cant be turned off as its property is hard coded, green cant be turned off but the length to decay can be so long that it is essentially infinite)