Operators in Constraints

[chrispahm]

I would like to know if it was possible to include the option to add up properties of variables in the constraints section. Let's see an example: I have a scenario where a farmer wants to optimise his profits by evaluating the most profitable growing option: { "name": "Crop Rotation Problem", "optimize": "Gross Margin", "opType": "max", "constraints": { "field0": { "max": 1 }, "field1": { "max": 1 }, "Wheat": { "max": 10 } }, "variables": { "field00": { "field0": 1, "Wheat": 3.6, "Gross Margin": 3052.44 }, "field01": { "field0": 1, "Summer Wheat": 3.6, "Gross Margin": 1787.58 }, "field02": { "field0": 1, "Summer Barley": 3.6, "Gross Margin": 1789.65 }, "field10": { "field1": 1, "Corn": 11.9, "Gross Margin": 7243.830000000001 }, "field11": { "field1": 1, "Soybeans": 11.9, "Gross Margin": 4710.235 }, "field12": { "field1": 1, "Onions": 11.9, "Gross Margin": 46602.325000000004 }, }, "ints": { "field00": 1, "field01": 1, "field02": 1, "field10": 1, "field11": 1, "field12": 1, } }

Now for example imagine there was a policy restriction only allowing the farmer to grow a maximum of 75% of his entire acerage with the sum of Wheat and Summer Barley. To solve this, it would be nice if the Solver would be able to handle the following:

{ "name": "Crop Rotation Problem", "optimize": "Gross Margin", "opType": "max", "constraints": { "field0": { "max": 1 }, "field1": { "max": 1 }, "Wheat" + "Summer Barley": { "max": 10 } },...

Maybe there is a nicer and mathematically more appealing way to achieve this, but for now that's the only way I could think of. Thanks already for the awesome work!

[bchevalier]

Hi Toffi,

I think that the simplest way would be to add the contribution to a constraint that corresponds to the maximum total area (maxFields here) for all the variables that have an impact on the total area (all the variables in this case):

"constraints": {
   "maxFields": { "max": 10 },
   "field0": { "max": 1 },
   "field1": { "max": 1 },
   "Wheat": { "max": 10 } }
},
"variables": {
   "field00": { "maxFields": 1, "field0": 1, "Wheat": 3.6, "Gross Margin": 3052.44 },  }
   "field01": { "maxFields": 1, "field0": 1, "Summer Wheat": 3.6, "Gross Margin": 1787.58 },
   "field02": { "maxFields": 1, "field0": 1, "Summer Barley": 3.6, "Gross Margin": 1789.65 },
   "field10": { "maxFields": 1, "field1": 1, "Corn": 11.9, "Gross Margin": 7243.83 },
   "field11": { "maxFields": 1, "field1": 1, "Soybeans": 11.9, "Gross Margin": 4710.235 },
   "field12": { "maxFields": 1, "field1": 1, "Onions": 11.9, "Gross Margin": 46602.325 },
}

What do you think?

[chrispahm]

Nice, that makes a lot of sense! Though I'm still struggling with another concept related to this issue: the policy may force our example farmer to grow a minimum of 2 crops on his farm. In GAMS one could create a variable CropsUsed and check whether the sum of CropsUsed was greater than 2. Now I don't see a way to get this to match the setup above, I was thinking of assigning different values for each crop, but realised that it would not get me any further. Do you probably have an idea? Thanks a lot already!

[bchevalier]

I am not sure I understand. If you want to force a minimum number of fields, you can achieve it the following way:

"constraints": {
   "cropsUsed": { "min": 2, "max": 10 },
   "field0": { "max": 1 },
   "field1": { "max": 1 },
   "Wheat": { "max": 10 } }
},
"variables": {
   "field00": { "cropsUsed": 1, "field0": 1, "Wheat": 3.6, "Gross Margin": 3052.44 },  }
   "field01": { "cropsUsed": 1, "field0": 1, "Summer Wheat": 3.6, "Gross Margin": 1787.58 },
   "field02": { "cropsUsed": 1, "field0": 1, "Summer Barley": 3.6, "Gross Margin": 1789.65 },
   "field10": { "cropsUsed": 1, "field1": 1, "Corn": 11.9, "Gross Margin": 7243.83 },
   "field11": { "cropsUsed": 1, "field1": 1, "Soybeans": 11.9, "Gross Margin": 4710.235 },
   "field12": { "cropsUsed": 1, "field1": 1, "Onions": 11.9, "Gross Margin": 46602.325 },
}

Notice that I just renamed maxFields into cropsUsed and added a lower bound to the constraint.

[chrispahm]

Sorry for being unclear, I think the first example is too minified. The farmer can only grow one kind of plant on each of his fields. He has multiple options though, for field0 example he has 9 options (field00 - field 08), for field1 he has only 5 options. He would like to chose the combination of options leading to the highest gross margin, which in this case would be to grow Onions on both field0 and field1. But the policy requires him to grow at least 2 crops on his entire farm (or even 3 if he had more than 2 fields), so any other combination would be acceptable, like field0 Onions and field1 Winter Wheat. { "name": "Crop Rotation", "optimize": "Gross Margin", "opType": "max", "constraints": { "field0": { "max": 1 }, "feld1": { "max": 1 }, "Potato": { "max": 10 } }, "variables": { "field00": { "field": 1, "Sugar Beets": 11.9, "Gross Margin": 9915.792 }, "field01": { "field": 1, "Greening": 11.9, "Gross Margin": 11.9 }, "field02": { "field": 1, "Winter Wheat": 11.9, "Gross Margin": 10121.07 }, "field03": { "field": 1, "Winter Barley": 11.9, "Gross Margin": 8123.695000000001 }, "field04": { "field": 1, "Winter Rye": 11.9, "Gross Margin": 6995.218 }, "field05": { "field": 1, "Summer Barley": 11.9, "Gross Margin": 5935.2 }, "field06": { "field": 1, "Oats": 11.9, "Gross Margin": 7517.308 }, "field07": { "field": 1, "Triticale": 11.9, "Gross Margin": 6527.24 }, "field08": { "field": 1, "Onions": 11.9, "Gross Margin": 46494.196 }, "feld10": { "feld1": 1, "Summer Wheat": 2.8, "Gross Margin": 1389.98 }, "feld11": { "feld1": 1, "Summer Barley": 2.8, "Gross Margin": 1391.65 }, "feld12": { "feld1": 1, "Winter Rye": 2.8, "Gross Margin": 1639.4109999999998 }, "feld13": { "feld1": 1, "Triticale": 2.8, "Gross Margin": 1529.98 }, "feld14": { "feld1": 1, "Onions": 11.9, "Gross Margin": 46494.196 } }, "ints": { "field0": 1, "field1": 1, "field2": 1, "field3": 1, "field4": 1, "field5": 1, "field6": 1, "field7": 1, "field8": 1, "feld10": 1, "feld11": 1, "feld12": 1, "feld13": 1, "feld14": 1 } }

I hope that helps clarifying the issue, I've been thinking about it all day and it's driving me crazy. Thought about assigning prime numbers to each crop and then dividing the sum of them through each individual prime number (if the results was even, then just one crop would be grown), but a) this only works for 2 crops (not 3 or even more) and b) I still can't use operators in the constraint section. Thanks for helping me out!

[bchevalier]

Basically you want to get at least n different types of crops. The solution would be to use the "big M" method. In your example, the idea is to force diversity of crops to 2. Let's say there are only 3 types of crops b, r and o (for barley, rye and onion), we can create 3 binary variables y_b, y_r and y_o corresponding to whether each crop is grown or not (binary variable => value either 0 or 1).

Then we can create a constraint y_b + y_r + y_o >= 2 (diversity constraint), that will force our diversity to be at least 2.

Now, we need to force those binary variables to 0 or 1 when the correspond crops are grown. 2 constraints per type of crops are needed: one to restrain the binary variable to 0 and one to force it to 1:

• The restrain constraint is quite straightforward: y_o <= nb_onion_fields (can be rewritten 0 <= nb_onion_fields - y_o). This constraint ensures that if no onion field is grown, the associated binary variable will be 0.
• The force constraint is a tiny bit more tricky: nb_onion_fields <= y_o * M (can be rewritten nb_onion_fields - y_o * M <= 0). This constraint ensures that if onion is grown, the associated binary variable will be 1. Notice that the value for M should be big enough so that it does not constraint the number of onion fields you can grow.

Here is the json formulation for what I stated above (by the way, the simpler the example the better).

{
    "name": "Crop Rotation",
    "optimize": "Gross Margin",
    "opType": "max",
    "constraints": {
        "field0": {
            "max": 6
        },
        "feld1": {
            "max": 6
        },
        "cropsUsed": {
            "max": 10
        },
        "diversity": {
            "min": 2
        },
        "restrainOnion": {
            "min": 0
        },
        "restrainSummerBarley": {
            "min": 0
        },
        "restrainWinterRye": {
            "min": 0
        },
        "forceOnion": {
            "max": 0
        },
        "forceSummerBarley": {
            "max": 0
        },
        "forceWinterRye": {
            "max": 0
        }
    },
    "variables": {
        "growOnion": {
            "diversity": 1,
            "restrainOnion": -1,
            "forceOnion": -999
        },
        "growSummerBarley": {
            "diversity": 1,
            "restrainSummerBarley": -1,
            "forceSummerBarley": -999
        },
        "growWinterRye": {
            "diversity": 1,
            "restrainWinterRye": -1,
            "forceWinterRye": -999
        },
        "field00": {
            "field0": 1,
            "cropsUsed": 1,
            "Winter Rye": 11.9,
            "Gross Margin": 6995.218,
            "restrainWinterRye": 1,
            "forceWinterRye": 1
        },
        "field01": {
            "field0": 1,
            "cropsUsed": 1,
            "Summer Barley": 11.9,
            "Gross Margin": 5935.2,
            "restrainSummerBarley": 1,
            "forceSummerBarley": 1
        },
        "field02": {
            "field0": 1,
            "cropsUsed": 1,
            "Onions": 11.9,
            "Gross Margin": 46494.196,
            "restrainOnion": 1,
            "forceOnion": 1
        },
        "feld10": {
            "feld1": 1,
            "cropsUsed": 1,
            "Summer Barley": 2.8,
            "Gross Margin": 1391.65,
            "restrainSummerBarley": 1,
            "forceSummerBarley": 1
        },
        "feld11": {
            "feld1": 1,
            "cropsUsed": 1,
            "Winter Rye": 2.8,
            "Gross Margin": 1639.411,
            "restrainWinterRye": 1,
            "forceWinterRye": 1
        },
        "feld12": {
            "feld1": 1,
            "cropsUsed": 1,
            "Onions": 11.9,
            "Gross Margin": 46494.196,
            "restrainOnion": 1,
            "forceOnion": 1
        }
    },
    "ints": {
        "field00": 1,
        "field01": 1,
        "field02": 1,
        "feld10": 1,
        "feld11": 1,
        "feld12": 1,
    },
    "binaries": {
        "growOnion": 1,
        "growSummerBarley": 1,
        "growWinterRye": 1
    }
}

You can see that I decided to set M = 999 but M=10 would have been sufficient since no more than 10 crops can be grown.

[chrispahm]

Wow that perfectly solves my issue! Thanks so much for your advice, hope farmers will benefit from it someday 👍

[bchevalier]

no problem, let us know if you need further help

[chrispahm]

Sorry for coming back at you, but as I was implementing and checking the code I realised it wasn't solving as expected. I attached a MWE for recreating the issue, sorry for that it's so long. When solving the attached problem, the diversity constraint (diversity > 3) is fulfilled by growPotatoes having a value of 2, and growOnions having a value of 1, despite them being declared as binary variables. It looks like my declaration of these variables as binaries is being ignored somehow. Another constraint that restricts the amount of each pair of crops grown to 95% of the entire acerage also seems to be ignored (as this constraint should also enforce the diversity of crops to a minimum of 3). Thanks for your help once again in advance!

Update - Figured out that the binaries were not working because I was working in the wrong directory (duh!). Still the 95% issue persists, hopefully I'll find the issue to that too. crop_rotation.txt

Update 2 - Alright, fixed the issue with the 95% rule, but now I'm facing some interesting other issues. The attached example crop_rotation featuring 20 fields (consisting of 35 constraints, 100 variables and 100 integers) won't solve at all (calculated to be around 10h). Here, all options for all fields are equal. When the options per field are increased and vary per field (see crop_rotation2, with then 140 constraints, 140 variables and 140 integers), the model is solved within 14-20ms. Even if the number of fields is increased to 100 (220 constraints, 710 variables, 710 ints) in scenario 2, the solve time is still within 50-60ms. Could it be that in the second scenario the 95% rule does not become binding (because of the different options per field), thus saving computation time?

crop_rotation2.txt crop_rotation.txt

[bchevalier]

Sorry for the delay,

One possible reason why it won't solve for particular problem configurations is because the solver can be numerically unstable (and loop forever) but it's rare. It is a work in progress to make it stable but it takes a consequent amount of time. If that is the reason, you might just have to wait for the update (a reasonable ETA may be not before 3 months).

Is it important for that configuration to work? Does your work relying on the solver need to be ready soon?

[chrispahm]

Well thanks for having a look once again! The solver is used (as you already figured I guess) to optimize crop rotations for farmers considering different aspects. It's a university project, so it will be free of charge when available. Right now, I'm hoping to get it online some time next year, also releasing it on github then. If you're interested in the project or the way the tableau is created just drop me an email, as I don't feel the code is ready to be published yet.

[JWally]

@Toffi-123 Without digging too deep into it, what about something like this:

You have a farm that has 4 fields on it. You can only grow 1 crop per field. Your crop choices are:

1. Barley (not available in field 4 because ...)
2. Wheat
3. Summer Wheat
4. Soy (not available for field 3 because of ...)
5. Onions

You can only have 1 crop / field. Wheat and Barley cannot make up more than 75% of your crops

{
    name: "problem_2",
    opType: "max",
    optimize: "profit",
    constraints: {
        acres: {max: 4},
        a_limiter: {max: 3},
        f1: {max: 1},
        f2: {max: 1},
        f3: {max: 1},
        f4: {max: 1}
    },
    variables: {
        // Field 1 can grow these crops
        f1_barley: {acres: 1, a_limiter: 1, f1: 1, profit: ?},
        f1_wheat: {acres: 1, a_limiter: 1, f1: 1, profit: ?},
        f1_summer_wheat: {acres: 1, f1: 1, profit: ?},
        f1_soy: {acres: 1, profit: f1: 1, profit: ?},
        f1_onions: {acres: 1, f1: 1, profit: ?},

        // Field 2 can grow these crops
        f2_barley: {acres: 1, a_limiter: 1, f2: 1, profit: ?},
        f2_wheat: {acres: 1, a_limiter: 1, f2: 1, profit: ?},
        f2_summer_wheat: {acres: 1, f2: 1, profit: ?},
        f2_soy: {acres: 1, f1: 1, f2: 1, profit: ?},
        f2_onions: {acres: 1, f2: 1, profit: ?},

        // Field 3 can grow these crops...Notice Soy is Gone
        f3_barley: {acres: 1, a_limiter: 1, f3: 1,profit: ?},
        f3_wheat: {acres: 1, a_limiter: 1, f3: 1,profit: ?},
        f3_summer_wheat: {acres: 1, f3: 1, profit: ?},
        f3_onions: {acres: 1, f3: 1, profit: ?},

        // Field 4 can grow these crops...Notice Barley is Gone
        f4_wheat: {acres: 1, a_limiter: 1, f4: 1,profit: ?},
        f4_summer_wheat: {acres: 1, f4: 1, profit: ?},
        f4_soy: {acres: 1, f4: 1, profit: ?},
        f4_onions: {acres: 1, f4: 1, profit: ?},

    },
    ints: {
        f1_barley: 1,
        f2_barley: 1,
        ...
        ..
        .
        f4_onions: 1
    }
}

To prevent any combination of wheat and barley from making up more than 75% of your crops, I added an attribute to wheat and barley a_limiter which is set to 1. In the constraints, I set a max on a_limiter to 3 (3 fields out of 4 -> 75%).

In the variables section, I created every combination of allowable field-crop that's allowed. That way, you won't end up with field 3 having soy since f3_soy doesn't exist. I also gave each variable an attribute denoting which field it belongs to, set at 1. For example, for a variable in field 4, it has an "f4" attribute.

Because you give each variable a field attribute, you can set a max on that attribute to prevent multiple crops from showing up in that field.

Also, I set each variable to be an integer.

I haven't tested it, so I don't know if it'll work or not. Is this kind of what you had in mind? If it works, does that solve your issue?

[chrispahm]

Thanks for the contribution! Being new to GitHub I just realized that I should close this issue, as the original question was resolved by bchevalier's answer (using the big M-Method). The setup currently used is working similar to your approach, with some differences in the constraints to fulfill the requirements of the Common Agricultural Policy of the EU. The first issue is that each field is different in size, so restricting each crop production area to 75% of the total has to be done through the actual crop area grown. Assuming the total area would be 100, then Wheat: 75 Soy: 75 ... and in the variable field1_option1: {field1: 1, Wheat: 5.6, Profit: ?} field1_option2: {field1: 1, Soy: 5.6, Profit: ?} Then, each combination of crops cannot exceed 95% of the cropping area, therefore the following is added to the constraints WheatBarley: 95 WheatSoy: 95 ... and in the variables field1_option1: {field1: 1, Wheat: 5.6, WheatBarley: 5.6, WheatSoy: 5.6, ..., Profit: ?} field1_option2: {field1: 1, Soy: 5.6, WheatSoy: 5.6, SoyOnions: 5.6, ..., Profit: ?} As you can already see, the more growing options there are, the longer the constraints are going to be for each field. Right now, as described above, this seems to be the source of the issues whith really long solve times for problems where this 95% constraints become binding. To get a little closer to reality, even more constraints will be added, like available work hours during different times of the year, available machine hours and others. It would be perfect to achieve all this within JavaScript, as the available options for including GAMS are not as promising.