Dynamic programming for pipe design

[barneydobson]

The current method of pipe-by-pipe design seems to provide a highly local solution - it's messing with my simulation metrics.

Duque et al. suggest that dynamic programming is applicable for such a problem. I think it would give a global solution - which is attractive. Their suggestion is that it is computationally exhausting to do, however, I think my current implementation in pipe_by_pipe already performs the appropriate number of computations (since the pipe design is evaluated for every depth for every pipe, which ultimately is all you would need for dynamic programming... I think..).

Before I do this there are a couple things to test:

• [ ] Look at sensitivity analysis results (running now anyway) and see which parameters are most sensitive. I guess the design parameters being so would point to this being a potentially useful solution.
• [ ] Use the real subcatchments and real topology and run the pipe-by-pipe design to verify that the resultant simulations are not great. For me this is a strong indicator that it would need improvement.
• [ ] Inspect the design process in detail - doing this on #156 .

My rough outline for adjusting to a dynamic programming approach:

• Still evaluate every pipe for every depth. But instead of retaining the best (in terms of feasibility and cost) pipe overall, we retain the best pipe for each depth. This depth:cost relationship is our value function.
• When evaluating pipes immediately downstream, include the cost of the upstream pipe by evaluating its value function for the depth that would be required for this pipes design in the cost of each design.

This is just a rough outline - don't worry if you don't follow my DP approach, I have plenty of practice at it and I'm pretty sure I could get it to work. It's more the big picture stuff above to check whether you agree with. @cheginit and @tijanajovanovic it'd be great if you could sanity check my thinking here ;)

[cheginit]

I think overall it's reasonable, but my main concern is the “cost” definition. I think an important point to consider is that this package is not meant to be used for designing a BUSN from scratch, like many other existing solutions. Instead, it's meant to derive an existing BUSN that were built and evolved based on urban design changes. So, we're looking for a hydraulically feasible design based on the current infrastructure and I think the current direction is deviating from this idea: We want to take advantage of the existing publicly available data to derive a BUSN as computationally inexpensive as possible. So instead of directly considering monetary cost, we use indirect measures, such as their proximity to highly populated areas, urban functions, and land cover. I'd argue that this is the novelty of the approach, since there exists a slew of literature on sewer design solutions.

[barneydobson]

Thanks @cheginit that makes sense - I guess this question is more linked with #151 and should be considered in the wider context of optimization and why someone would be doing it in their case study - but you're right, that's not really the case that we're considering.

Additionally - I am testing about the design method on the 'real' topology in #156 and the results actually look sensible, so probably the issue is not with the design.

Furthermore, I have removed the hydraulic structures from the 'real' network model to see how good the performance is and it is much better. Since we do not account for hydraulic structures there is no reason this method could feasibly produce good results in places where they are important (which the literature suggests is many), thus - I think in terms of method performance we're nearly there and I don't need to fiddle around with complicated design methods ;) Though I have made an issue, #158 about hydraulic structures in case we want to discuss at another time (certainly not this paper!)

[cheginit]

Right, I think any hydraulically feasible design that's based on an accurate topology should give reasonably accurate results. So, by more focusing on getting the topology right, we should get more bang for our bucks!