Closed hyunjimoon closed 2 years ago
Leaving debugged outputs to get the sense of what predictor, outcome, lookup_function, function_name, stock_initial_value looks like for inventory management example.
Q4. Why is lookup function dict and datastructure set?
lookup_function_names: Dict[str, str]
datastructure_function_names: Set[str]
Q1 what was the problem?
The two options were
I actually think the first method may be possible. In fact, we can just create a generic interpolate
stan function like so:
real vector_interpolate(vector v, real time){
real slope = v[ceil(time)] - v[floor(time)];
real intercept = v[floor(time)];
return intercept + slope * time % 1; # equivantly, fmod(time, 1)
}
And then if we need to linear interpolate given a data vector D
, we can just call vector_interpolate(D, time)
.
Q2 role of normal type?
I don't know. Apparently from your screenshot it's some option in Vensim.
Q3: could you explain how you used ReferenceStructure for topological sort (e.g. code with brief comment)?
ReferenceStructure
is the way auxillary variable assignment statements use another auxillary variable on the RHS. The topological sorting of auxillary variables is needed because stan programs are sequential, meaning we need x, y
to exist in the program state before executing z = x + y
.
Topological sort can be implemented trivially by doing a depth first search on a graph. The graph that arouses from a Vensim model is a dependency graph. The dependency graph is a directed graph between variables in expressions, such that you have edges going from LHS variables to RHS. So the above example would have two edges: z -> x, z -> y
.
The code that creates the graph is here: https://github.com/Data4DM/BayesSD/blob/b474f62e51531a4757115c423f4b35669dee6469/ContinuousCode/5_BayesCalib/stanify/builders/stan/stan_block_builder.py#L304 For every element node in AST, it traverses through its RHS and finds any auxillary variables that the element node depends on.
Once you have a graph, you can do the topological sort using DFS. I made a generic class that handles storing a dependency graph and sorting: https://github.com/Data4DM/BayesSD/blob/b474f62e51531a4757115c423f4b35669dee6469/ContinuousCode/5_BayesCalib/stanify/builders/stan/utilities.py#L25
add_stmt(A, B)
will add an edge going from A to B. The important function is recursive_order_search
. It's a simple DFS implementation. It checks if child nodes are present in a list, and will append it if it's not in it. Unless you have cyclic assignment statements(so something weird like a = b+ c, c = a + b
), you will get the correct order in which auxillary variables should be defined in Stan.
**********
name: Production Rate bf P Noise
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['*', '*', '*'] (ReferenceStructure(reference='production_rate', subscripts=None), ReferenceStructure(reference='p_noise_scale', subscripts=None), ArithmeticStructure(operators=['^'], arguments=(ArithmeticStructure(operators=['/'], arguments=(ArithmeticStructure(operators=['-'], arguments=(2, ReferenceStructure(reference='dt_tc', subscripts=None))), ReferenceStructure(reference='dt_tc', subscripts=None))), 0.5)), ReferenceStructure(reference='ran_norm2', subscripts=None))
**********
name: Production Start Rate bf P Noise
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['*', '*', '*'] (ReferenceStructure(reference='production_start_rate', subscripts=None), ReferenceStructure(reference='p_noise_scale', subscripts=None), ArithmeticStructure(operators=['^'], arguments=(ArithmeticStructure(operators=['/'], arguments=(ArithmeticStructure(operators=['-'], arguments=(2, ReferenceStructure(reference='dt_tc', subscripts=None))), ReferenceStructure(reference='dt_tc', subscripts=None))), 0.5)), ReferenceStructure(reference='ran_norm1', subscripts=None))
**********
name: P Noise Scale
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
0.1
**********
name: Process Corr Time
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
3
**********
name: Change in Exp Orders
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ArithmeticStructure(operators=['-'], arguments=(ReferenceStructure(reference='customer_order_rate', subscripts=None), ReferenceStructure(reference='expected_order_rate', subscripts=None))), ReferenceStructure(reference='time_to_average_order_rate', subscripts=None))
**********
name: Customer Order Rate
length: 1
type: Data
subtype: Normal
subscript: ([], [])
DataStructure
**********
name: Production Rate af P Noise Change Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ArithmeticStructure(operators=['-'], arguments=(ReferenceStructure(reference='production_rate_bf_p_noise', subscripts=None), ReferenceStructure(reference='production_rate_af_p_noise', subscripts=None))), ReferenceStructure(reference='process_corr_time', subscripts=None))
**********
name: dt Tc
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ReferenceStructure(reference='time_step', subscripts=None), ReferenceStructure(reference='process_corr_time', subscripts=None))
**********
name: Production Start Rate af P Noise
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
IntegStructure:
ReferenceStructure:
production_start_rate_af_p_noise_change_rate
,
ReferenceStructure:
production_start_rate
**********
name: Production Start Rate af P Noise Change Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ArithmeticStructure(operators=['-'], arguments=(ReferenceStructure(reference='production_start_rate_bf_p_noise', subscripts=None), ReferenceStructure(reference='production_start_rate_af_p_noise', subscripts=None))), ReferenceStructure(reference='process_corr_time', subscripts=None))
**********
name: Ran Norm1
length: 1
type: Data
subtype: Normal
subscript: ([], [])
DataStructure
**********
name: Order Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ReferenceStructure:
customer_order_rate
**********
name: Expected Order Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
IntegStructure:
ReferenceStructure:
change_in_exp_orders
,
ReferenceStructure:
customer_order_rate
**********
name: Ran Norm4
length: 1
type: Data
subtype: Normal
subscript: ([], [])
DataStructure
**********
name: Production Rate af P Noise
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
IntegStructure:
ReferenceStructure:
production_rate_af_p_noise_change_rate
,
ReferenceStructure:
production_rate
**********
name: Production Rate with MP Noise
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['*'] (ReferenceStructure(reference='production_rate_af_p_noise', subscripts=None), ReferenceStructure(reference='ran_norm4', subscripts=None))
**********
name: Ran Norm3
length: 1
type: Data
subtype: Normal
subscript: ([], [])
DataStructure
**********
name: Ran Norm2
length: 1
type: Data
subtype: Normal
subscript: ([], [])
DataStructure
**********
name: Prodcution Start Rate with MP Noise
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['*'] (ReferenceStructure(reference='production_start_rate_af_p_noise', subscripts=None), ReferenceStructure(reference='ran_norm3', subscripts=None))
**********
name: Production Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ReferenceStructure(reference='work_in_process_inventory', subscripts=None), ReferenceStructure(reference='manufacturing_cycle_time', subscripts=None))
**********
name: Adjustment for WIP
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ArithmeticStructure(operators=['-'], arguments=(ReferenceStructure(reference='desired_wip', subscripts=None), ReferenceStructure(reference='work_in_process_inventory', subscripts=None))), ReferenceStructure(reference='wip_adjustment_time', subscripts=None))
**********
name: Adjustment from Inventory
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ArithmeticStructure(operators=['-'], arguments=(ReferenceStructure(reference='desired_inventory', subscripts=None), ReferenceStructure(reference='inventory', subscripts=None))), ReferenceStructure(reference='inventory_adjustment_time', subscripts=None))
**********
name: Backlog
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
IntegStructure:
ArithmeticStructure:
['-'] (ReferenceStructure(reference='order_rate', subscripts=None), ReferenceStructure(reference='order_fulfillment_rate', subscripts=None)),
ArithmeticStructure:
['*'] (ReferenceStructure(reference='order_rate', subscripts=None), ReferenceStructure(reference='target_delivery_delay', subscripts=None))
**********
name: Desired Inventory
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['*'] (ReferenceStructure(reference='desired_inventory_coverage', subscripts=None), ReferenceStructure(reference='expected_order_rate', subscripts=None))
**********
name: Desired Inventory Coverage
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['+'] (ReferenceStructure(reference='minimum_order_processing_time', subscripts=None), ReferenceStructure(reference='safety_stock_coverage', subscripts=None))
**********
name: Desired Production
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
CallStructure:
ReferenceStructure:
max
(
0
,
ArithmeticStructure:
['+'] (ReferenceStructure(reference='expected_order_rate', subscripts=None), ReferenceStructure(reference='adjustment_from_inventory', subscripts=None)))
**********
name: Desired Production Start Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['+'] (ReferenceStructure(reference='desired_production', subscripts=None), ReferenceStructure(reference='adjustment_for_wip', subscripts=None))
**********
name: Desired Shipment Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ReferenceStructure(reference='backlog', subscripts=None), ReferenceStructure(reference='target_delivery_delay', subscripts=None))
**********
name: Desired WIP
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['*'] (ReferenceStructure(reference='manufacturing_cycle_time', subscripts=None), ReferenceStructure(reference='desired_production', subscripts=None))
**********
name: Inventory
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
IntegStructure:
ArithmeticStructure:
['-'] (ReferenceStructure(reference='production_rate', subscripts=None), ReferenceStructure(reference='shipment_rate', subscripts=None)),
ReferenceStructure:
desired_inventory
**********
name: Inventory Adjustment Time
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
8
**********
name: Manufacturing Cycle Time
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
8
**********
name: Maximum Shipment Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['/'] (ReferenceStructure(reference='inventory', subscripts=None), ReferenceStructure(reference='minimum_order_processing_time', subscripts=None))
**********
name: Minimum Order Processing Time
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
2
**********
name: Order Fulfillment Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ReferenceStructure:
shipment_rate
**********
name: Order Fulfillment Ratio
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
CallStructure:
ReferenceStructure:
table_for_order_fulfillment
(
ArithmeticStructure:
['/'] (ReferenceStructure(reference='maximum_shipment_rate', subscripts=None), ReferenceStructure(reference='desired_shipment_rate', subscripts=None)))
**********
name: Production Start Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
CallStructure:
ReferenceStructure:
max
(
0
,
ReferenceStructure:
desired_production_start_rate
)
**********
name: Safety Stock Coverage
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
2
**********
name: Shipment Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ArithmeticStructure:
['*'] (ReferenceStructure(reference='desired_shipment_rate', subscripts=None), ReferenceStructure(reference='order_fulfillment_ratio', subscripts=None))
**********
name: Table for Order Fulfillment
length: 1
type: Lookup
subtype: Hardcoded
subscript: ([], [])
LookupStructure (interpolate):
x (0, 2) = (0.0, 0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0, 2.0)
y (0, 1) = (0.0, 0.2, 0.4, 0.58, 0.73, 0.85, 0.93, 0.97, 0.99, 1.0, 1.0, 1.0)
**********
name: Target Delivery Delay
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
2
**********
name: Time to Average Order Rate
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
8
**********
name: WIP Adjustment Time
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
2
**********
name: Work in Process Inventory
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
IntegStructure:
ArithmeticStructure:
['-'] (ReferenceStructure(reference='production_start_rate', subscripts=None), ReferenceStructure(reference='production_rate', subscripts=None)),
ReferenceStructure:
desired_wip
**********
name: FINAL TIME
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
100
**********
name: INITIAL TIME
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
0
**********
name: SAVEPER
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
ReferenceStructure:
time_step
**********
name: TIME STEP
length: 1
type: Auxiliary
subtype: Normal
subscript: ([], [])
0.0625
Initial stock value which refers the existing data structure should have been recorded. Resolved by writing f'{dataFunc_'}(0) stan code to the first value of stock
plot of stock value
In commit https://github.com/Data4DM/BayesSD/commit/ea6c3588f0ab8867ca31d7a6a46836f58827c010 We included
DataStructureCodeGenWalker
class, which unlikeLookupStructureCodeGenWalker
doesn't need name dictionary as AST node comes with its name as below which is the output of printing element and component with this script.There were two implementation options: vector
times
vstime
scalar as an input. We tried the first, but due to [Q1 what was the problem? ] we ended up with the second. New things I learned are:ReferenceStructure
@Dashadower On top of the two questions above, Q3: could you explain how you used
ReferenceStructure
for topological sort (e.g. code with brief comment)? Not priority but I think we could benefit from some documentation within functions. To be specific, comparing AST node namedInventory
and vensim interface teaches us howIntegStructure
,ArithmeticStructure
,ReferenceStructure
map toINTEG
,-
, using variable in vensim. However, my brain is not following the mechanism of topological sort to create dependency graph (construction and walking on it?)