search

KLayout / klayout

KLayout Main Sources
http://www.klayout.org
GNU General Public License v3.0
794 stars 205 forks source link

very slow width_check in DRC flat mode on a small layout #1194

Closed atorkmabrains closed 1 year ago

atorkmabrains commented 1 year ago

Hi @klayoutmatthias ,

I'm still working on investigating the performance issues in DRC that we are facing. I ran another profiling to see where is the main issue:

2022-11-22 21:36:11 +0200: Memory Usage (7321292K) : DRC Total Run time 556.591414 seconds
  %   cumulative   self              self     total
 time   seconds   seconds    calls  ms/call  ms/call  name
 42.15   232.23    232.23       14 16587.62 16587.62  RBA::Edges#width_check
 13.18   304.86     72.63       48  1513.22  1513.22  RBA::Region#complex_op
  4.97   332.24     27.38       66   414.83   414.83  RBA::Region#not_inside
  4.76   358.45     26.21       70   374.49   374.49  RBA::Edges#&
  4.54   383.45     25.00      175   142.85   142.85  RBA::Region#not_interacting
  3.49   402.69     19.23       88   218.56   218.56  RBA::Edges#interacting
  2.63   417.16     14.47       46   314.55   314.55  RBA::Region#space_check
  2.52   431.03     13.87      195    71.14    71.14  RBA::Region#-
  2.41   444.28     13.25       81   163.59   163.59  RBA::Region#not_outside
  1.92   454.86     10.58      153    69.17    69.17  RBA::Region#interacting
  1.89   465.27     10.41      159    65.48    65.48  RBA::Region#|
  1.68   474.51      9.24       79   116.93   116.93  RBA::Region#enclosing_check
  1.55   483.05      8.54       55   155.32   155.32  RBA::Region#sized
  1.48   491.20      8.14      146    55.77    55.77  RBA::Region#&
  1.22   497.94      6.74      283    23.81    23.81  RBA::Region#inside
  1.16   504.35      6.41       41   156.43   156.43  RBA::Edges#with_length
  1.04   510.07      5.72       26   219.84   219.84  RBA::Edges#enclosing_check
  0.84   514.70      4.64      243    19.08    19.08  RBA::Region#edges
  0.84   519.31      4.61       11   418.94   418.94  RBA::Edges#enclosed_check
  0.82   523.85      4.54       68    66.74    66.74  RBA::Edges#-
  0.80   528.24      4.39       14   313.66   313.66  RBA::EdgePairs#edges
  0.57   531.36      3.12      149    20.94    20.94  RBA::Region#outside
  0.48   534.00      2.64      106    24.91    24.91  RBA::Region#merged
  0.44   536.42      2.42       94    25.70    25.70  RBA::Region#separation_check
  0.32   538.15      1.74       15   115.82   115.82  RBA::Edges#separation_check
  0.25   539.51      1.36       15    90.35    90.35  RBA::Region#corners
  0.24   540.82      1.31       28    46.88    46.88  RBA::Edges#centers
  0.21   541.99      1.17       35    33.45    33.45  RBA::Edges#extended
  0.20   543.11      1.12        6   185.92   185.92  RBA::Region#with_bbox_min
  0.17   544.02      0.92        3   305.57   305.57  RBA::Edges#with_angle
  0.15   544.83      0.81       19    42.44    42.44  RBA::Region#covering
  0.13   545.56      0.73       10    72.59    72.59  RBA::Edges#not_in
  0.09   546.03      0.47       41    11.44    11.44  RBA::Region#width_check
  0.08   546.48      0.45        1   450.54   450.54  RBA::Edges#in
  0.08   546.92      0.45    27555     0.02     0.06  DRC::DRCEngine#src_line
  0.07   547.33      0.41       12    34.03    34.03  RBA::Edges#not_interacting
  0.06   547.64      0.31    56698     0.01     0.01  Kernel#!~
  0.04   547.85      0.21       23     8.99     8.99  RBA::EdgePairs#first_edges
  0.03   548.02      0.18     4212     0.04   130.27  DRC::DRCEngine#run_timed
  0.03   548.20      0.17       11    15.85    15.85  RBA::EdgePairs#with_length_both
  0.02   548.33      0.13    23536     0.01    26.06  DRC::DRCEngine#_context
  0.02   548.45      0.13    36806     0.00     0.01  DRC::DRCLayer#data
  0.02   548.58      0.13       22     5.73     5.73  RBA::Edges#|
  0.02   548.70      0.13      106     1.18     1.18  RBA::Region#with_angle
  0.02   548.83      0.12       22     5.58     5.58  RBA::EdgePairs#second_edges
  0.02   548.94      0.11    56698     0.00     0.00  String#=~
  0.02   549.05      0.11     8426     0.01     0.02  DRC::DRCEngine#_process_events
  0.02   549.16      0.11     4735     0.02     0.18  Array#each
  0.02   549.26      0.09      620     0.15     0.15  Kernel#`
  0.02   549.35      0.09      345     0.26     0.26  RBA::EdgePairs#polygons
  0.02   549.44      0.09        4    21.40    21.40  RBA::Edges#space_check
  0.01   549.52      0.08     5996     0.01   181.42  DRC::DRCEngine#_tcmd
  0.01   549.60      0.08       70     1.17     1.17  RBA::Region#overlapping
  0.01   549.68      0.08      106     0.72     0.72  RBA::Region#grid_check
  0.01   549.74      0.06     4212     0.02     0.02  Kernel#caller
  0.01   549.80      0.06     4214     0.01     0.04  DRC::DRCEngine#info
  0.01   549.86      0.06     9020     0.01     0.01  Class#new
  0.01   549.91      0.04    29048     0.00     0.00  Kernel#is_a?
  0.01   549.95      0.04       19     2.25     2.25  RBA::Region#holes
  0.01   549.99      0.04        1    37.38    37.38  RBA::Layout#read
  0.01   550.02      0.04     1229     0.03     0.04  DRC::DRCEngine#_prep_value
  0.01   550.06      0.04    16852     0.00     0.00  RBA::Application.instance
  0.01   550.09      0.03      620     0.05     0.23  DRC::DRCExecutable#execute
  0.01   550.12      0.03     1796     0.02     0.04  DRC::DRCLayer#requires_edges_or_region
  0.01   550.15      0.03     4214     0.01     0.01  RBA::Logger.log
  0.00   550.18      0.03     4212     0.01     0.19  Enumerable#find
  0.00   550.20      0.02       30     0.78     0.78  RBA::Region#with_area
  0.00   550.22      0.02     1105     0.02     0.05  DRC::DRCLayer#requires_same_type
  0.00   550.24      0.02      690     0.03     0.07  DRC::DRCEngine#_output
  0.00   550.26      0.02       12     1.70     1.70  RBA::Region#extents
  0.00   550.28      0.02      482     0.04   124.50  DRC::DRCLayer#interacting
  0.00   550.30      0.02      620     0.03     0.32  Logger#add
  0.00   550.31      0.01      521     0.03    57.76  DRC::DRCLayer#enclosing
  0.00   550.33      0.01      534     0.03    15.92  DRC::DRCLayer#separation
  0.00   550.34      0.01      526     0.03    70.70  DRC::DRCLayer#-
  0.00   550.36      0.01     8426     0.00     0.00  RBA::Application#process_events
  0.00   550.37      0.01      807     0.02     0.04  DRC::DRCLayer#requires_edges_texts_or_region
  0.00   550.38      0.01     4213     0.00     0.00  RBA::Timer#stop
  0.00   550.40      0.01     1380     0.01     0.24  DRC::DRCEngine#_vcmd
  0.00   550.41      0.01      432     0.03   159.83  DRC::DRCLayer#&
  0.00   550.42      0.01     1014     0.01     0.53  DRC::DRCLayer#polygons
  0.00   550.43      0.01     4213     0.00     0.00  File.basename
  0.00   550.44      0.01     1380     0.01     0.43  DRC::DRCLayer#output
  0.00   550.46      0.01      566     0.02    24.40  DRC::DRCLayer#inside
  0.00   550.47      0.01      374     0.03   136.39  DRC::DRCLayer#not_interacting
  0.00   550.48      0.01      620     0.02     0.02  Time#to_s
  0.00   550.49      0.01     4212     0.00     0.00  RBA::Timer#start
  0.00   550.50      0.01     1240     0.01     0.03  Logger::LogDevice#write
  0.00   550.51      0.01     1046     0.01     9.44  DRC::DRCEngine#_cmd
  0.00   550.52      0.01     2546     0.00     0.00  RBA::Region#enable_progress
  0.00   550.53      0.01     4212     0.00     0.00  RBA::Timer#initialize
  0.00   550.54      0.01      861     0.01     0.06  DRC::DRCEngine#_make_value
  0.00   550.54      0.01      514     0.02    35.67  DRC::DRCLayer#edges
  0.00   550.55      0.01      690     0.01     0.01  RBA::ReportDatabase#create_category
  0.00   550.56      0.01      298     0.02    21.51  DRC::DRCLayer#outside
  0.00   550.57      0.01      362     0.02    58.85  DRC::DRCLayer#|
  0.00   550.57      0.01     4657     0.00     0.00  Kernel#class
  0.00   550.58      0.01     4214     0.00     0.00  String#*
  0.00   550.59      0.01      324     0.02     6.74  DRC::DRCLayer#with_angle
  0.00   550.59      0.01      690     0.01     0.01  RBA::RdbCategory#scan_collection
  0.00   550.60      0.01      941     0.01     0.01  DRC::DRCEngine#_check_numeric
  0.00   550.61      0.01      913     0.01     0.01  DRC::DRCLayer#check_is_layer
  0.00   550.61      0.01      620     0.01     0.01  IO#write
  0.00   550.62      0.01      216     0.03  2154.96  DRC::DRCLayer#width
  0.00   550.62      0.01     2546     0.00     0.00  RBA::Region#disable_progress
  0.00   550.63      0.01     3516     0.00     0.00  DRC::DRCLayer#initialize
  0.00   550.63      0.00      620     0.01     0.02  Monitor#synchronize
  0.00   550.64      0.00      622     0.01     0.01  Time.now
  0.00   550.64      0.00      221     0.02    10.91  DRC::DRCLayer#extended
  0.00   550.65      0.00      166     0.02   103.35  DRC::DRCLayer#sized
  0.00   550.65      0.00      620     0.01     0.24  Logger#format_message
  0.00   550.65      0.00      200     0.02   145.87  DRC::DRCLayer#space
  0.00   550.66      0.00      390     0.01     0.02  DRC::DRCLayer#requires_edge_pairs
  0.00   550.66      0.00      620     0.01     0.03  MonitorMixin#mon_synchronize
  0.00   550.67      0.00      620     0.01     0.32  Logger#info
  0.00   550.67      0.00      379     0.01     0.02  DRC::DRCLayer#requires_region
  0.00   550.67      0.00      524     0.01    70.99  DRC::DRCLayer#not
  0.00   550.68      0.00      212     0.02     1.31  DRC::DRCLayer#ongrid
  0.00   550.68      0.00      420     0.01   158.81  DRC::DRCLayer#and
  0.00   550.68      0.00      162     0.02   164.13  DRC::DRCLayer#not_outside
  0.00   550.69      0.00       96     0.03  1513.95  DRC::DRCLayer#drc
  0.00   550.69      0.00      140     0.02     1.66  DRC::DRCLayer#overlapping
  0.00   550.69      0.00     1888     0.00     0.00  DRC::DRCEngine#dbu
  0.00   550.69      0.00      463     0.01     0.04  Array#collect
  0.00   550.70      0.00      212     0.01    25.21  DRC::DRCLayer#merged
  0.00   550.70      0.00      132     0.02   415.38  DRC::DRCLayer#not_inside
  0.00   550.70      0.00      690     0.00     0.00  RBA::CplxTrans#initialize
  0.00   550.70      0.00      163     0.01    51.30  DRC::DRCLayer#with_length
  0.00   550.70      0.00      454     0.00     0.00  RBA::Edges#enable_progress
  0.00   550.71      0.00     1142     0.00     0.00  Float#floor
  0.00   550.71      0.00        1     2.10     2.10  TracePoint#enable
  0.00   550.71      0.00      362     0.01    58.86  DRC::DRCLayer#or
  0.00   550.71      0.00      690     0.00     0.00  RBA::RdbCategory#description=
  0.00   550.72      0.00      620     0.00     0.00  String#[]
  0.00   550.72      0.00        1     1.85     1.85  RBA::ReportDatabase#save
  0.00   550.72      0.00       44     0.04   209.97  DRC::DRCLayer#enclosed
  0.00   550.72      0.00      415     0.00     0.00  RBA::EdgePairs#enable_progress
  0.00   550.72      0.00      212     0.01     0.40  DRC::DRCSource#polygons
  0.00   550.72      0.00      283     0.01     0.01  DRC::DRCEngine#euclidian
  0.00   550.73      0.00     1145     0.00     0.00  Integer#to_i
  0.00   550.73      0.00       81     0.02     0.13  DRC::DRCOpNode#create_node
  0.00   550.73      0.00      341     0.00     0.00  RBA::Region::Metrics#==
  0.00   550.73      0.00       47     0.03     0.17  DRC::DRCOpNodeCheck#do_create_node
  0.00   550.73      0.00      528     0.00     0.00  DRC::DRCLayer#minmax_count
  0.00   550.73      0.00      621     0.00     0.00  String#+
  0.00   550.73      0.00      161     0.01     0.02  Array#select
  0.00   550.73      0.00      622     0.00     0.00  Time#initialize
  0.00   550.74      0.00        1     1.26     1.26  RBA::Region#insert
  0.00   550.74      0.00       66     0.02     0.09  DRC::DRCEngine#_cop_enclosed
  0.00   550.74      0.00      620     0.00     0.00  Process.pid
  0.00   550.74      0.00      622     0.00     0.00  Integer#to_s
  0.00   550.74      0.00      620     0.00     0.00  String#strip
  0.00   550.74      0.00      212     0.01     0.43  DRC::DRCEngine#polygons
  0.00   550.74      0.00       30     0.04     0.04  DRC::DRCEngine#_prep_value_area
  0.00   550.74      0.00      690     0.00     0.00  String#to_s
  0.00   550.75      0.00      454     0.00     0.00  RBA::Edges#disable_progress
  0.00   550.75      0.00      107     0.01     0.02  DRC::DRCSource#parse_input_layers
  0.00   550.75      0.00       60     0.02     1.27  DRC::DRCLayer#with_area
  0.00   550.75      0.00      107     0.01     0.02  DRC::DRCEngine#_input
  0.00   550.75      0.00      620     0.00     0.00  Logger#format_severity
  0.00   550.75      0.00      415     0.00     0.00  RBA::EdgePairs#disable_progress
  0.00   550.75      0.00       45     0.02    60.61  DRC::DRCLayer#corners
  0.00   550.75      0.00       56     0.02    47.48  DRC::DRCLayer#centers
  0.00   550.75      0.00       38     0.02    43.04  DRC::DRCLayer#covering
  0.00   550.75      0.00      620     0.00     0.00  Kernel#block_given?
  0.00   550.75      0.00      620     0.00     0.00  Integer#round
  0.00   550.76      0.00       46     0.02     9.49  DRC::DRCLayer#first_edges
  0.00   550.76      0.00       88     0.01     0.04  DRC::DRCEngine#_make_value_with_nil
  0.00   550.76      0.00       32     0.02     0.06  DRC::DRCEngine#_make_node
  0.00   550.76      0.00      106     0.01     0.02  DRC::DRCLayer#count
  0.00   550.76      0.00       69     0.01     0.02  DRC::DRCLayer#requires_edges
  0.00   550.76      0.00      485     0.00     0.00  Float#um
  0.00   550.76      0.00       52     0.01     0.03  DRC::DRCLayer#requires_edges_or_edge_pairs
  0.00   550.76      0.00       42     0.02   116.12  DRC::DRCLayer#without_length
  0.00   550.76      0.00      335     0.00     0.00  DRC::DRCMetrics#initialize
  0.00   550.76      0.00       20     0.03     0.41  DRC::DRCLayer#overlap
  0.00   550.76      0.00       33     0.02     0.02  RBA::CompoundRegionOperationNode.new_enclosed_check
  0.00   550.76      0.00       44     0.01     0.20  DRC::DRCEngine#enclosed
  0.00   550.76      0.00       47     0.01     0.03  DRC::DRCOpNodeCheck#initialize
  0.00   550.76      0.00       44     0.01     6.12  DRC::DRCLayer#second_edges
  0.00   550.77      0.00       24     0.02    61.18  DRC::DRCLayer#not_in
  0.00   550.77      0.00        1     0.58     0.58  RBA::Layout#delete_layer
  0.00   550.77      0.00       58     0.01     0.04  DRC::DRCEngine#_make_area_value_with_nil
  0.00   550.77      0.00      126     0.00     0.00  RBA::Region::OppositeFilter#!=
  0.00   550.77      0.00       28     0.02     0.05  DRC::DRCComparable#<=
  0.00   550.77      0.00       38     0.01     2.73  DRC::DRCLayer#holes
  0.00   550.77      0.00      214     0.00     0.00  DRC::DRCEngine#layout
  0.00   550.77      0.00       25     0.02     0.06  DRC::DRCComparable#<
  0.00   550.77      0.00       70     0.01     0.01  Kernel#dup
  0.00   550.77      0.00       72     0.01     0.01  Float#==
  0.00   550.77      0.00       70     0.01     0.01  Kernel#initialize_dup
  0.00   550.77      0.00      128     0.00     0.00  RBA::CompoundRegionOperationNode#result_type
  0.00   550.77      0.00       18     0.02     0.34  DRC::DRCLayer#size
  0.00   550.77      0.00        8     0.05     0.69  DRC::DRCLayer#not_overlapping
  0.00   550.77      0.00       59     0.01     0.02  DRC::DRCComparable#_self_or_original
  0.00   550.77      0.00       30     0.01     0.09  DRC::DRCEngine#width
  0.00   550.77      0.00       50     0.01     0.02  DRC::DRCEngine#projection
  0.00   550.77      0.00      268     0.00     0.00  Array#push
  0.00   550.77      0.00       28     0.01     0.19  DRC::DRCLayer#isolated
  0.00   550.77      0.00       15     0.02     0.07  DRC::DRCEngine#_cop_width
  0.00   550.77      0.00       12     0.03     0.44  DRC::DRCLayer#inside_part
  0.00   550.77      0.00       48     0.01     0.01  DRC::DRCOpNodeWithCompare#initialize
  0.00   550.77      0.00      126     0.00     0.00  RBA::Region::RectFilter#!=
  0.00   550.78      0.00       16     0.02     0.11  DRC::DRCEngine#_cop_separation
  0.00   550.78      0.00      106     0.00     0.00  RBA::Layout#find_layer
  0.00   550.78      0.00       52     0.01     0.01  Integer#um
  0.00   550.78      0.00       11     0.03     0.04  DRC::DRCEngine#both
  0.00   550.78      0.00       14     0.02     0.54  DRC::DRCLayer#not_covering
  0.00   550.78      0.00       28     0.01     0.01  DRC::DRCComparable#set_lt
  0.00   550.78      0.00      127     0.00     0.00  RBA::CompoundRegionOperationNode::ResultType#==
  0.00   550.78      0.00      108     0.00     0.00  RBA::Region#initialize
  0.00   550.78      0.00       17     0.02     0.02  RBA::CompoundRegionOperationNode.new_width_check
  0.00   550.78      0.00      163     0.00     0.00  Kernel#object_id
  0.00   550.78      0.00       24     0.01     2.24  DRC::DRCLayer#extents
  0.00   550.78      0.00       32     0.01     0.01  RBA::CompoundRegionOperationNode.new_secondary
  0.00   550.78      0.00       81     0.00     0.00  DRC::DRCOpNode#initialize
  0.00   550.78      0.00        7     0.04     0.04  RBA::Region#isolated_check
  0.00   550.78      0.00       35     0.01     0.36  Enumerable#each_with_index
  0.00   550.78      0.00        1     0.23     0.23  RBA::AbstractProgress#_destroy
  0.00   550.78      0.00       16     0.01     0.01  RBA::CompoundRegionOperationNode.new_edges
  0.00   550.78      0.00       22     0.01     0.01  RBA::RecursiveShapeIterator#initialize
  0.00   550.78      0.00       11     0.02     0.10  Float#<=
  0.00   550.78      0.00       12     0.02   187.51  DRC::DRCLayer#with_bbox_min
  0.00   550.78      0.00       64     0.00     0.00  DRC::DRCComparable#_check_bounds
  0.00   550.78      0.00        7     0.03     0.03  RBA::CompoundRegionOperationNode.new_separation_check
  0.00   550.78      0.00       14     0.01     0.39  DRC::DRCLayer#with_holes
  0.00   550.78      0.00       14     0.01     0.18  DRC::DRCEngine#separation
  0.00   550.78      0.00      122     0.00     0.00  Integer#abs
  0.00   550.78      0.00        4     0.04    18.87  DRC::DRCEngine#source
  0.00   550.78      0.00       12     0.01     0.02  DRC::DRCEngine#_make_numeric_value_with_nil
  0.00   550.78      0.00        1     0.17     0.17  RBA::MacroExecutionContext.remove_debugger_scope
  0.00   550.78      0.00       13     0.01     0.02  DRC::DRCComparable#set_ge
  0.00   550.78      0.00      106     0.00     0.00  RBA::Region#count
  0.00   550.78      0.00        1     0.16     3.52  DRC::DRCEngine#_finish
  0.00   550.78      0.00      109     0.00     0.00  Range#begin
  0.00   550.78      0.00       19     0.01     0.01  DRC::DRCComparable#set_le
  0.00   550.78      0.00        8     0.02   113.05  DRC::DRCLayer#in
  0.00   550.78      0.00       13     0.01     0.01  RBA::CompoundRegionOperationNode.new_geometrical_boolean
  0.00   550.78      0.00      107     0.00     0.00  RBA::Cell#cell_index
  0.00   550.78      0.00      108     0.00     0.00  Range#end
  0.00   550.78      0.00        3     0.04     0.04  RBA::CompoundRegionOperationNode.new_join
  0.00   550.78      0.00       48     0.00     0.00  RBA::CompoundRegionOperationNode#distance
  0.00   550.78      0.00        1     0.13     3.82  DRC::DRCExecutable#cleanup
  0.00   550.78      0.00       17     0.01     0.01  RBA::CompoundRegionOperationNode.new_edge_pair_to_first_edges
  0.00   550.78      0.00        2     0.06     0.14  DRC::DRCComparable#==
  0.00   550.78      0.00        2     0.06     0.06  RBA::LayoutView.current
  0.00   550.78      0.00        4     0.03     0.23  DRC::DRCEngine#enclosing
  0.00   550.78      0.00        6     0.02     0.09  DRC::DRCEngine#_cop_enclosing
  0.00   550.78      0.00       56     0.00     0.00  Kernel#respond_to?
  0.00   550.78      0.00       47     0.00     0.00  Symbol#to_s
  0.00   550.78      0.00       92     0.00     0.00  Float#to_f
  0.00   550.78      0.00       74     0.00     0.00  Integer#to_f
  0.00   550.78      0.00       70     0.00     0.00  Kernel#initialize_copy
  0.00   550.78      0.00       79     0.00     0.00  BasicObject#==
  0.00   550.78      0.00        4     0.02     0.02  RBA::CompoundRegionOperationNode.new_enclosing_check
  0.00   550.78      0.00       11     0.01     0.02  DRC::DRCComparable#coerce
  0.00   550.78      0.00       23     0.00     0.00  RBA::Layout#cell
  0.00   550.78      0.00       11     0.01     0.02  DRC::DRCEngine#joined
  0.00   550.79      0.00        2     0.04     0.67  DRC::DRCSource#finish
  0.00   550.79      0.00        1     0.08     0.39  DRC::DRCOpNodeEdgeLengthFilter#do_create_node
  0.00   550.79      0.00        6     0.01     0.46  DRC::DRCLayer#non_rectangles
  0.00   550.79      0.00        3     0.02     0.05  DRC::DRCComparable#!=
  0.00   550.79      0.00        2     0.03     0.03  RBA::CompoundRegionOperationNode.new_overlap_check
  0.00   550.79      0.00        3     0.02     0.44  DRC::DRCLayer#without_angle
  0.00   550.79      0.00        4     0.02     0.02  RBA::Region#overlap_check
  0.00   550.79      0.00        2     0.03     0.21  DRC::DRCEngine#length
  0.00   550.79      0.00        1     0.06     0.06  RBA::CompoundRegionOperationNode.new_edge_length_filter
  0.00   550.79      0.00        3     0.02     0.09  DRC::DRCEngine#_cop_overlap
  0.00   550.79      0.00        2     0.03     0.74  DRC::DRCEngine#_flush
  0.00   550.79      0.00        1     0.06     0.28  DRC::DRCEngine#_cleanup
  0.00   550.79      0.00       33     0.00     0.00  DRC::DRCOpNode#do_create_node
  0.00   550.79      0.00       22     0.00     0.00  RBA::ICplxTrans#initialize
  0.00   550.79      0.00        7     0.01     0.01  RBA::Region#with_holes
  0.00   550.79      0.00        1     0.05     0.29  DRC::DRCEngine#_make_path
  0.00   550.79      0.00        4     0.01     0.01  RBA::Region#not_overlapping
  0.00   550.79      0.00        7     0.01     0.01  RBA::Region#not_covering
  0.00   550.79      0.00        2     0.02     0.13  DRC::DRCEngine#report
  0.00   550.79      0.00       22     0.00     0.00  RBA::RecursiveShapeIterator#global_dtrans=
  0.00   550.79      0.00        7     0.01     0.01  Integer#==
  0.00   550.79      0.00        3     0.01     1.33  DRC::DRCLayer#insert
  0.00   550.79      0.00       12     0.00     0.00  String#%
  0.00   550.79      0.00       22     0.00     0.00  RBA::RecursiveShapeIterator#shape_flags=
  0.00   550.79      0.00        1     0.04     0.14  DRC::DRCEngine#_cop_length
  0.00   550.79      0.00       25     0.00     0.00  RBA::Layout#dbu
  0.00   550.79      0.00        2     0.02     2.06  DRC::DRCSource#extent
  0.00   550.79      0.00        2     0.02     0.21  DRC::DRCEngine#overlap
  0.00   550.79      0.00        6     0.01     0.01  RBA::Region#size
  0.00   550.79      0.00        1     0.03     0.03  RBA::ReportDatabase#initialize
  0.00   550.79      0.00        2     0.01     0.29  DRC::DRCLayer#area
  0.00   550.79      0.00        3     0.01     0.01  RBA::Region#non_rectangles
  0.00   550.79      0.00        1     0.03     0.03  File.absolute_path
  0.00   550.79      0.00       11     0.00     0.00  DRC::DRCJoinFlag#initialize
  0.00   550.79      0.00        1     0.02     0.07  DRC::DRCComparable#>
  0.00   550.79      0.00        3     0.01     0.02  DRC::DRCEngine#transparent
  0.00   550.79      0.00        4     0.01     0.01  DRC::DRCComparable#set_gt
  0.00   550.79      0.00        6     0.00     0.00  RBA::Edges#inside_part
  0.00   550.79      0.00        2     0.01     0.64  DRC::DRCSource#input
  0.00   550.79      0.00        1     0.02     0.02  RBA::ReportDatabase#create_cell
  0.00   550.79      0.00        1     0.02     0.05  DRC::DRCEngine#make_source
  0.00   550.79      0.00        1     0.02     0.07  DRC::DRCOpNode#length
  0.00   550.79      0.00        1     0.02     0.08  Logger#initialize
  0.00   550.79      0.00        1     0.02     0.02  RBA::Layout#initialize
  0.00   550.79      0.00        2     0.01     0.02  DRC::DRCLayer#forget
  0.00   550.79      0.00        1     0.02     0.04  DRC::DRCEngine#polygon_layer
  0.00   550.79      0.00        2     0.01     0.02  DRC::DRCEngine#square
  0.00   550.79      0.00        1     0.02     0.04  DRC::DRCEngine#primary
  0.00   550.79      0.00        6     0.00     0.00  DRC::DRCEngine#is_tiled?
  0.00   550.79      0.00        2     0.01     2.09  DRC::DRCEngine#extent
  0.00   550.79      0.00        2     0.01     0.12  DRC::DRCEngine#_tdcmd
  0.00   550.79      0.00        1     0.01     0.01  Time#-
  0.00   550.79      0.00        1     0.01     0.02  DRC::DRCEngine#use_dbu
  0.00   550.79      0.00        1     0.01     0.03  DRC::DRCOpNodeEdgeLengthFilter#initialize
  0.00   550.79      0.00       12     0.00     0.00  BasicObject#initialize
  0.00   550.79      0.00        3     0.00     0.00  RBA::Region#in
  0.00   550.79      0.00        1     0.01     0.06  DRC::DRCEngine#_make_area_value
  0.00   550.79      0.00        1     0.01     0.04  Logger::LogDevice#initialize
  0.00   550.79      0.00        1     0.01     0.01  Logger::LogDevice#set_dev
  0.00   550.79      0.00        3     0.00     0.00  DRC::DRCShielded#initialize
  0.00   550.79      0.00        2     0.00     0.00  RBA::Region#not_in
  0.00   550.79      0.00        1     0.01     0.02  MonitorMixin#mon_initialize
  0.00   550.79      0.00        1     0.01     0.01  RBA::CompoundRegionOperationNode.new_primary
  0.00   550.79      0.00        1     0.01     0.74  Hash#each
  0.00   550.79      0.00        1     0.01     0.01  RBA::LayerInfo#initialize
  0.00   550.79      0.00        1     0.01     0.01  RBA::Cell#bbox
  0.00   550.79      0.00        1     0.01     0.02  DRC::DRCSource#initialize
  0.00   550.79      0.00        1     0.01     0.01  RBA::Layout#insert_layer
  0.00   550.79      0.00        1     0.01     0.01  Logger#level=
  0.00   550.79      0.00        1     0.01     0.01  RBA::Cell#name
  0.00   550.79      0.00        1     0.01     0.01  DRC::DRCEngine#verbose
  0.00   550.79      0.00        1     0.01     0.01  RBA::Region#_destroy
  0.00   550.79      0.00        1     0.01     0.01  Kernel#proc
  0.00   550.79      0.00        1     0.00     0.01  DRC::DRCEngine#threads
  0.00   550.79      0.00        1     0.00     0.00  RBA::DBox#transformed
  0.00   550.79      0.00        2     0.00     0.00  RBA::DBox#*
  0.00   550.79      0.00        3     0.00     0.00  DRC::DRCShielded#value
  0.00   550.79      0.00        1     0.00     0.00  RBA::Region#each
  0.00   550.79      0.00        1     0.00     0.00  RBA::CompoundRegionOperationNode::ResultType#!=
  0.00   550.79      0.00        3     0.00     0.00  Integer#*
  0.00   550.79      0.00        1     0.00     0.00  RBA::DBox.from_ibox
  0.00   550.79      0.00        1     0.00     0.00  RBA::Region#area
  0.00   550.79      0.00        1     0.00     0.00  RBA::DCplxTrans#initialize
  0.00   550.79      0.00        1     0.00     0.00  File.dirname
  0.00   550.79      0.00        1     0.00     0.00  Float#to_s
  0.00   550.79      0.00        1     0.00     0.00  RBA::Box.from_dbox
  0.00   550.79      0.00        1     0.00     0.00  RBA::ReportDatabase#generator=
  0.00   550.79      0.00        1     0.00     0.00  RBA::ReportDatabase#description=
  0.00   550.79      0.00        1     0.00     0.00  String#to_i
  0.00   550.79      0.00        1     0.00     0.00  Logger#datetime_format=
  0.00   550.79      0.00        1     0.00     0.00  RBA::ReportDatabase#top_cell_name=
  0.00   550.79      0.00        1     0.00     0.00  DRC::DRCEngine#flat
  0.00   550.79      0.00        2     0.00     0.00  BasicObject#singleton_method_added
  0.00   550.79      0.00        1     0.00     0.00  DRC::DRCSource#path
  0.00   550.79      0.00        1     0.00     0.00  DRC::DRCEngine#_before_cleanup
  0.00   550.79      0.00        1     0.00     0.00  TracePoint#disable
  0.00   550.79      0.00        1     0.00     0.00  TrueClass#to_s
  0.00   550.79      0.00        1     0.00     0.00  Monitor#mon_owned?
  0.00   550.79      0.00        1     0.00     0.00  Logger::Formatter#initialize
  0.00   550.79      0.00        1     0.00     0.00  Float._dbu=
  0.00   550.79      0.00        1     0.00     0.00  Integer._dbu=
  0.00   550.79      0.00        1     0.00     0.00  DRC::DRCEngine#_generator
  0.00   550.79      0.00        1     0.00     0.00  DRC::DRCEngine#verbose=
  0.00   550.79      0.00        2     0.00     0.00  DRC::DRCSource#layout
  0.00   550.79      0.00        1     0.00     0.00  DRC::DRCSource#cell_obj
  0.00   550.79      0.00        1     0.00     0.00  Integer#fdiv
  0.00   550.95      0.00        1     0.00 550954.64  #toplevel
22-Nov-2022 21:36:15 | INFO    | 
Congratulations !!. No DRC Violations found in user_proj_example for gf180mcu.drc rule deck with switch gfC
22-Nov-2022 21:36:15 | INFO    | Klayout GDS DRC Clean

The layout was not that big. I don't recommend running tiled version as it gives false errors.

I don't recommend using Google version as it's not updated. Please use Mabrains version as it's updated.

To duplicate this run, you need the following:

Here is the run command:

python run_drc.py --gf180mcu=C --path=./user_proj_example.gds

From my initial analysis, I do believe the main bottleneck most likely lies in the "do_process" function here: https://github.com/KLayout/klayout/blob/327345f5ca86e3d5f4490d199e586c7bf5d4599d/src/db/db/dbBoxScanner.h#L304

I'm not sure if that's true or not. But it seems to be iterative process.

atorkmabrains commented 1 year ago

Also, I believe you don't do pruning for edges for example for width_check here: https://github.com/KLayout/klayout/blob/7a9e9989d3fb8259d9ee3b5d6c8fe803092e7f2b/src/db/db/dbAsIfFlatEdges.cc#L695

Would it be possible to prune the edges to the ones that are in the same polygon only: https://github.com/KLayout/klayout/blob/7a9e9989d3fb8259d9ee3b5d6c8fe803092e7f2b/src/db/db/dbAsIfFlatEdges.cc#L705

klayoutmatthias commented 1 year ago

@atorkmabrains Thanks for the analysis. My experience with profiling is a mixed blessing. IMHO it's much better to break down a test case. Once you have the bottleneck operation, you can try to understand what it is doing.

I don't see how long the different steps take. But from my experiences with the other cases, I think the problem is in the MDN.3b and MDN.4b rules. These contain an edge width operation and I think this is where your observed runtime penalty comes from. Here is a clip from my log:

...
2022-11-23 22:24:19 +0100: Memory Usage (5215992K) : Executing rule MDN.2b
2022-11-23 22:24:19 +0100: Memory Usage (5215992K) : Executing rule MDN.3a
2022-11-23 22:24:20 +0100: Memory Usage (5215992K) : Executing rule MDN.3b
2022-11-23 22:26:12 +0100: Memory Usage (6318536K) : Executing rule MDN.4a
2022-11-23 22:26:12 +0100: Memory Usage (6318536K) : Executing rule MDN.4b
2022-11-23 22:26:12 +0100: Memory Usage (6318536K) : Executing rule MDN.5ai

The core implementation of MDN.3b is this:

mdn3b_l1 = poly2.edges.and(ncomp).or(mvsd_mdn).and(ldmos_xtor).and(dualgate).not(ngate.not(mvsd).edges.interacting(poly2.edges.and(ncomp).or(mvsd_mdn)).width(20.001.um).edges)

I reduced the DRC deck to this piece and tried to unfold the long expressions:

verbose

source("user_proj_example.gds")
target("test_out.gds")

comp           = polygons(22 , 0 )
nplus          = polygons(32 , 0 )
pplus          = polygons(31 , 0 )
poly2          = polygons(30 , 0 )
mvsd           = polygons(210, 0 )
ldmos_xtor     = polygons(226, 0 )
dualgate       = polygons(55 , 0 )

ncomp      =  comp      & nplus
tgate      =  poly2     & comp
ngate      =  nplus     & tgate

mvsd_mdn = mvsd.
            edges.
            and(ncomp).
            and(poly2)

mdn3b_l1 = poly2.
            edges.
            and(ncomp).
            or(mvsd_mdn).
            and(ldmos_xtor).
            and(dualgate).
            not(ngate.
                  not(mvsd).
                  edges.
                  interacting(poly2.
                                edges.
                                and(ncomp).
                                or(mvsd_mdn)
                             ).
                  width(20.001.um).
                  edges
             )

Running that with klayout -b -r reduced.drc shows me the individual operations. The long runner is "width(20.001.um)" with 116s (of a total of 123s) as your profiling indicates too.

This is an edge collection width operation. The remark about pruning is correct. But the edges don't know about the polygons they come from. Hence no pruning with respect to original can be done. I tried to explain this problem earlier. An edge set is a collection of disconnected edges. "width" is not referring to "inner distance" but only to a certain relative edge orientation indicative.

This gets visible when the inputs (red) and outputs (green) of "width" are plotted:

image

The green mess are the actual edge pairs produced by "width"! Not kidding. This is just a tiny piece. This whole output is a plethora of >8M polygons. This does not make sense at all.

A "width" operation is much more efficient when used on polygons. In that case, only individual polygons are considered and pruning in your sense is implied. But I think this is not applicable here as the computation for the inputs are already done in edge domain.

An elaborate solution was a "edge width inside polygon" feature. That is not available (yet), but I think it can be emulated:

The base polygon for this check is ngate.not(mvsd). This means all relevant edges are on the boundary of these polygons. So instead of width-checking the edges, we could first width-check the base region (which is usually a rectangle, so this check is trivial) and then mask the sensitive edges with those candidates.

Like this:

ngate_wo_mvsd = ngate.not(mvsd)
wide_ngate = ngate_wo_mvsd.drc(width(projection) <= 20.um).edges

mdn3b_l1 = poly2.
            edges.
            and(ncomp).
            or(mvsd_mdn).
            and(ldmos_xtor).
            and(dualgate).
            not(ngate_wo_mvsd.
                  edges.
                  interacting(poly2.
                                edges.
                                and(ncomp).
                                or(mvsd_mdn)
                             ).
                  and(wide_ngate)
            )

I don't have a test case that triggers this check, so correctness still needs to be confirmed. But in my case that executes in a few seconds.

I guess there are even smarter ways to solve this. For example, the "drc" function basically creates a local context which so that basically also provides the pruning you ask for. However, without a test case - specifically one with mvsd devices - this will be difficult to optimize.

And finally a last word regarding tiled mode: I think you need to supply a border that takes these long range interactions into account. It should basically be done automatically, but I'm not sure if long ranges are captured correctly. As the maximum distance encountered in the deck is 50µm, trying with a border of 50µm should be a first guess. However, that is quite a large and a noticeable overhead will be present.

In any case, flat mode is prohibitive for large logic blocks or memory arrays. And tiled or deep mode are the only modes supporting multiple cores.

Best regards,

Matthias

klayoutmatthias commented 1 year ago

Here is another remark because I just noticed that:

In the code above this expression is given (unfolded version of the original code):

mdn3b_l1 = poly2.
            edges.
            and(ncomp).
            or(mvsd_mdn).
            and(ldmos_xtor).
            and(dualgate).
            not(ngate_wo_mvsd.
                  edges.
                  interacting(poly2.
                                edges.
                                and(ncomp).
                                or(mvsd_mdn)
                             ).
                  and(wide_ngate)
            )

I contains the following subexpression two times:

poly2.
   edges.
   and(ncomp).
   or(mvsd_mdn)

This expression is actually computed twice, so it can be optimized:

ngate_wsides = poly2.edges.and(ncomp).or(mvsd_mdn)

mdn3b_l1 = ngate_wsides.
            and(ldmos_xtor).
            and(dualgate).
            not(ngate_wo_mvsd.
                  edges.
                  interacting(ngate_wsides).
                  and(wide_ngate)
            )

I think there are many such optimization opportunities. But again, difficult without a test case (or better: a test suite).

Matthias

atorkmabrains commented 1 year ago

@klayoutmatthias Thanks for the code review. We have a test suite for this rule deck. I'll send you the link.

I'll take a look at the coding of this rule and see if we can change the style of coding avoiding the use of edge with. BTW, in the commercial tool it keeps track of the edge from which polygon it comes from. I thought you might be doing something similar.

As for tiling, we have used double the maximum measurement in the rule deck as border, and yet it generated weird results. Let me test it again.

Deep didn't improve the runtime either.

klayoutmatthias commented 1 year ago

Tiling is not necessarily a runtime improvement, but it keeps memory low as it only works on one tile and with multiple cores, tiles are run in parallel. However there is some overhead (because of border work example) and hence there is some trade-off. But I have hardly encountered a case when tiling creates artefacts and core scaling usually is very good.

If you work on larger designs - specifically with memory blocks - you cannot use flat mode. Memory footprint will simply explode.

Matthias

atorkmabrains commented 1 year ago

@klayoutmatthias Thanks for your input. I believe. I need to redo many parts of the script as I believe we use edges then width/separation check very often. It was a bad assumption on my part that you prune based on original polygon.

klayoutmatthias commented 1 year ago

@atorkmabrains I don't think you need to undo all - in fact only a few.

If the width you check is a small one, the edge width check is very efficient. Even if it does not consider the polygon it comes from it will do an efficient local clustering (scan line algorithm in flat case). So if all relevant edges are within the search range and the local clusters do not involve many foreign or irrelevant edges, there is no performance or memory penalty.

The issue with the large-range width check is that the local cluster will be very large and in case of dense layout you collect a huge neighbourhood for each input edge. This creates manifold interactions as we've seen in the green error markers before. The number of interactions roughly grows with the square of the check distance which explains the extreme runtime penalty.

My gut feeling is that for this technology node every distance below 1 or 2µm can safely be checked with edge operations (I assume you will not test against pathologic cases :) ).

Best regards,

Matthias

atorkmabrains commented 1 year ago

@klayoutmatthias that's really helpful information. I would consider this in our implementation.

atorkmabrains commented 1 year ago

Thanks @klayoutmatthias I'll close this issue as it's no longer needed.