cqfn / pdd

Command line toolkit for collecting TODO markers from your code, known as Puzzle Driven Development (PDD)
https://www.0pdd.com
MIT License
76 stars 25 forks source link

[WIP] C-lang Extensions to increase parsing speed #194

Closed Doldrums closed 2 years ago

Doldrums commented 2 years ago

Description

Checklist of main steps:

✅ Conduct Profiling and Benchmarking

Preliminary Analysis was done via ruby-prof Benchmarking tool. Absence of Flamegraph support in ruby-prof force us to use stackprof Profiling tool.

✅ Reveal Bottlenecks through PDD.rb

`==================================
  Mode: cpu(1000)
  Samples: 109 (2.68% miss rate)
  GC: 10 (9.17%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
        75  (68.8%)          56  (51.4%)     PDD::Source#puzzles
        10   (9.2%)          10   (9.2%)     PDD::Puzzle#initialize
         7   (6.4%)           7   (6.4%)     (sweeping)
         6   (5.5%)           6   (5.5%)     #<Module:0x00007f99ab0292d8>.shellescape
        11  (10.1%)           6   (5.5%)     PDD::Sources#binary?
`

✅ Prioritization of TOP-5 Bottlenecks methods and division of responsibility by Extensions

In general, the obvious solution seems to start with the takeaway PDD::Source#puzzles from Ruby to C.

PDD::Source#puzzles (/Users/amoriodi/RubymineProjects/pdd/lib/pdd/source.rb:40)
  samples:    56 self (51.4%)  /     75 total (68.8%)
  callers:
     158  (  210.7%)  PDD::Source#puzzles
      75  (  100.0%)  PDD::VerboseSource#puzzles
  callees (19 total):
     158  (  831.6%)  PDD::Source#puzzles
      16  (   84.2%)  PDD::Source#puzzle
       3  (   15.8%)  Logger#info
  code:
                                  |    40  |     def puzzles
    3    (2.8%)                   |    41  |       PDD.log.info "Reading #{@path}..."
                                  |    42  |       puzzles = []
    2    (1.8%) /     2   (1.8%)  |    43  |       lines = File.readlines(@file, encoding: 'UTF-8')
   70   (64.2%)                   |    44  |       lines.each_with_index do |line, idx|
                                  |    45  |         begin
                                  |    46  |           check_rules(line)
   70   (64.2%)                   |    47  |           ["\x40todo", 'TODO:?'].each do |pfx|
   21   (19.3%) /     3   (2.8%)  |    48  |             %r{(.*(?:^|\s))#{pfx}\s+#([\w\-\.:/]+)\s+(.+)}.match(line) do |m|
   16   (14.7%)                   |    49  |               puzzles << puzzle(lines.drop(idx + 1), m, idx)
    2    (1.8%) /     2   (1.8%)  |    50  |             end
   49   (45.0%) /    49  (45.0%)  |    51  |           end
                                  |    52  |         rescue Error, ArgumentError => ex

✅ Research topic and preparing the environment for extensions ✅ Write unit functions for extensions ✅ Write basic implementation for useful structures (Dynamic arrays and etc) for C-lang ✅ Transfer Ruby regexp to POSIX standart C format regexp (Need for unit functions like check puzzle rules) 🔧 Write PDD::Source#puzzles like C-lang extension 🔧 Testing extension [ ] Conduct checking Profiling and Benchmarking

yegor256 commented 2 years ago

@Doldrums ready to merge?

yegor256 commented 2 years ago

@Doldrums what's up with this?

Doldrums commented 2 years ago

@yegor256 as it was discussed on weekly meeting - no need for this pr in performance-speed-up field for pdd project (we had reached our perfomance-speed goals with ruby refactoring)