Break point based on source file line number breaks on wrong line

[zhopan77]

I have a script about 1,600 lines long which uses a bunch of installed packages as well as self-created modules.

After I set a break point on a line (using either Juno's GUI or the "bp add line" command from the REPL after using the Debugger package) then continue, the actual break happens about 100 lines after the line where I placed the break point.

I tried enabling or disabling the Complied Mode (either through Juno GUI or the "C" command in REPL), and I got the same problem.

However, if I add the break point by adding a line "@bp" inside my script, the break happens at the right location, regardless if I use Juno.@enter or @enter.

The script is long, but I don't know how to get a minimal example that can re-produce the same problem.

Here's my Julia client's Debug Info:

# Atom:
Version: 1.40.1
Dev Mode: false
Official Release: true
{
  "http_parser": "2.8.0",
  "node": "10.2.0",
  "v8": "6.6.346.32",
  "uv": "1.20.3",
  "zlib": "1.2.11",
  "ares": "1.14.0",
  "modules": "64",
  "nghttp2": "1.29.0",
  "napi": "3",
  "openssl": "1.1.0h",
  "electron": "3.1.10",
  "chrome": "66.0.3359.181"
}
# julia-client:
Version: 0.9.4
Config:
{
  "firstBoot": false,
  "juliaOptions": {
    "optimisationLevel": 0
  },
  "uiOptions": {
    "enableMenu": true,
    "enableToolBar": true,
    "errorNotifications": false,
    "layouts": {
      "console": {
        "defaultLocation": "right"
      },
      "defaultPanes": {
        "plotPane": false
      },
      "documentation": {
        "defaultLocation": "right",
        "split": "no split"
      },
      "linter": {
        "defaultLocation": "right"
      },
      "plotPane": {
        "defaultLocation": "right",
        "split": "no split"
      },
      "profiler": {
        "defaultLocation": "right",
        "split": "no split"
      },
      "terminal": {
        "defaultLocation": "right"
      },
      "workspace": {
        "defaultLocation": "right",
        "split": "no split"
      }
    },
    "usePlotPane": false
  }
}

# ink:
Version: 0.10.12
Config:
undefined

# uber-juno:
Version: 0.2.0
Config:
{
  "disable": true
}

# language-julia:
Version: 0.19.1
Config:
undefined

# language-weave:not installed

# indent-detective:
Version: 0.4.0
Config:
undefined

Julia Version 1.1.1
Commit 55e36cc308 (2019-05-16 04:10 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = "C:\Users\zpan\AppData\Local\atom\app-1.40.1\atom.exe"  -a
  JULIA_NUM_THREADS = 2

    Status `C:\Users\zpan\.julia\environments\v1.1\Project.toml`
  [e6d88f4b] ASTInterpreter2 v0.1.1+ #73711a4 (https://github.com/JuliaDebug/ASTInterpreter2.jl.git)
  [537997a7] AbstractPlotting v0.9.8
  [b19378d9] ArrayFire v1.0.1
  [c52e3926] Atom v0.9.0
  [6e4b80f9] BenchmarkTools v0.4.2
  [ad839575] Blink v0.12.0
  [336ed68f] CSV v0.5.9
  [c5f51814] CUDAdrv v3.1.0
  [be33ccc6] CUDAnative v2.2.1
  [49dc2e85] Calculus v0.5.0
  [aaaa29a8] Clustering v0.13.2
  [da1fd8a2] CodeTracking v0.5.7
  [a8cc5b0e] Crayons v4.0.0
  [3a865a2d] CuArrays v1.1.0
  [a93c6f00] DataFrames v0.19.1
  [864edb3b] DataStructures v0.17.0
  [31a5f54b] Debugger v0.5.0
  [67417a49] DebuggerFramework v0.1.2+ #master (https://github.com/JuliaDebug/DebuggerFramework.jl.git)
  [7806a523] DecisionTree v0.8.3
  [0c46a032] DifferentialEquations v6.6.0
  [31c24e10] Distributions v0.21.1
  [ffa9a821] DocumentFormat v0.4.1
  [7a1cc6ca] FFTW v0.3.0
  [59287772] Formatting v0.3.5
  [38e38edf] GLM v1.1.1
  [e9467ef8] GLMakie v0.0.6
  [28b8d3ca] GR v0.41.0
  [c91e804a] Gadfly v1.0.1
  [4c0ca9eb] Gtk v0.17.0
  [f67ccb44] HDF5 v0.12.0
  [09f84164] HypothesisTests v0.8.0
  [6a3955dd] ImageFiltering v0.6.4
  [6deec6e2] IndexedTables v0.12.2
  [d0351b0e] InspectDR v0.3.6
  [c601a237] Interact v0.10.2
  [a98d9a8b] Interpolations v0.12.2
  [6042db11] JDBC v0.4.1
  [4138dd39] JLD v0.9.1
  [682c06a0] JSON v0.21.0
  [494afd89] JavaCall v0.7.1
  [a93385a2] JuliaDB v0.12.0
  [aa1ae85d] JuliaInterpreter v0.6.1
  [e5e0dc1b] Juno v0.7.1
  [1902f260] Knet v1.2.3
  [b964fa9f] LaTeXStrings v1.0.3
  [fc18253b] LazyJSON v0.1.1
  [093fc24a] LightGraphs v1.2.0
  [9c8b4983] LightXML v0.8.0
  [6f1432cf] LoweredCodeUtils v0.3.6
  [10e44e05] MATLAB v0.7.3
  [ee78f7c6] Makie v0.9.4
  [e7bfaba1] NumericalIntegration v0.2.0
  [09606e27] ODEInterfaceDiffEq v3.3.1
  [47be7bcc] ORCA v0.2.1
  [429524aa] Optim v0.19.2
  [3b7a836e] PGFPlots v3.1.3
  [9b87118b] PackageCompiler v0.6.4
  [d96e819e] Parameters v0.10.3
  [58dd65bb] Plotly v0.2.0
  [f0f68f2c] PlotlyJS v0.12.5
  [91a5bcdd] Plots v0.26.0
  [f27b6e38] Polynomials v0.5.2
  [438e738f] PyCall v1.91.2
  [d330b81b] PyPlot v2.8.1
  [6f49c342] RCall v0.13.3
  [ee283ea6] Rebugger v0.3.2
  [295af30f] Revise v2.1.6
  [f2b01f46] Roots v0.8.1
  [276daf66] SpecialFunctions v0.7.2
  [2913bbd2] StatsBase v0.32.0
  [b8865327] UnicodePlots v1.1.0
  [fdbf4ff8] XLSX v0.5.4
  [a5390f91] ZipFile v0.8.3

[KristofferC]

The script is long, but I don't know how to get a minimal example that can re-produce the same problem.

Keep removing code until the problem stops happening?

Also can you reproduce this completely outside of Juno?

[zhopan77]

Thanks @KristofferC for quick response.

I confirmed that the same problem also happens when I have no Juno involvement - just running from a simple Windows console I have exactly the same symptom.

I will try to reproduce the problem using a smaller script.

[zhopan77]

Just to update my progress. I narrowed down the problem between line 1185 and 1188. If I set break point at line 1185, I can break properly; line 1186 is empty line and line 1187 is a comment; but if I set break point at line 1188, somehow the debugger will continue until line 1358 which is a println() statement (but it's NOT the only println() statement between line 1187 and 1358).

In addition, adding break point for any line between 1188 and 1358 will result in breaking at 1358. But adding break point AFTER line 1358 results in breaking at the correct line.

I don't see anything special for the code between lines 1188 and 1358 though. :-(

[zhopan77]

Tried creating simpler script file with similar imports, functions, loops, comments, multi-line statements... Nothing can reproduce the problem.

Tried a simpler script that goes to 1,400+ lines long with many empty lines in between. Still cannot reproduce the problem when I set break point between line 1188 and 1358.

One hint though. In the original script, the first 1,188 lines contains 62,590 characters, while the first 1,358 lines contains 75,021 characters. Since these two numbers crosses 65,536 which is 2^16, not sure if this made any difference in terms of buffer allocation etc. :-)

[zhopan77]

It seems it's still related to the statement rather than the location.

I removed ~40 lines of code up front, but the break point problem still happens at around the same statements. Difference is I can break at line 1,148 (which was line 1,188) correctly now; I can even go to line 1,149, but the problem starts at line 1,151 now (58,882 characters), and any break point in between 1,151 and 1,318 now jumps to 1,318 (which is the same statement as line 1,358), including 71,144 characters.

However the statement where the problem starts is nothing special. Similar function calls happened several times earlier in the script.

Keep cutting down the code is tough as later code relies on things defined/calculated earlier...

[zhopan77]

I verified the cause is not number of characters.

I created a large script in the following format:

function main()
    n = 2; NPRINT = 1;
    n += 1; if n % NPRINT == 0 println("This is line $n"); end
    n += 1; if n % NPRINT == 0 println("This is line $n"); end
    n += 1; if n % NPRINT == 0 println("This is line $n"); end
    ...
end

main();

The "..." above means the same line is repeated for a total of about 1100 times. This will ensure the number of characters is well above 65,536.

I then tried setting break point from line 800 to line 1100 randomly and the debugger always breaks at the correct line.

So why the original source code has such a problem is still a mystery. :-(

[zhopan77]

OK I guess the problem may be related to this closed issue #219 for JuliaInterpreter.

I didn't notice that issue #219 had been closed, so in my original main script above I had a global variable

isPlot = false;
if isPlot
    using PyPlot;
end

and I wrapped all PyPlot uses within blocks like

if isPlot 
    # do PyPlot operations ...
end

Only when I want to show the plot I'll set isPlot = true; otherwise I leave it to be false, including when I use the Debugger. Note that I also used a self-made module which contains "using PyCall" and "using PyPlot", but in the main script I never called any function in that self-made module that will actually use anything from PyCall or PyPlot.

Now the strange thing is, once I REMOVED all lines related to PyPlot from the main script, I cannot reproduce the problem above anymore. (The "using PyCall" and "using PyPlot" statements are still there in the self-made module used.)

I did try a simple script with only a few statements, and I deliberately sprinkled PyPlot related blocks in several places. However, I cannot reproduce the problem above. So I only know the problem seems to be related to PyPlot, but I still cannot find a simple and deterministic way to re-create it.

Using @bp is an acceptable workaround in the rare case this problem shows up, so I'll leave this issue open here but won't spend more time on it.