mratsim
commented
6 years ago I can reproduce on my Mac.
Open timotheecour opened 6 years ago
mratsim
commented
6 years ago I can reproduce on my Mac.
ghost
commented
6 years ago @timotheecour btw, it's better to get stacktrace via ./koch temp <compiler options>, like ./koch temp c myfile.nim
timotheecour
commented
6 years ago thanks; indeed, it gives a better stacktrace and also tells "stack overflow" at then end; here:
/Users/timothee/git_clone/nim/Arraymancer/src/io/io_npy.nim(104, 55) template/generic instantiation from here
/Users/timothee/git_clone/nim/Arraymancer/src/io/io_npy.nim(36, 15) Warning: Cannot prove that 'result' is initialized. This will become a compile time error in the future. [ProveInit]
of ">u8": result.parser = proc(x: FileStream): T = x.readUInt64BE.T
^
Traceback (most recent call last)
/Users/timothee/git_clone/nim/Nim/compiler/nim.nim(133) nim
/Users/timothee/git_clone/nim/Nim/compiler/nim.nim(97) handleCmdLine
/Users/timothee/git_clone/nim/Nim/compiler/main.nim(162) mainCommand
/Users/timothee/git_clone/nim/Nim/compiler/main.nim(73) commandCompileToC
/Users/timothee/git_clone/nim/Nim/compiler/modules.nim(124) compileProject
/Users/timothee/git_clone/nim/Nim/compiler/modules.nim(71) compileModule
/Users/timothee/git_clone/nim/Nim/compiler/passes.nim(194) processModule
/Users/timothee/git_clone/nim/Nim/compiler/passes.nim(103) processTopLevelStmt
/Users/timothee/git_clone/nim/Nim/compiler/sem.nim(592) myProcess
/Users/timothee/git_clone/nim/Nim/compiler/sem.nim(560) semStmtAndGenerateGenerics
/Users/timothee/git_clone/nim/Nim/compiler/semstmts.nim(1891) semStmt
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(912) semExprNoType
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2553) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/importer.nim(192) evalImport
/Users/timothee/git_clone/nim/Nim/compiler/importer.nim(168) impMod
/Users/timothee/git_clone/nim/Nim/compiler/importer.nim(150) myImportModule
/Users/timothee/git_clone/nim/Nim/compiler/modules.nim(84) importModule
/Users/timothee/git_clone/nim/Nim/compiler/modules.nim(71) compileModule
/Users/timothee/git_clone/nim/Nim/compiler/passes.nim(194) processModule
/Users/timothee/git_clone/nim/Nim/compiler/passes.nim(103) processTopLevelStmt
/Users/timothee/git_clone/nim/Nim/compiler/sem.nim(592) myProcess
/Users/timothee/git_clone/nim/Nim/compiler/sem.nim(560) semStmtAndGenerateGenerics
/Users/timothee/git_clone/nim/Nim/compiler/semstmts.nim(1891) semStmt
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(912) semExprNoType
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2413) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(894) semDirectOp
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(790) afterCallActions
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(31) semTemplateExpr
/Users/timothee/git_clone/nim/Nim/compiler/sem.nim(411) semAfterMacroCall
/Users/timothee/git_clone/nim/Nim/compiler/semstmts.nim(1891) semStmt
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(912) semExprNoType
(1874 calls omitted) ...
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(800) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2382) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(2419) semExpr
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(242) semConv
/Users/timothee/git_clone/nim/Nim/compiler/semexprs.nim(136) checkConvertible
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(337) cmpTypes
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(1751) typeRelImpl
/Users/timothee/git_clone/nim/Nim/compiler/sigmatch.nim(798) tryResolvingStaticExpr
/Users/timothee/git_clone/nim/Nim/compiler/semtypinst.nim(566) replaceTypesInBody
/Users/timothee/git_clone/nim/Nim/compiler/semtypinst.nim initLayeredTypeMap
/Users/timothee/git_clone/nim/Nim/lib/system/assign.nim(230) genericReset
/Users/timothee/git_clone/nim/Nim/lib/system/assign.nim(217) genericResetAux
/Users/timothee/git_clone/nim/Nim/lib/system/assign.nim(215) genericResetAux
/Users/timothee/git_clone/nim/Nim/lib/system/assign.nim(230) genericReset
/Users/timothee/git_clone/nim/Nim/lib/system/assign.nim(217) genericResetAux
/Users/timothee/git_clone/nim/Nim/lib/system/assign.nim(215) genericResetAux
/Users/timothee/git_clone/nim/Nim/lib/system/assign.nim(228) genericReset
/Users/timothee/git_clone/nim/Nim/lib/system/gc.nim(284) unsureAsgnRef
/Users/timothee/git_clone/nim/Nim/lib/system/gc_common.nim(295) isOnStack
/Users/timothee/git_clone/nim/Nim/lib/system/gc_common.nim getActiveStack
Stack overflow
errorHere is a smaller test case following my investigation in https://github.com/mratsim/Arraymancer/issues/260
proc relative_error*[T](y_true, y: T): T {.inline.} =
## Relative error, |y_true - y|/max(|y_true|, |y|)
## Normally the relative error is defined as |y_true - y| / |y_true|,
## but here max is used to make it symmetric and to prevent dividing by zero,
## guaranteed to return zero in the case when both values are zero.
let denom = max(abs(y_true), abs(y))
if denom == 0.T:
return 0.T
result = abs(y_true - y) / denom
proc numerical_gradient*[T](input: T, f: (proc(x: T): T), h: T = 1e-5.T): T {.inline.} =
## Compute numerical gradient for any function w.r.t. to an input value,
## useful for gradient checking, recommend using float64 types to assure
## numerical precision. The gradient is calculated as:
## (f(x + h) - f(x - h)) / (2*h)
## where h is a small number, typically 1e-5.
result = (f(input + h) - f(input - h)) / (2.0.T * h)
proc f(x: float): float = x*x + x + 1.0
echo relative_error(numerical_gradient(2.0, f), 5.0) < 1e-8The compiler just quits without any message.
Note that this is a completely different code path than result.parser = proc(x: FileStream): T = x.readUInt64BE.T reported that Timothée.
The common points are closure procs.
mratsim
commented
6 years ago Further investigation shows that closure + generics do not work anymore as the following works fine:
proc relative_error*(y_true, y: float): float {.inline.} =
## Relative error, |y_true - y|/max(|y_true|, |y|)
## Normally the relative error is defined as |y_true - y| / |y_true|,
## but here max is used to make it symmetric and to prevent dividing by zero,
## guaranteed to return zero in the case when both values are zero.
let denom = max(abs(y_true), abs(y))
if denom == 0.0:
return 0.0
result = abs(y_true - y) / denom
proc numerical_gradient*(input: float, f: (proc(x: float): float), h = 1e-5): float {.inline.} =
## Compute numerical gradient for any function w.r.t. to an input value,
## useful for gradient checking, recommend using float64 types to assure
## numerical precision. The gradient is calculated as:
## (f(x + h) - f(x - h)) / (2*h)
## where h is a small number, typically 1e-5.
result = (f(input + h) - f(input - h)) / (2.0 * h)
proc f(x: float): float = x*x + x + 1.0
echo relative_error(numerical_gradient(2.0, f), 5.0) < 1e-8
skilchen
commented
6 years ago The problem of @mratsim was introduced via commit 7cec5d1cfb0ada682b8cc0b605adc6c1c8b32758
Merge pull request #7681 from nim-lang/typedesc-reforms
Typedesc reformI think this is something @zah should take a look at
timotheecour
commented
6 years ago @skilchen did you use git bisect to find that commit?
skilchen
commented
6 years ago @timotheecour i tried, but for some reason it is impossible to find https://github.com/nim-lang/Nim/commit/7cec5d1cfb0ada682b8cc0b605adc6c1c8b32758 using git bisect as the bisecting will eventually find one of the single commits from @zah collectively merged by @araq (but keeping them in the history) and then you end up in an uncompilable state where the only way out (as far as i know) is git bisect skip until you have skipped all of @zah's single commits.
LemonBoy
commented
6 years ago Here's a minimal reproducer:
proc f[T](h: T = 1.T) = discard
f()The problem is the 1.T argument that throws the typeRel in a loop as T is not known yet.
timotheecour
commented
6 years ago @LemonBoy I can't reproduce error on your minimal case:
rnim bugs/compiler/t07_bug_8432.nim
nim c --nimcache:/tmp/nim//nimcache/ -o:/tmp/nim//app -r bugs/compiler/t07_bug_8432.nim
# no errornim --version Nim Compiler Version 0.18.1 [MacOSX: amd64] Compiled at 2018-08-06 Copyright (c) 2006-2018 by Andreas Rumpf
git hash: bbc404d80d77365a1c402a0862a3d26958b16e7b active boot switches: -d:release
(ie, latest)
mratsim
commented
6 years ago Following discussion with @Araq on IRC https://irclogs.nim-lang.org/07-09-2018.html#08:33:40
I fixed the issue and confirmed it has to do with default params + generics. There seems to be 2 ways this can happen according to this commit https://github.com/mratsim/Arraymancer/pull/275/commits/f278ea5f261e3b783a0249d42d103af84964d001 in Arraymancer:
using this signature proc numerical_gradient*[T](input: T, f: (proc(x: T): T), h: T = 1e-5.T): T instead of this one proc numerical_gradient*[T](input: T, f: (proc(x: T): T), h = T(1e-5)): T. The :T throws the compiler in a loop. Minimal example already provided in this thread.
Using let layer_3neurons = ctx.variable(randomTensor(3, 2, 2.0f) .- 1.0f, requires_grad = true) instead of let layer_3neurons = ctx.variable(randomTensor(3, 2, 2.0f) .- 1.0f, true) with the function signature being proc variable*[TT](ctx: Context[TT], value: TT, requires_grad = false): Variable[TT]. Unfortunately, I can't reproduce in a minimal working example for some reason.
Araq
commented
6 years ago Easy workaround exists, no showstopper.
to reproduce: checkout code from here: https://github.com/mratsim/Arraymancer/pull/259 (which itself fixes https://github.com/mratsim/Arraymancer/issues/258 to work with nim devel) (eg: git clone https://github.com/timotheecour/Arraymancer/ && git checkout fix_258) then: choosenim devel nimble test
it fails with: Error: Execution failed with exit code 35584 ... Command: "/Users/timothee/.nimble/bin/nim" c --noNimblePath --path:"/Users/timothee/.nimble/pkgs/nimblas-0.2.1" --path:"/Users/timothee/.nimble/pkgs/nimlapack-0.1.1" --path:"/Users/timothee/.nimble/pkgs/nimcuda-0.1.5" --path:"/Users/timothee/.nimble/pkgs/nimcl-0.1.3" --path:"/Users/timothee/.nimble/pkgs/opencl-1.0" --path:"/Users/timothee/.nimble/pkgs/clblast-0.0.1" --path:"/Users/timothee/.nimble/pkgs/opencl-1.0" --path:"/Users/timothee/.nimble/pkgs/stb_image-2.1" "--out:./build/tests_cpu" "--run" "--nimcache:"nimcache"" "tests/tests_cpu.nim"
and, after further simplification, this also fails:
prints:
errorat the end (just because I added&& echo ok || echo error) (and build/tests_cpu4 isn't generated)note: it works with
choosenim stable, so looks like a regressionNOTE: after using nim_temp from ./koch temp --out:$nimc_D/bin/nim_temp, it fails with: segmentation fault (and no error)
NOTE: after further investigation, with
lldb -- $nimc_D/bin/nim_temp ...I get following stacktrace under lldb (didn't work under gdb... not sure whether gdb is supported?):this probably indicates stack overflow from infinite recursion:
(bt)
NOTE: I'm on OSX
NOTE: the last msg (with --verbosity:3) was: /Users/timothee/git_clone/nim/Nim/lib/pure/unittest.nim(487, 9) Hint: global variable declared here [GlobalVar] var testStatusIMPL {.inject.} = OK
NOTE: this would be a perfect application of D's dustmite tool, if we adapted it to handle Nim (see https://github.com/nim-lang/Nim/issues/8276 )