search

nim-lang / Nim

Nim is a statically typed compiled systems programming language. It combines successful concepts from mature languages like Python, Ada and Modula. Its design focuses on efficiency, expressiveness, and elegance (in that order of priority).
https://nim-lang.org
Other
16.59k stars 1.47k forks source link

Tracking of some ARC/ORC bugs #14224

Closed ghost closed 3 years ago

ghost commented 4 years ago

This issue is for some libraries which don't work with ARC and there's no short example. Also see https://github.com/Yardanico/random-stuff/issues/1

Update: managed to reproduce it - https://github.com/nim-lang/Nim/issues/14236

Also see https://github.com/nim-lang/Nim/wiki/Status-of-gc:arc-and-gc:orc-(library-compatibility)

ghost commented 4 years ago

type RotatedString* = object underlying: string shift: int

proc rotate*(s: string, i: int): RotatedString = result = RotatedString(underlying: s, shift: i)

proc rotate*(s: var string, i: int): RotatedString = result = RotatedString(shift: i) shallowCopy(result.underlying, s)

proc []*(r: RotatedString, i: int): char {.inline.} = let L = r.underlying.len assert 0 <= i and i < L let s = i + r.shift if s < L: return r.underlying[s] else: return r.underlying[s - L]

proc []=*(r: var RotatedString, i: int, c: char) {.inline.} = let L = r.underlying.len assert 0 <= i and i < L let s = i + r.shift if s < L: r.underlying[s] = c else: r.underlying[s - L] = c

proc $*(r: RotatedString): string = r.underlying[r.shift .. r.underlying.high] & r.underlying[0 ..< r.shift]

var x = "Hello, world" y = x.rotate(5)

y[0] = 'f' assert x[5] == 'f'

kaushalmodi commented 4 years ago

Steps to reproduce the issue

  1. nimble install glob
  2. Create this t.nim: 
    import glob
for path in walkGlob("src"):
  echo path
  3. Run nim c --gc:arc t.nim

Error

/home/kmodi/.nimble/pkgs/glob-0.9.0/glob.nim(507, 32) Hint: passing 'matchPattern' to a sink parameter introduces an implicit copy; use 'move(matchPattern)' to prevent it [Performance]
fatal.nim(49)            sysFatal
Error: unhandled exception: ccgexprs.nim(777, 11) `ty.kind in {tyTuple, tyObject}`  [AssertionDefect]
andreaferretti commented 4 years ago

Neo has a suite checking that some simple operations do not copy data, but rather share storage with the original matrix - for instance, taking a transpose. This fails with gc: arc. Moreover, changing - say - the definition of the transpose from

proc t*[A](m: Matrix[A]): Matrix[A] =
  if m.isFull:
    matrix(
      order = (if m.order == rowMajor: colMajor else: rowMajor),
      M = m.N,
      N = m.M,
      data = m.data
    )
  else:
    m.clone().t

to

proc t*[A](m: Matrix[A]): Matrix[A] =
  if m.isFull:
    result = matrix(
      order = (if m.order == rowMajor: colMajor else: rowMajor),
      M = m.N,
      N = m.M,
      data = @[]
    )
    shallowCopy(result.data, m.data)
  else:
    m.clone().t

i.e., using shallowCopy explicitly, results in

Error: unhandled exception: liftdestructors.nim(465, 14) `t.destructor != nil`  [AssertionDefect]
andreaferretti commented 4 years ago

Emmy has a similar failure about sharing storage, but also a failure in some tests that apparently do not have anything to do with sharing storage.

andreaferretti commented 4 years ago

Rosencrantz just fails to compile with

/Users/andrea/esperimenti/rosencrantz/rosencrantz/headersupport.nim(31, 26) Error: undeclared identifier: 'await'
cooldome commented 4 years ago

@Yardanico. Could you please continue testing Nim packages latest Nim dev, I think we have fixed a number of issues in the last week.

ghost commented 4 years ago

@cooldome yeah, I'll do that :)

ghost commented 4 years ago

An update on latest optimizer + cursorifier - right now I found that nim-markdown and RBTreeNim trigger some kind of Nim cursorifier bug which makes them crash (in tests).

nimly - some other ARC bug, not caused by the cursorifier.

Also NiGui example_04_msgboxes.nim crashes with ARC when clicking on a button and then clicking "ok" in the messagebox.

Any help in minimizing these so they're easy to reproduce would be very welcome :)

Quite a lot of stuff works though - including bigger projects like https://github.com/fox0430/moe

ghost commented 4 years ago

Seems like nimly works with ORC on latest devel now, same for markdown, rbtreenim, moe.

NiGui issue is still there.

ghost commented 4 years ago

Did some testing of nimforum with ORC both single-threaded/multi-threaded.

It really seems to me that httpbeast and ORC don't like each other at all

ghost commented 4 years ago

Also with multi-threaded mode ARC and ORC behave roughly the same - leak the same amount of memory (800-900MB RAM), and are slower than single-threaded.

ghost commented 4 years ago

Tested httpbeast directly - both ARC/ORC don't leak any memory with the hello world example of it, but ARC/ORC is 3x slower in multi-threaded helloworld example

Araq commented 4 years ago

Test for a better example, helloworld is meaningless.

ghost commented 4 years ago

Sorry, I didn't make a better httpbeast example yet, but found an ORC crash in https://github.com/kostya/benchmarks/blob/master/havlak/havlak.nim again (it worked before, but stopped working relatively recently); I'll try to minimize and git bisect to find the commit when that behaviour started showing up.

ghost commented 4 years ago

The first bad commit seems to be d19316bbb986a7dd1d6d091173963f6e74c65991, not minimised test case is https://gist.github.com/Yardanico/1ad44c09a70e2d781b84f147cd049134

ghost commented 4 years ago

It seems to run fine if I replace while j.traceStack.len > until: with old while j.traceStack.len > 0: like it was before in https://github.com/nim-lang/Nim/blob/devel/lib/system/orc.nim#L138

ghost commented 3 years ago

I feel like this issue is no longer needed, if there are any specific ARC/ORC bugs it's better to have separate issues for them.