"Computation has been terminated" when terminating a process while resuming from an exception

LinqLover commented 5 months ago

Hi @isCzech, all,

I've discovered another situation while I get a "Computation has terminated" error from Process>>#terminate. The entire situation is pretty complex, depends on some experimental code that I have not yet commited to third-party packages, and occurs only very sporadically (that terminate is maybe getting sent thousands of times per day and fails every second day...) - so it won't be possible to clearly reproduce right now. Nevertheless, I'm trying to collect as many information about the bug here as possible and hope it helps:

First debugger:

Bug report

15 January 2024 5:43:58.314083 pm

VM: Win32 - Smalltalk
Image: Squeak6.1alpha [latest update: #22943]

SecurityManager state:
Restricted: false
FileAccess: true
SocketAccess: true
Working Dir C:\Users\Christoph\OneDrive\Dokumente\Squeak
Trusted Dir C:\Users\Christoph\OneDrive\Dokumente\Squeak\Christoph
Untrusted Dir C:\Users\Christoph\OneDrive\Dokumente\My Squeak

Context(Object)>>error:
    Receiver: FullBlockClosure(BlockClosure)>>ensure:
    Arguments and temporary variables: 
        aString:    'Computation has been terminated!'
    Receiver's instance variables: 
        sender:     [] in Context>>handleSignal:
        pc:     nil
        stackp:     4
        method:     (BlockClosure>>#ensure: "a CompiledMethod(1093181)")
        closureOrNil:   nil
        receiver:   [closure] in Context>>handleSignal:

Context>>cannotReturn:
    Receiver: FullBlockClosure(BlockClosure)>>ensure:
    Arguments and temporary variables: 
        result:     FullBlockClosure(BlockClosure)>>on:do:
    Receiver's instance variables: 
        sender:     [] in Context>>handleSignal:
        pc:     nil
        stackp:     4
        method:     (BlockClosure>>#ensure: "a CompiledMethod(1093181)")
        closureOrNil:   nil
        receiver:   [closure] in Context>>handleSignal:

FullBlockClosure(BlockClosure)>>ensure:
    Receiver: [closure] in Context>>handleSignal:
    Arguments and temporary variables: 
        aBlock:     [closure] in Context>>handleSignal:
        complete:   true
        returnValue:    nil
    Receiver's instance variables: 
        outerContext:   Context>>handleSignal:
        startpcOrMethod:    ([] in Context>>#handleSignal: "a CompiledBlock(2105497)")
        numArgs:    0
        receiver:   FullBlockClosure(BlockClosure)>>on:do:

[] in Context>>handleSignal:
    Receiver: FullBlockClosure(BlockClosure)>>on:do:
    Arguments and temporary variables: 
        exception:  UndeclaredVariableNotification: 
        val:    nil
    Receiver's instance variables: 
        sender:     [] in FullBlockClosure(BlockClosure)>>on:do:on:do:
        pc:     34
        stackp:     4
        method:     (BlockClosure>>#on:do: "a CompiledMethod(1946039)")
        closureOrNil:   nil
        receiver:   [closure] in ECGlobalEntry(ECEntry)>>tryPredictResultWith:andDo:

FullBlockClosure(BlockClosure)>>ensure:
    Receiver: [closure] in Context class>>contextEnsure:
    Arguments and temporary variables: 
        aBlock:     [closure] in FullBlockClosure(BlockClosure)>>valueAndWaitWhileUnwinding:...etc...
        complete:   true
        returnValue:    nil
    Receiver's instance variables: 
        outerContext:   Context class>>contextEnsure:
        startpcOrMethod:    ([] in Context class>>#contextEnsure: "a CompiledBlock(2240957...etc...
        numArgs:    0
        receiver:   Context

--- The full stack ---
Context(Object)>>error:
Context>>cannotReturn:
FullBlockClosure(BlockClosure)>>ensure:
[] in Context>>handleSignal:
FullBlockClosure(BlockClosure)>>ensure:

At the same time, the UI hangs, and I need to press Cmd-dot to continue. The interrupt reveals where the UI process was stuck:

Bug report

15 January 2024 5:45:40.243083 pm

VM: Win32 - Smalltalk
Image: Squeak6.1alpha [latest update: #22943]

SecurityManager state:
Restricted: false
FileAccess: true
SocketAccess: true
Working Dir C:\Users\Christoph\OneDrive\Dokumente\Squeak
Trusted Dir C:\Users\Christoph\OneDrive\Dokumente\Squeak\Christoph
Untrusted Dir C:\Users\Christoph\OneDrive\Dokumente\My Squeak

FullBlockClosure(BlockClosure)>>valueAndWaitWhileUnwinding:
    Receiver: [closure] in [] in Process>>terminate
    Arguments and temporary variables: 
        contextToUnwind:    Context>>pop
        semaphore:  a Semaphore()
        newBottom:  FullBlockClosure(BlockClosure)>>ensure:
    Receiver's instance variables: 
        outerContext:   [] in Process>>terminate
        startpcOrMethod:    ([] in Process>>#terminate "a CompiledBlock(3245925)")
        numArgs:    1
        receiver:   a Process(50758) in [] in Context>>unwindAndStop:

[] in Process>>terminate
    Receiver: a Process(50758) in [] in Context>>unwindAndStop:
    Arguments and temporary variables: 

    Receiver's instance variables: 
        nextLink:   nil
        suspendedContext:   [] in Context>>unwindAndStop:
        priority:   40
        myList:     nil
        threadAffinity:     nil
        effectiveProcess:   nil
        name:   nil
        island:     nil
        env:    a Dictionary()

FullBlockClosure(BlockClosure)>>ensure:
    Receiver: [closure] in Process>>terminate
    Arguments and temporary variables: 
        aBlock:     [closure] in Process>>terminate
        complete:   true
        returnValue:    nil
    Receiver's instance variables: 
        outerContext:   Process>>terminate
        startpcOrMethod:    ([] in Process>>#terminate "a CompiledBlock(1088437)")
        numArgs:    0
        receiver:   a Process(50758) in [] in Context>>unwindAndStop:

Process>>terminate
    Receiver: a Process(50758) in [] in Context>>unwindAndStop:
    Arguments and temporary variables: 

    Receiver's instance variables: 
        nextLink:   nil
        suspendedContext:   [] in Context>>unwindAndStop:
        priority:   40
        myList:     nil
        threadAffinity:     nil
        effectiveProcess:   nil
        name:   nil
        island:     nil
        env:    a Dictionary()

[] in ECMenuMorph>>postNarrow
    Receiver: an ECMenuMorph(3220153)
    Arguments and temporary variables: 
        p:  a Process(50758) in [] in Context>>unwindAndStop:
    Receiver's instance variables: 
        bounds:     759@582 corner: 1036@1017
        owner:  nil
        submorphs:  #()
        fullBounds:     759@582 corner: 1036@1017
        color:  (Color r: 0.9 g: 0.9 b: 0.9)
        extension:  a MorphExtension (772249) [other:  (borderStyle -> a SimpleBorder) (...etc...
        selected:   1
        firstVisible:   1
        titleStringMorph:   nil
        controller:     an ECBrowserController
        context:    an ECContext
        pageHeight:     26
        itemHeight:     16
        detailMorph:    nil
        detailPosition:     1035@582
        lastInteraction:    2024-01-15T17:30:29.192083+01:00
        alpha:  0.0
        processes:  an OrderedCollection()

OrderedCollection>>removeAllSuchThat:
    Receiver: an OrderedCollection()
    Arguments and temporary variables: 
        aBlock:     [closure] in ECMenuMorph>>postNarrow
        n:  1
        index:  192
        element:    a Process(50758) in [] in Context>>unwindAndStop:
    Receiver's instance variables: 
        array:  #(nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil ni...etc...
        firstIndex:     1
        lastIndex:  0

ECMenuMorph>>postNarrow
    Receiver: an ECMenuMorph(3220153)
    Arguments and temporary variables: 

    Receiver's instance variables: 
        bounds:     759@582 corner: 1036@1017
        owner:  nil
        submorphs:  #()
        fullBounds:     759@582 corner: 1036@1017
        color:  (Color r: 0.9 g: 0.9 b: 0.9)
        extension:  a MorphExtension (772249) [other:  (borderStyle -> a SimpleBorder) (...etc...
        selected:   1
        firstVisible:   1
        titleStringMorph:   nil
        controller:     an ECBrowserController
        context:    an ECContext
        pageHeight:     26
        itemHeight:     16
        detailMorph:    nil
        detailPosition:     1035@582
        lastInteraction:    2024-01-15T17:30:29.192083+01:00
        alpha:  0.0
        processes:  an OrderedCollection()

ECBrowserController(ECController)>>handleKeystrokeAfter:editor:
    Receiver: an ECBrowserController
    Arguments and temporary variables: 
        aKeyboardEvent:     [170@179 keystroke 'n' (110) 138401125]
        anEditor:   a SmalltalkEditor
    Receiver's instance variables: 
        model:  a WeakArray(a TreeBrowser)
        menuMorph:  nil
        editor:     a WeakArray(a SmalltalkEditor)
        context:    nil
        oppositeChar:   nil
        caret:  nil
        inverseMapping:     nil

ECToolSet class>>codeCompletionAround:textMorph:keyStroke:
    Receiver: ECToolSet
    Arguments and temporary variables: 
        aBlock:     [closure] in TextMorphForEditView(TextMorph)>>keyStroke:
        aTextMorph:     a TextMorphForEditView(491455)
        evt:    [170@179 keystroke 'n' (110) 138401125]
        completionAllowed:  true
        editor:     a SmalltalkEditor
        stringHolder:   a TreeBrowser
        controller:     an ECBrowserController
    Receiver's instance variables: 
        superclass:     StandardToolSet
        methodDict:     a MethodDictionary()
        format:     0
        instanceVariables:  nil
        organization:   ('as yet unclassified')

        subclasses:     nil
        name:   #ECToolSet
        classPool:  nil
        sharedPools:    nil
        environment:    Smalltalk
        category:   #'Autocompletion-SqueakCompatibility'

ToolSet class>>codeCompletionAround:textMorph:keyStroke:
    Receiver: ToolSet
    Arguments and temporary variables: 
        aBlock:     [closure] in TextMorphForEditView(TextMorph)>>keyStroke:
        aTextMorph:     a TextMorphForEditView(491455)
        evt:    [170@179 keystroke 'n' (110) 138401125]
    Receiver's instance variables: 
        superclass:     AppRegistry
        methodDict:     a MethodDictionary()
        format:     0
        instanceVariables:  nil
        organization:   ('as yet unclassified')

        subclasses:     nil
        name:   #ToolSet
        classPool:  nil
        sharedPools:    nil
        environment:    Smalltalk
        category:   #'System-Applications'
        registeredClasses:  an OrderedCollection(StandardToolSet CommandLineToolSet ECToolSet...etc...
        default:    ECToolSet

TextMorphForEditView(TextMorph)>>keyStroke:
    Receiver: a TextMorphForEditView(491455)
    Arguments and temporary variables: 
        evt:    [170@179 keystroke 'n' (110) 138401125]
    Receiver's instance variables: 
        bounds:     0@0 corner: 795@260
        owner:  a TransformMorph(1299609)
        submorphs:  #()
        fullBounds:     0@0 corner: 795@260
        color:  Color black
        extension:  a MorphExtension (4182057) [other:  (unfocusedSelectionColor -> (Col...etc...
        borderWidth:    0
        borderColor:    Color black
        textStyle:  a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal
        text:   a Text for 'contentWithStyledProbabilities

    | confidentColor unconfident...etc...
        wrapFlag:   true
        paragraph:  a NewParagraph
        editor:     a SmalltalkEditor
        container:  nil
        predecessor:    nil
        successor:  nil
        backgroundColor:    nil
        margins:    3@2 corner: 3@2
        readOnly:   false
        autoFit:    true
        plainTextOnly:  false
        numCharactersPerLine:   nil
        editView:   a PluggableTextMorphPlus(3139481)
        acceptOnCR:     false

TextMorphForEditView>>keyStroke:
    Receiver: a TextMorphForEditView(491455)
    Arguments and temporary variables: 
        evt:    [170@179 keystroke 'n' (110) 138401125]
        view:   a PluggableTextMorphPlus(3139481)
    Receiver's instance variables: 
        bounds:     0@0 corner: 795@260
        owner:  a TransformMorph(1299609)
        submorphs:  #()
        fullBounds:     0@0 corner: 795@260
        color:  Color black
        extension:  a MorphExtension (4182057) [other:  (unfocusedSelectionColor -> (Col...etc...
        borderWidth:    0
        borderColor:    Color black
        textStyle:  a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal
        text:   a Text for 'contentWithStyledProbabilities

    | confidentColor unconfident...etc...
        wrapFlag:   true
        paragraph:  a NewParagraph
        editor:     a SmalltalkEditor
        container:  nil
        predecessor:    nil
        successor:  nil
        backgroundColor:    nil
        margins:    3@2 corner: 3@2
        readOnly:   false
        autoFit:    true
        plainTextOnly:  false
        numCharactersPerLine:   nil
        editView:   a PluggableTextMorphPlus(3139481)
        acceptOnCR:     false

TextMorphForEditView(Morph)>>handleKeystroke:
    Receiver: a TextMorphForEditView(491455)
    Arguments and temporary variables: 
        anEvent:    [170@179 keystroke 'n' (110) 138401125]
        handler:    a TextMorphForEditView(491455)
    Receiver's instance variables: 
        bounds:     0@0 corner: 795@260
        owner:  a TransformMorph(1299609)
        submorphs:  #()
        fullBounds:     0@0 corner: 795@260
        color:  Color black
        extension:  a MorphExtension (4182057) [other:  (unfocusedSelectionColor -> (Col...etc...
        borderWidth:    0
        borderColor:    Color black
        textStyle:  a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal
        text:   a Text for 'contentWithStyledProbabilities

    | confidentColor unconfident...etc...
        wrapFlag:   true
        paragraph:  a NewParagraph
        editor:     a SmalltalkEditor
        container:  nil
        predecessor:    nil
        successor:  nil
        backgroundColor:    nil
        margins:    3@2 corner: 3@2
        readOnly:   false
        autoFit:    true
        plainTextOnly:  false
        numCharactersPerLine:   nil
        editView:   a PluggableTextMorphPlus(3139481)
        acceptOnCR:     false

TextMorphForEditView(TextMorph)>>handleKeystroke:
    Receiver: a TextMorphForEditView(491455)
    Arguments and temporary variables: 
        anEvent:    [170@179 keystroke 'n' (110) 138401125]
        pasteUp:    nil
    Receiver's instance variables: 
        bounds:     0@0 corner: 795@260
        owner:  a TransformMorph(1299609)
        submorphs:  #()
        fullBounds:     0@0 corner: 795@260
        color:  Color black
        extension:  a MorphExtension (4182057) [other:  (unfocusedSelectionColor -> (Col...etc...
        borderWidth:    0
        borderColor:    Color black
        textStyle:  a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal
        text:   a Text for 'contentWithStyledProbabilities

    | confidentColor unconfident...etc...
        wrapFlag:   true
        paragraph:  a NewParagraph
        editor:     a SmalltalkEditor
        container:  nil
        predecessor:    nil
        successor:  nil
        backgroundColor:    nil
        margins:    3@2 corner: 3@2
        readOnly:   false
        autoFit:    true
        plainTextOnly:  false
        numCharactersPerLine:   nil
        editView:   a PluggableTextMorphPlus(3139481)
        acceptOnCR:     false

KeyboardEvent>>sentTo:
    Receiver: [170@179 keystroke 'n' (110) 138401125]
    Arguments and temporary variables: 
        anObject:   a TextMorphForEditView(491455)
    Receiver's instance variables: 
        timeStamp:  138401125
        source:     a HandMorph(333670)
        type:   #keystroke
        buttons:    0
        position:   170@179
        handler:    a TextMorphForEditView(491455)
        wasHandled:     true
        wasIgnored:     false
        keyValue:   110
        keyCode:    78

TextMorphForEditView(Morph)>>handleEvent:
    Receiver: a TextMorphForEditView(491455)
    Arguments and temporary variables: 
        anEvent:    [170@179 keystroke 'n' (110) 138401125]
        filteredEvent:  [170@179 keystroke 'n' (110) 138401125]
    Receiver's instance variables: 
        bounds:     0@0 corner: 795@260
        owner:  a TransformMorph(1299609)
        submorphs:  #()
        fullBounds:     0@0 corner: 795@260
        color:  Color black
        extension:  a MorphExtension (4182057) [other:  (unfocusedSelectionColor -> (Col...etc...
        borderWidth:    0
        borderColor:    Color black
        textStyle:  a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal
        text:   a Text for 'contentWithStyledProbabilities

    | confidentColor unconfident...etc...
        wrapFlag:   true
        paragraph:  a NewParagraph
        editor:     a SmalltalkEditor
        container:  nil
        predecessor:    nil
        successor:  nil
        backgroundColor:    nil
        margins:    3@2 corner: 3@2
        readOnly:   false
        autoFit:    true
        plainTextOnly:  false
        numCharactersPerLine:   nil
        editView:   a PluggableTextMorphPlus(3139481)
        acceptOnCR:     false

TextMorphForEditView(Morph)>>handleFocusEvent:
    Receiver: a TextMorphForEditView(491455)
    Arguments and temporary variables: 
        anEvent:    [170@179 keystroke 'n' (110) 138401125]
    Receiver's instance variables: 
        bounds:     0@0 corner: 795@260
        owner:  a TransformMorph(1299609)
        submorphs:  #()
        fullBounds:     0@0 corner: 795@260
        color:  Color black
        extension:  a MorphExtension (4182057) [other:  (unfocusedSelectionColor -> (Col...etc...
        borderWidth:    0
        borderColor:    Color black
        textStyle:  a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal
        text:   a Text for 'contentWithStyledProbabilities

    | confidentColor unconfident...etc...
        wrapFlag:   true
        paragraph:  a NewParagraph
        editor:     a SmalltalkEditor
        container:  nil
        predecessor:    nil
        successor:  nil
        backgroundColor:    nil
        margins:    3@2 corner: 3@2
        readOnly:   false
        autoFit:    true
        plainTextOnly:  false
        numCharactersPerLine:   nil
        editView:   a PluggableTextMorphPlus(3139481)
        acceptOnCR:     false

MorphicEventDispatcher>>doHandlingForFocusEvent:with:
    Receiver: a MorphicEventDispatcher
    Arguments and temporary variables: 
        currentEvent:   [532@567 keystroke 'n' (110) 138401125]
        focusMorph:     a TextMorphForEditView(491455)
        localEvent:     [170@179 keystroke 'n' (110) 138401125]
        filteredEvent:  nil
    Receiver's instance variables: 
        lastType:   nil
        lastDispatch:   nil

MorphicEventDispatcher>>dispatchFocusEvent:with:
    Receiver: a MorphicEventDispatcher
    Arguments and temporary variables: 
        anEventWithGlobalPosition:  [532@567 keystroke 'n' (110) 138401125]
        focusMorph:     a TextMorphForEditView(491455)
        currentEvent:   [532@567 keystroke 'n' (110) 138401125]
    Receiver's instance variables: 
        lastType:   nil
        lastDispatch:   nil

--- The full stack ---
FullBlockClosure(BlockClosure)>>valueAndWaitWhileUnwinding:
[] in Process>>terminate
FullBlockClosure(BlockClosure)>>ensure:
Process>>terminate
[] in ECMenuMorph>>postNarrow
OrderedCollection>>removeAllSuchThat:
ECMenuMorph>>postNarrow
ECBrowserController(ECController)>>handleKeystrokeAfter:editor:
ECToolSet class>>codeCompletionAround:textMorph:keyStroke:
ToolSet class>>codeCompletionAround:textMorph:keyStroke:
TextMorphForEditView(TextMorph)>>keyStroke:
TextMorphForEditView>>keyStroke:
TextMorphForEditView(Morph)>>handleKeystroke:
TextMorphForEditView(TextMorph)>>handleKeystroke:
KeyboardEvent>>sentTo:
TextMorphForEditView(Morph)>>handleEvent:
TextMorphForEditView(Morph)>>handleFocusEvent:
MorphicEventDispatcher>>doHandlingForFocusEvent:with:
MorphicEventDispatcher>>dispatchFocusEvent:with:
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
TextMorphForEditView(Morph)>>processFocusEvent:using:
TextMorphForEditView(Morph)>>processFocusEvent:
[] in [] in [] in HandMorph>>sendFocusEvent:to:clear:
[] in ActiveEventVariable class(DynamicVariable class)>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveEventVariable class(DynamicVariable class)>>value:during:
[] in ActiveEventVariable class>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveEventVariable class>>value:during:
KeyboardEvent(MorphicEvent)>>becomeActiveDuring:
[] in [] in HandMorph>>sendFocusEvent:to:clear:
[] in ActiveHandVariable class(DynamicVariable class)>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveHandVariable class(DynamicVariable class)>>value:during:
[] in ActiveHandVariable class>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveHandVariable class>>value:during:
HandMorph>>becomeActiveDuring:
[] in HandMorph>>sendFocusEvent:to:clear:
[] in ActiveWorldVariable class(DynamicVariable class)>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveWorldVariable class(DynamicVariable class)>>value:during:
[] in ActiveWorldVariable class>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveWorldVariable class>>value:during:
PasteUpMorph>>becomeActiveDuring:
HandMorph>>sendFocusEvent:to:clear:
HandMorph>>sendEvent:focus:clear:
HandMorph>>sendKeyboardEvent:
HandMorph>>handleEvent:
HandMorph>>processEvents
[] in [] in WorldState>>doOneCycleNowFor:
[] in ActiveHandVariable class(DynamicVariable class)>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveHandVariable class(DynamicVariable class)>>value:during:
[] in ActiveHandVariable class>>value:during:
FullBlockClosure(BlockClosure)>>ensure:
ActiveHandVariable class>>value:during:
HandMorph>>becomeActiveDuring:
[] in WorldState>>doOneCycleNowFor:
Array(SequenceableCollection)>>do:
WorldState>>handsDo:
-- and more not shown --

Exploring the receiver of the selected context in the first debugger (FullBlockClosure(BlockClosure)>>on:do:) reveals the following stack (it is cyclic/infinite, I used self stackOfSize: 100):

Exploring the contextToUnwind from the interrupted context of the second debugger reveals the following (note the print-it displays the full stack of the interrupted context's receiver):

These are the methods from my package that are relevant to the bug (just look at the <-- pointer):

ECMenuMorph>>postNarrow

    self selected: 0.
    firstVisible := 1.
    self model notEmpty ifTrue: [ self selected: 1 ].
    (self model entries size < 4 and: [ self model expanded not])
        ifTrue: ["Automatically expand if there are too few entries" self expand].
    processes ifNotNil: [processes removeAllSuchThat: [:p | p terminate. true]]. "<-- here the process should be terminated"
    processes ifNil: [processes := OrderedCollection new].
    self model entries do: [:entry |
        processes add: ([entry tryStorePredictedResultWith: context. self future changed] forkAt: Processor userBackgroundPriority)].
    self show.
    ^ true

tryPredictResultWith: anECContext andDo: resultBlock

    | controller editor code method result |
    controller := anECContext instVarNamed: 'controller'.
    editor := controller editor.
    editor ifNil: [^ nil].
    code := ((editor text first: editor startIndex - anECContext completionToken size - 1) , self completion) lines last.
    method := [controller model doItContext class compilerClass new
        compiledMethodFor: code
        in: controller model doItContext
        to: controller model doItReceiver
        notifying: nil
        ifFail: [^ nil]]
            on: UndeclaredVariableNotification do: [:ex | ex resume] "prevent Transcript output" "<-- here an exception is resumed during parsing"
            on: SyntaxErrorNotification do: [:ex | ^ nil].
    result := [Sandbox evaluate:
        [[(method
            valueWithReceiver: controller model doItReceiver
            arguments:
                (controller model doItContext ifNil: [#()] ifNotNil: [:context | {context}]))
                    printString]
                        on: Error , Warning , Halt do: [:ex | ex]]]
                            valueWithin: 5 seconds onTimeout: [].
    ^ result ifNotNil: [resultBlock value: result]

To me, this looks as if the process was attempted to terminate while the UndeclaredVariableNotification was on the stack (maybe while it was already being handled/resumed from), and something in the stack manipulation logic has prevented the termination from working correctly. Maybe this is related to the fact that the context stack is temporarily invalidated during Context class>>#contextEnsure: et al. (cf. stepping into all the details of cut: during thisContext insertSender: (Context contextEnsure: []))?

I wish I could reproduce this issue, but I cannot for now. I have a vague hope that this information might be enough to suggest any ideas of what might be wrong, and maybe create a simpler example to reproduce ...

LinqLover commented 5 months ago

I was able to reproduce!

block := [[Compiler evaluate: 'x'] on: UndeclaredVariableNotification do: [:ex | ex resume]].

"context := block asContext.
steps := 0.
[context willReturn and: [context sender isNil]] whileFalse:
    [steps := steps + 1.
    context := context step].

steps. ""11145""

0 to: 11145"#(6402) do: [:n |
    Transcript showln: n.
    context := block asContext.
    n timesRepeat: [context := context step].
    process := Process forContext: context priority: 40.
    process terminate.
    self assert: process isTerminated].

As the out-commented code shows, terminate should not fail at whatever point the process is interrupted ...

isCzech commented 5 months ago

Hi Christoph, This is an amazing way to test an operation (unwinding/termination in this case) at every step! You should make it a standard testing procedure (method?) ;)

My guess what is happening at step 6402 is the process is in the middle of unwinding the ex resume and you initiate termination, i.e. another unwind on top of the previous one - and the outside unwind kind of slips through some crack in the inner one and the procedure fails with the BCR (see the two ensure contexts inserted at the bottom of the contextToUnwind at your last screenshot). Although I tried to make unwind resilient against termination at any arbitrary point (see tests covering termination of a terminating process etc.) I clearly failed to cover all such situations :)

I'll investigate more thoroughly next week. Thanks for this lovely problem! Jaromir

PS: have you tried it in 5.3 ?

isCzech commented 5 months ago

Hi Christoph, try Kernel-jar.1552 please.

I've described the root cause and quickly drafted a solution. If you find it going in the right direction we can polish the final form.

It turns out the problem lies in the way the ensure guard contexts are being generated by contextEnsure. I'm curious whether the suggested solution will fix all your observed problems or whether some more surface.

isCzech commented 5 months ago

Hi Christoph (@LinqLover),

I've modified #contextEnsure a bit - see Kernel-jar.1553. I guess this prevents the situation you observed. Please check and merge if you like it. Let me know of any other issues, thanks.

isCzech commented 5 months ago

Hi Christoph,

Have you had a chance to check Kernel-jar.1553 yet? I wonder if it solves the bug you observed?

I assume you found another occurrence of the BCR at a later step in your amazing example: this time it's a completely unrelated issue where two nested unwinds don't cooperate correctly. I've improved the "granularity" of the unwind to distinguish unwind blocks that started but not finished their execution and those that are really finished - see Kernel-jar.1554 - description of the bug and the fix.

Please let me know of this helps.

Please either merge if you approve or otherwise I could bundle all my recent changes into a single package... Let me know what you think.

LinqLover commented 5 months ago

Hi @isCzech, so sorry for not replying earlier! Having too many different things to do right now diving into the depths of simulation and unwinding is never something I can handle in a couple of minutes. :( Thank you for looking into this!

This is an amazing way to test an operation (unwinding/termination in this case) at every step! You should make it a standard testing procedure (method?) ;)

Good idea! The original example was pretty slow but I have just added ProcessTest>>#testTerminateEverywhere (KernelTests-ct.451) with a more minified block, which runs in under 1 second. If you think it makes sense, we can adapt the same pattern for testing "termination of a terminating process" etc. as well. :-)

I confirm that Kernel-jar.1553 fixes the original issue and together with Kernel-jar.1554 makes the new test entirely pass, so I am tempted to merge both of them. :-) Nevertheless, I do not yet fully understand why this issue is specific to Context class>>contextEnsure:, why would the following example not trigger the original bug?

Once I have understood that, I will merge both versions. I guess Kernel-jar.1552 can be moved to treated, or do you have any further plans for it? One disadvantage with this solution is that you hardcode expectations about the bytecodes of ensure: there. I have already merged both versions into my multiprocessing-extensive image and will observe it further. :-)

block := [[| c |
    c := thisContext swapSender: nil.
    thisContext swapSender: c.
    42] ensure: []].

isCzech commented 5 months ago

Hi Christoph (@LinqLover),

why this issue is specific to Context class>>contextEnsure:, why would the following example not trigger the original bug?

The crucial difference between the two examples is that the original one (block := [[Compiler evaluate: 'x'] on: UndeclaredVariableNotification do: [:ex | ex resume]]) has a non-local return that calls #contextEnsure: (via #unwindTo: and #runUntilReturnFrom:). When the computation gets stopped at the precise moment the #contextEnsure: disconnects the remainder of its sender chain before jumping back to the top of disconnected remainder and your ingenious testing scenario terminates the process, the termination start at the chopped off head end and fails.

If you're happy with the fix I'd do the same modification to #contextOn:do: for the sake of consistency. I can't see a scenario where this would cause a similar problem but who would have thought about what you observed :)

Kernel-jar.1552 can be moved to treated

Sure, thanks for the cleanup. It was just the first approximation but as you said it's bound to bytecodes which is bad...

diving into the depths of simulation and unwinding is never something I can handle in a couple of minutes

mildly put :)

I have just added ProcessTest>>#testTerminateEverywhere (KernelTests-ct.451) with a more minified block, which runs in under 1 second.

Great! I'd love to see this really as a "pattern" to just feed it a block, a test method (anything, not just terminate) and an expectation and watch the result (ok or where it derails). I haven't really thought it through it just feels so powerful :)

(PS: regarding the other issues I haven't had time yet but I'll get there)

LinqLover commented 5 months ago

Ah, I got it! So a more minimal example would be something like

[| c |
[c := thisContext sender sender swapSender: nil]
    ensure: [thisContext sender sender swapSender: c]] value]

Yes, it sounds wise to patch contextOn:do: as well, even though I wonder why we cannot find an example where this is a problem.

Oh no, here is another example that still fails when put inside #testTerminateEverywhere:

block := [| c |
    c := thisContext.
    [] ensure: [c jump].
    42].

What is going on here?

I have just added ProcessTest>>#testTerminateEverywhere (KernelTests-ct.451) with a more minified block, which runs in under 1 second.

Great! I'd love to see this really as a "pattern" to just feed it a block, a test method (anything, not just terminate) and an expectation and watch the result (ok or where it derails). I haven't really thought it through it just feels so powerful :)

Sure, feel free to extract the logic from #testTerminateEverywhere as it fits your needs. :-)

isCzech commented 5 months ago

Oh no, here is another example that still fails when put inside #testTerminateEverywhere: What is going on here?

I think this is inevitable... jump inside the ensure argument block is a nightmare and if it jumps over my unwind guard in #runUntilReturnFrom: I don't know what can be done about it. I'll explore this along with your other examples around the stepOver bug. Coming soon, I hope :)

Yes, it sounds wise to patch contextOn:do: as well, even though I wonder why we cannot find an example where this is a problem.

I guess the example would have to involve #runUntilErrorOrReturnFrom: which is usually called only in simulation... I haven't had the energy to try so I only guess :) I'll send the fix.

isCzech commented 5 months ago

contextOn:do: fix in Kernel-jar.1555

LinqLover commented 5 months ago

@isCzech Thanks, willl take a look soon! In the meantime, here is just another example that triggers the same error:

block := [(Context runSimulated: [41]) + 1].

This one is actually of practical relevance for me, because in my scenario I am running a lot of sandboxed simulations in background processes ... I wonder whether we could think this together with the stepOver/runUntilErrorOrReturnFrom issue which shares the same vulnerability to irregular context switches ... Like, place a marker on the context stack/a safe bottom context while performing a temporary context switch and when we are terminating a process, check for such a marker, and if yes, continue execution regularly until the marker is no longer set? Or a bit stateless, use a pragma in known context switching methods such as #contextEnsure: (the old version) that instruct the unwinding logic to defer unwinding until that method has popped? Hm ... this is tricky ^^

LinqLover commented 5 months ago

Ah, I see. The ensure context in Context>>#runSimulated:contextAtEachStep: incorrectly acts like valueUninterruptably, forcing a return from the home context even when the execution shall not continue. This was introduced to support situations like

[Context runSimulated: [2/0]] on: ZeroDivide do: [:ex | ex return: 1]

but when the execution is actually to abort, this behavior makes no sense. Hmm... maybe ensure: should have an aborting variable and cull it to aBlock? ;-) Or how else could we find out from within that specific ensure block that we must not try to proceed?

squeak-smalltalk / squeak-object-memory

"Computation has been terminated" when terminating a process while resuming from an exception #112