mathias-brandewinder / CNTK.FSharp

MIT License
67 stars 9 forks source link

add epoch to summary and add option to cancel the training and fix logistic example #23

Closed AviAvni closed 5 years ago

mathias-brandewinder commented 6 years ago

@AviAvni sorry for the delay in responding. I have been giving some thought to the cancellation feature, which is a good idea, but I am not convinced that the CancellationToken should be exposed in the Configuration record. How about exposing a method on the Learner instead, along these lines:

type Learner () =
        // code removed for brevity
        let mutable cancellation : CancellationTokenSource option = None
        member this.Interrupt () =
            match cancellation with
            | None -> ignore ()
            | Some(token) -> token.Cancel ()

        member this.learn 
            (source:MinibatchSource) 
            (featureStreamName:string, labelsStreamName:string) 
            (config:Config) 
            (spec:Specification) =

            // code removed for brevity
            let cancel = new CancellationTokenSource ()
            cancellation <- Some cancel
            let token = cancel.Token

            let rec learnEpoch (step,epoch) = 
                // code removed for brevity
                if epoch <= 0
                // we are done : return function
                then predictor
                elif token.IsCancellationRequested
                then predictor
                else //

That way, the base scenario (no interruption, just learn) is still easy, and if interruptions are desired, it can be done along these lines (based on the MIST-CNN example):

open System.Threading.Tasks
let usingTask = 
    Task.Factory.StartNew(fun _ ->
        trainer.learn minibatchSource (featureStreamName,labelsStreamName) config spec)
trainer.Interrupt ()
usingTask.Result

Do you see an issue with this approach?