Persisting and restoring the Log: user level functions would be nice

[habbler]

Hello Alberto,

shouldn't there be user level functions in the Logged module to allow fetching and setting the log?

I wan't to persist the log to disk. Currently I would have to use Log from the internal package together with getData and setData.

Do I need to to persist all components of Log?

I am also not sure how to set all the variables of Log correctly when setting the restored log I have retrieved from the disk before starting the computation.

Rene.

[agocorona]

Hi, For what purpose you need it? Probably to save state and continue execution later.

I have to create a basic save-stop-recovery-continue execution functionality.

The structure is: Log (recovery mode or not) (pointer to the remaining elements not read) (all elements in reverse order)

Really only is necessary to persist the last element since it has all the log (reversed)

to save the state:

saveAndStop file= do
   Log recovery _ log <- getData
   if recovery then return () else do
        writeFile file log
        empty

loadAndRestart file  proc= do
     log <- readFile file
     setData True (reverse log) log
     proc

main= keep $ loadAndRestart  "file" $ do
     logged dothis
     logged dothat
     saveAndStop "file"
     logged doOther
     ....

May execute, stop and continue if re-executed.

But I have to test it. the problem is that since TransIO can execute many threads, saveAndStop my write the log many times, one for each thread. Tomorrow I will try to make it work.

[habbler]

Hi Alberto, Thankyou for the speed response. Yes to stop and continue execution later. Basically suspend and resume. No hurry though. I am just playing around.

I would have thought that when we have f = do logged f1 logged f2 logged f3 suspend later steps

That there would be three entries in the log, and that we need all three, as I will be running f again and each logged statement will read one of the log entries each?

Rene.

[agocorona]

Yes the file will have three entries. In a general suspend primitive for the TransIO monad, the file may have different many logs. For example :

f = do 
    logged $ f1
    r <-  logged $ choose [1.. 100]
    logged $ print r
    suspend
    later steps

then the file will have 100 logs!!! of three elements each. but the first element is common to all the 100 logs. Since logs are generally very short, repetitions may not be a big problem in principle.

The logs are short since logged erases the log steps that are inside. For example if f1 has dozens of log steps inside, it does not matter, there will be three entries in each log when suspend is executed.

Then the restart would need to re-execute the 100 logs in 100 threads.

[agocorona]

Hi Rene:

I solved the single threaded case that may be useful for you: suspend, checkpoint and restore defined here, with an example:

{-# LANGUAGE   ScopedTypeVariables #-}

import Transient.Base
import Control.Applicative
import Control.Monad.IO.Class
import Transient.Internals
import Transient.Logged
import Control.Exception

suspend :: String -> a -> TransIO a
suspend file x = do
   Log recovery _ log <- getData `onNothing` return (Log False [] [])
   if recovery then return x else do
        liftIO $ writeFile file $ show log
        exit x

checkpoint :: String -> TransIO ()
checkpoint file = do
   Log recovery _ log <- getData  `onNothing` return (Log False [] [])
   if recovery then return () else do
        liftIO $ writeFile file  $ show log
        return ()

restore :: String -> TransIO a -> TransIO a
restore file  proc = do
     log <- liftIO $ (readFile file >>= return . read) `catch` (\(e ::SomeException) -> return [])
     setData $ Log True (reverse log) log
     proc

main = keep $ restore "file" $ do
     logged $ liftIO $ print "hello"
     suspend "file" ()
     logged $ liftIO $ print "world"

[habbler]

Thankyou!

[agocorona]

Added restore, suspend and checkpoint for the general case:

https://github.com/agocorona/transient/commit/b3a8b6d2fc8f4cf0543684135cf7919e4827a399

If you can test that it works for you, it would be wonderful