Scripting

[jeb2239]

Does hi allow us to execute scripts. In a way similar to doing python file.py. hi file.hob seems to open a repl after evaluating the module.

[kthielen]

Yes as you say, if you run 'hi file.hob' then it should load the file (picking up new type/term definitions, compiling and evaluating code as necessary) and then put you at a prompt where you can use those definitions if you want.

If you just want to run the script to completion and then exit, you can use the -x option. You can also use the -e option to evaluate a single expression and the -s option to silence the usual text output.

$ hi -x -s -e '[{a=i,b=i*i,c=i*i*i}|i<-[0..]]'
a  b   c
- -- ---
0  0   0
1  1   1
2  4   8
3  9  27
4 16  64
5 25 125
6 36 216
7 49 343
8 64 512
9 81 729
...
$

You can do several things at the same time through hi via I/O multiplexing, because it sits in an epoll/kqueue loop (on Linux/BSD) with stdin just one of the watched FDs. We use this feature to e.g. react to updates in several different files (for indexing, alerting, etc) and also service queries at the same time.

[jeb2239]

Can we set up a GitHub wiki for this project? I would be nice to have a place where we could add notes and documentation and relevant papers. I was actually wondering about which paper you used for the pattern matching compiling method. Not for the regex ( I believe your method came from a HMU's theory of computation text book) but for tuples and such.

[kthielen]

Good idea. I just made a blank page here with the default github wiki feature:

https://github.com/Morgan-Stanley/hobbes/wiki

I can put together some notes on pattern match compilation. I have a whole presentation on my work machine that goes over the process.

The regex compilation method is essentially just Ken Thompson's from the late 60s, as you say very standard. We get a small speed improvement by translating DFAs to machine code functions directly (rather than the table interpreters that most regex implementations use), also there are a couple of small adjustments made to integrate regex matching with data match tables (since one row could match at the regex but not another pattern, and you want to reuse the knowledge you gained with the matching regex in subsequent rows rather than redundantly running the regex match again).

[jeb2239]

Potential Windows replacement for futex http://preshing.com/20111124/always-use-a-lightweight-mutex/

[kthielen]

I'm not sure if you can use critical sections in shared memory in Windows. We could even just use a spinlock.

It might be a while before Windows happens. I've been pulled into a few new issues related to structured logging.

[jeb2239]

I am trying to translate the following ocaml code to hobbes using the recursively defined list

let encode list =
    let rec aux count acc = function
      | [] -> [] (* Can only be reached if original list is empty *)
      | [x] -> (count+1, x) :: acc
      | a :: (b :: _ as t) -> if a = b then aux (count + 1) acc t
                              else aux 0 ((count+1,a) :: acc) t in
    List.rev (aux 0 [] list);;

Output:

encode ["a";"a";"a";"a";"b";"c";"c";"a";"a";"d";"e";"e";"e";"e"];;
- : (int * string) list =
[(4, "a"); (1, "b"); (2, "c"); (2, "a"); (1, "d"); (4, "e")]

I know hobbes requires you to explicitly unroll lists before pattern matching against them. I have not seen any examples of how to destructure longer lists.

It should be something like this correct? Only this doesn't compile.

rlencode :: (^x.(()+([char]*x))) -> (^x.(()+(([char]* int)*x)))
rlencode s =
    let aux count acc =
      match unroll(acc) with
      | |0=()|-> nil()
      | |1=(h,nil())| -> cons((count+1,x),acc)
      | |1=(a,|1=(b,t)|) -> if a=b then aux(count+1,acc,t)
                            else aux(0,cons((count+1,a),acc),t) in
      aux(0,nil(),s)

[kthielen]

The second level doesn't type-check because it needs to be unrolled. This is doable but we tend to match on arrays, so working with linked lists might be overly painful compared to ocaml.

Matching is actually heavily overloaded, and there's a "HasCtor" constraint (to match "HasField") that is really the right way to plug in the logic you want here. However "HasField" can work too, and if we feed in this instance definition to define "record-like field lookup" on lists:

instance SLookup (^x.(()+(a*x))) "cons" (()+(a*(^x.(()+(a*x))))) where
  slookup = unroll

Then if that instance definition has been introduced (e.g. in an imported .hob file) then we can project out a "cons" field from any list type, and this can be used in pattern matching like:

> match cons(1,cons(2,cons(3,nil()))) with | {cons=|1=(1,{cons=|1=(2,{cons=|1=(3,{cons=|0=()|})|})|})|} -> "one two three" | _ -> "beats me"
"one two three"
> match cons(1,cons(2,cons(3,cons(4,nil())))) with | {cons=|1=(1,{cons=|1=(2,{cons=|1=(3,{cons=|0=()|})|})|})|} -> "one two three" | _ -> "beats me"
"beats me"

It's not the nicest-looking match, but hopefully the idea is clear enough. It's pretty useful.

[jeb2239]

Us there an example of you using arrays and matching against them in the standard library?

Sent from my Motorola XT1650 using FastHub

[kthielen]

Probably not in there, but array matching wouldn't be a good fit for your function above either because array matching requires specific-length matches. There's probably something interesting that we could do with data type overloading and match for arrays but I haven't had time to work on it.

Here's one way to do it, should be pretty fast:

$ cat rle.hob

rle xs = toArray(
  foldl(\r x.
    match unroll(r) with |
    |1=(h,t)| where h.0 == x -> do {h.1 <- (h.1+1); return r}
    | _ -> cons((x,1L),r),

    nil(),
    xs
  )
)

$ ./hi -s rle.hob
> rle("aaaabbbaacccccccc")
'c' 8
'a' 2
'b' 3
'a' 4

[jeb2239]

Wait so all pattern matching just de sugars into accessor functions?

Sent from my Motorola XT1650 using FastHub

[kthielen]

Pattern matching is not primitive and yes all match expressions get desugared into simpler expressions. That doesn't mean access via function calls necessarily, but it combines well with all of the overloading you can do with type classes (e.g.: Phil Wadler's "views" for free!).