unexpected bytes_of_sexp exception

[mudrz]

setup

• creating simple record with [@@deriving sexp]

requirement

• convert record to bytes

issue the exported conversion functions by Base throw exceptions, but the functions in Sexp do not

• string_of_sexp and bytes_of_sexp throw [exception] (Sexplib.Conv.Of_sexp_error (Failure "string_of_sexp: atom needed")
• Sexp.to_string (exported by Base) converts the record to string

question

• what is the intended way to convert between sexp and bytes? Is it to use Sexp.to_string and Bytes.of_string + Bytes.to_string + Parsexp.Single.parse_string_exn or is there another way to use bytes_of_sexp?

[bcc32]

I think there is a bit of confusion around the sexp/string conversion functions. Basically, there are two pairs of functions that "go between" sexps and strings:

To encode/decode an OCaml string value in a sexp:

to sexp	from sexp
sexp_of_string	string_of_sexp
String.sexp_of_t	String.t_of_sexp

For example:

Sexp.equal
  (Sexp.Atom "hello world")
  (String.sexp_of_t "hello world")

To parse/output a sexp's textual form:

parse	output
Sexp.of_string	Sexp.to_string{,_hum,_mach}
Parsexp.Single.parse_string_exn

For example:

(* Read a sexp from a file *)

let () =
  let file_contents = Stdio.In_channel.read_all "foo.txt" in
  let sexp : Sexp.t = Sexp.of_string file_contents in
  let ocaml_value : int list =
    [%of_sexp: int list] sexp
    (* This simply expands to:

       {[
         list_of_sexp int_of_sexp sexp
       ]}
    *)
  in
  (* Do something with the [int list] *)
;;

Bytes is analogous to string, so bytes_of_sexp expects a Sexp.Atom variant, because it expects to parse a bytes value embedded in a sexp.

But what you really want is to output a Sexp.t via bytes, which would be something like:

let _ =
  let bytes =
    record
    |> sexp_of_record
    |> Sexp.to_string
    |> Bytes.of_string
  in
  (* Output the bytes somewhere *)
;;

Then, to go the other way, you want something like:

let _ =
  let record =
    (* input the bytes from somewhere *)
    bytes
    |> Bytes.to_string
    |> Parsexp.Conv.parse_string_exn
         [%of_sexp: record]
  in
  (* do something with the record *)
;;

Please let me know if this was helpful, or if you have questions.

[mudrz]

thanks Aaron, this was super helpful and explained how it works and why, closing this