search

inhabitedtype / ocaml-webmachine

A REST toolkit for OCaml
Other
221 stars 31 forks source link

Dependency on calendar breaks usage with mirage #73

Closed ansiwen closed 6 years ago

ansiwen commented 7 years ago

We are using webmachine in a mirage unikernel, and version 0.4.0 contains a dependency on calendar which breaks xen builds:

Error: No implementations provided for the following modules:
         Str referenced from /home/svanders/.opam/4.04.1/lib/calendar/calendarLib.cmxa(CalendarLib)
         Unix referenced from /home/svanders/.opam/4.04.1/lib/calendar/calendarLib.cmxa(CalendarLib)

Installed package versions are:

$ opam list
# Installed packages for 4.04.1:
asn1-combinators               0.1.3  Define ASN.1 grammars in OCaml
astring                        0.8.3  Alternative String module for OCaml
base                          v0.9.2  Full standard library replacement for OCam
base-bigarray                   base  Bigarray library distributed with the OCam
base-bytes                      base  Bytes library distributed with the OCaml c
base-threads                    base  Threads library distributed with the OCaml
base-unix                       base  Unix library distributed with the OCaml co
base64                         2.1.2  For OCaml
biniou                        1.0.13  Binary data format designed for speed, saf
bos                            0.1.6  Basic OS interaction for OCaml
calendar                      2.03.2  Library for handling dates and times in yo
camlp4                        4.04+1  Camlp4 is a system for writing extensible 
cmdliner                       1.0.0  Declarative definition of command line int
cohttp                        0.22.0  HTTP(S) library for Lwt, Async and Mirage
conduit                       0.15.3  Network connection library for TCP and SSL
conf-gmp                           1  Virtual package relying on a GMP lib syste
conf-m4                            1  Virtual package relying on m4
conf-perl                          1  Virtual package relying on perl
conf-pkg-config                  1.0  Virtual package relying on pkg-config inst
conf-which                         1  Virtual package relying on which
cppo                           1.5.0  Equivalent of the C preprocessor for OCaml
cpuid                          0.1.1  Detect CPU features
crunch                         2.0.0  Convert a filesystem into a static OCaml m
cstruct                        2.4.1  access C structures via a camlp4 extension
depext                         1.0.4  Query and install external dependencies of
dispatch                       0.3.0  Path-based dispatching for client- and ser
dns                           0.20.0  DNS client and server implementation
duration                       0.1.0  Conversions to various time units
easy-format                    1.2.0  High-level and functional interface to the
fieldslib                     v0.9.0  Syntax extension to define first class val
fmt                            0.8.3  OCaml Format pretty-printer combinators
fpath                          0.7.2  File system paths for OCaml
functoria                      2.0.2  A DSL to organize functor applications.
functoria-runtime              2.0.0  
gmp-xen                        6.0.0  The GNU Multiple Precision Arithmetic Libr
hashcons                       1.0.1  OCaml hash-consing library
io-page                        1.6.1  Allocate memory pages suitable for aligned
ipaddr                         2.7.2  IP (and MAC) address manipulation
jbuilder                   1.0+beta8  Fast, portable and opinionated build syste
logs                           0.6.2  Logging infrastructure for OCaml
lwt                            2.7.1  Monadic promises and concurrent I/O
magic-mime                     1.0.0  Convert file extensions to MIME types
minios-xen                       0.9  A minimal OS for running under the Xen hyp
mirage                         3.0.2  The MirageOS library operating system
mirage-block                   1.0.0  Utilities and module definitions for deali
mirage-block-lwt               1.0.0  Utilities and module definitions for deali
mirage-bootvar-xen             0.4.0  Library for reading MirageOS unikernel boo
mirage-channel                 3.0.0  MirageOS channels
mirage-channel-lwt             3.0.0  MirageOS channels
mirage-clock                   1.2.0  Libraries and module types for two kinds o
mirage-clock-freestanding      1.2.0  
mirage-clock-lwt               1.2.0  Libraries and module types for two kinds o
mirage-conduit                 2.3.1  Virtual package for the MirageOS Conduit t
mirage-console                 2.2.0  A Mirage-compatible Console library for Xe
mirage-console-lwt             2.2.0  A Mirage-compatible Console library for Xe
mirage-console-xen             2.2.0  A Mirage-compatible Console library for Xe
mirage-console-xen-proto       2.2.0  Protocol for communicating with Xen consol
mirage-device                  1.0.0  Foundational module types for devices.
mirage-dns                     2.7.0  Virtual package for the MirageOS DNS trans
mirage-entropy                 0.4.0  Entropy source for MirageOS unikernels.
mirage-flow                    1.2.0  Various implementations of the MirageOS FL
mirage-flow-lwt                1.2.0  Various implementations of the MirageOS FL
mirage-fs                      1.0.0  MirageOS filesystem utilities
mirage-fs-lwt                  1.0.0  Module type definitions for filesystems, w
mirage-http                    3.1.0  MirageOS-compatible implementation of the 
mirage-kv                      1.0.0  MirageOS utilities for interfacing with ke
mirage-kv-lwt                  1.0.0  MirageOS utilities for interfacing with ke
mirage-logs                    0.3.0  A reporter for the Logs library that write
mirage-net                     1.0.0  MirageOS TCP/IP networking library
mirage-net-lwt                 1.0.0  MirageOS TCP/IP networking library
mirage-net-xen                 1.7.0  Ethernet network device driver for MirageO
mirage-no-solo5                    1  Virtual package conflicting with mirage-so
mirage-os-shim                 3.0.0  Portable shim for MirageOS APIs
mirage-profile                 0.7.0  Collect profiling information
mirage-protocols               1.1.0  Module type definitions for network protoc
mirage-protocols-lwt           1.1.0  
mirage-random                  1.0.0  Random signatures for MirageOS, and an imp
mirage-runtime                 3.0.0  A bundle of useful runtime functions for a
mirage-stack                   1.0.0  Module type definitions for network stacks
mirage-stack-lwt               1.0.0  Module type definitions for network stacks
mirage-time                    1.0.0  Module type definitions for dealing with t
mirage-time-lwt                1.0.0  Module type definitions for dealing with t
mirage-types                   3.0.0  Module type definitions for Mirage-compati
mirage-types-lwt               3.0.0  Lwt module type definitions for Mirage-com
mirage-xen                     3.0.0  MirageOS library for Xen compilation
mirage-xen-minios              0.8.0  Xen MiniOS guest operating system library
mirage-xen-ocaml               3.0.0  MirageOS headers for the OCaml runtime
mirage-xen-posix               2.6.0  MirageOS library for posix headers
nocrypto                       0.5.4  Simpler crypto
oasis                          0.4.8  Tooling for building OCaml libraries and a
ocaml-compiler-libs           v0.9.0  OCaml compiler libraries repackaged
ocaml-migrate-parsetree          1.0  Convert OCaml parsetrees between different
ocaml-src                     4.04.0  Compiler sources
ocamlbuild                    0.11.0  OCamlbuild is a build system with builtin 
ocamlfind                      1.7.1  A library manager for OCaml
ocamlgraph                     1.8.7  A generic graph library for OCaml
ocamlify                       0.0.1  Include files in OCaml code
ocamlmod                       0.0.8  Generate OCaml modules from source files
ocb-stubblr                    0.1.1  OCamlbuild plugin for C stubs
ocplib-endian                    1.0  Optimised functions to read and write int1
ounit                          2.0.0  Unit testing framework loosely based on HU
parse-argv                     0.0.3  process strings into sets of command-line 
ppx_ast                       v0.9.1  OCaml AST used by Jane Street ppx rewriter
ppx_core                      v0.9.0  Standard library for ppx rewriters
ppx_deriving                     4.1  Type-driven code generation for OCaml >=4.
ppx_driver                    v0.9.1  Feature-full driver for OCaml AST transfor
ppx_fields_conv               v0.9.0  Generation of accessor and iteration funct
ppx_metaquot                  v0.9.0  Write OCaml AST fragment using OCaml synta
ppx_optcomp                   v0.9.0  Optional compilation for OCaml
ppx_sexp_conv                 v0.9.0  Generation of S-expression conversion func
ppx_tools                        5.0  Tools for authors of ppx rewriters and oth
ppx_tools_versioned            5.0.1  A variant of ppx_tools based on ocaml-migr
ppx_traverse_builtins         v0.9.0  Builtins for Ppx_traverse
ppx_type_conv                 v0.9.0  Support Library for type-driven code gener
ptime                          0.8.3  POSIX time for OCaml
randomconv                     0.1.0  Convert from random bytes to random native
re                             1.7.1  RE is a regular expression library for OCa
result                           1.2  Compatibility Result module
rresult                        0.5.0  Result value combinators for OCaml
sexplib                       v0.9.1  Library for serializing OCaml values to an
shared-memory-ring             1.3.0  Shared memory rings for RPC and bytestream
stdio                         v0.9.0  Standard IO library for OCaml
stringext                      1.4.3  Extra string functions for OCaml
tcpip                          3.1.0  Userlevel TCP/IP stack
tls                            0.8.0  Transport Layer Security purely in OCaml
topkg                          0.9.0  The transitory OCaml software packager
uchar                          0.0.1  Compatibility library for OCaml's Uchar mo
uri                            1.9.2  RFC3986 URI/URL parsing library
webmachine                     0.4.0  A REST toolkit for OCaml
x509                           0.5.3  X.509 certificate (RFC5280) library
xen-evtchn                     1.0.7  Xen event channel bindings.
xen-gnt                        2.2.3  Xen grant table bindings
xenstore                       1.3.0  Xenstore protocol clients and server
yojson                         1.3.3  Yojson is an optimized parsing and printin
zarith                           1.4  Implements arithmetic and logical operatio
zarith-xen                       1.4  Implements arithmetic and logical operatio
seliopou commented 7 years ago

Seems like the right thing to do here is take an explicit time source in either the webmachine functor, or the dispatch function. Either way, this time source would have something like the following type:

type time_source : unit -> int

For Unix, this could just be gettimeofday or something similar. For Async or Lwt, this would read some state somewhere that was asynchronously updated by a timer.

This seems like a nice self-contained contribution. If you're interested in writing it I would review and merge. Give me a heads up on which approach you're thinking of beforehand, though.

ansiwen commented 7 years ago

I'm happy to help, but I'm pretty much an Ocaml-Novice still, so I can't really see now, which of the two approaches would be better. Maybe the more simple one? ;-) If you assist by leading me through that, I can certainly try. First question would be: shouldn't be there already a time source somewhere? I can't imagine such a basic thing is missing yet.

hannesm commented 7 years ago

ping. would be nice if someone can fix this

ansiwen commented 7 years ago

I'm happy to execute it with a small amount of direction and/or someone available to ask, in order to make it time efficient. (Don't have too much time to waste, as probably everybody, so I would not like to research for hours on the internet if problems arise.)

ansiwen commented 7 years ago

Yesterday I found out I can't upgrade to mirage 3.0.5, because it comes with implicit dependencies on cohttp higher than 0.20.2. So, our interest to solve that is even higher now. I could reserve like 3-5 hours a week on that. I would just ask for assistance, to keep it efficient. So, for some tutoring from one of you guys, you will get this! :smiley_cat:

hannesm commented 7 years ago

I can see two paths forward:

yomimono commented 7 years ago

I investigated this for a demo I put together involving webmachine and Mirage in March, and hacked out the bits requiring calendar at https://github.com/yomimono/ocaml-webmachine/commit/33f825de31f18f13d3d8f2fe558a0950f558e392 (forgive the OASIS crud, the interesting bits are at line 701 of lib/webmachine.ml and below in that view).

yomimono commented 7 years ago

@tmcgilchrist I don't think removing the functionality entirely is a good idea and I'll be very surprised if the maintainer accepts a patch that does so; I intended only to point at answers to @hannesm's first question above.

tmcgilchrist commented 7 years ago

Agreed it's probably not a good idea (just a quick hack on the train). I wanted to make the changes available if people wanted to use a branch against current master.

Building the date parser and working out how to pass in a time source will take me longer. Might get time this week to have a go at it.

ansiwen commented 7 years ago

@seliopou BTW: How about splitting the functionality into webmachine and webmachine-unix? Or is the unix-dependent part essential?

seliopou commented 7 years ago

Handling caching headers requires a time source, and parsing the headers (currently) requires calendarlib. So unfortunately it's gotta be functorized, or at the very least somehow parameterized.