"Production" use

[smolck]

I would like to use Opium as the backend of a website, and I was wondering if this library is suited to "production" use (in my case serving an SPA and a small RESTful API)? Are there any prerequisites to hosting a proper website with Opium, or is running an application built with Opium on a server all that's necessary?

This is a great project BTW! Keep up the good work! ;)

[anuragsoni]

Hi @smolck!

I really appreciate the interest in using this library. Opium is a set of opinionated set of abstractions on top cohttp Cohttp itself is a pretty mature library which is used by a lot of people in the OCaml community. I think serving a small restful API shouldn't be a problem. As for deployment, its pretty straightforward to either compile a binary for the necessary platform, or just deploy things with a docker container.

That being said, as far as "production" ready is concerned, i'd say you should also look at other libraries you might need for your application. Things like database libraries, etc. Apart from that if you do decide to stick with cohttp/opium, i'd be very interested your experience. Please don't hesitate to open issues and/or post queries on the ocaml form

[smolck]

I really appreciate the interest in using this library. Opium is a set of opinionated set of abstractions on top cohttp Cohttp itself is a pretty mature library which is used by a lot of people in the OCaml community. I think serving a small restful API shouldn't be a problem. As for deployment, its pretty straightforward to either compile a binary for the necessary platform, or just deploy things with a docker container.

Thank you! I thought deployment was that straightforward, but I wanted to make sure.

That being said, as far as "production" ready is concerned, i'd say you should also look at other libraries you might need for your application. Things like database libraries, etc. Apart from that if you do decide to stick with cohttp/opium, i'd be very interested your experience. Please don't hesitate to open issues and/or post queries on the ocaml form

Okay, will do! I have already started writing the backend in Haskell's Scotty, but after hearing what you said above I think I'll go for Opium on the backend instead! I'll be sure to open issues as/if necessary.

As an aside, one thing I have noticed is there's no documentation for serving HTML with Opium, and I'd like to contribute an example. How would you feel about a PR for a more extended example of serving an Elm SPA and a REST API? Maybe a separate repo for that would be more fitting, with a link to it in the examples/ README? I can also contribute a small example, but if I get the time I'd like to create a more involved example so that others have more material from which they can learn Opium. What do you think?

Something I am wondering about though is how does one set up CORS with Opium? Note that I haven't looked through all of the examples to see if any of them show an example of this, so I apologize if this question is clearly answered in one of the examples.

One more question, concerning security; how secure is Opium? I presume it prevents directory traversal attacks, but is it secure against more obscure attacks? I want to make sure I don't assume anything in this area.

Thank you for being welcoming to this newcomer! I really appreciate it ;)

[anuragsoni]

As an aside, one thing I have noticed is there's no documentation for serving HTML with Opium, and I'd like to contribute an example. How would you feel about a PR for a more extended example of serving an Elm SPA and a REST API? Maybe a separate repo for that would be more fitting, with a link to it in the examples/ README? I can also contribute a small example, but if I get the time I'd like to create a more involved example so that others have more material from which they can learn Opium. What do you think

I think if its a small self contained example it'd be nice to have it in the repo. But if its a bigger sample showing integrating with other languages like Elm, i'd be happy to include a link to a separate repo that has more elaborate instructions describing what the example is about.

As for security, opium itself mostly just provides a mechanism for a router and adding/composing middlewares. The actual heavy lifting is done by cohttp. That being said, there might definitely be gaps. Specifically for CORS opium doesn't add any headers by default and that's up to the individual application for now.

Thank you for being welcoming to this newcomer! I really appreciate it ;)

You are very welcome :)

[shonfeder]

re: authentication and REST, it may be worth having a look at https://github.com/roddyyaga/ocoi. I only found this project last week, and have not used it myself, but the aim is to build up a framework that takes care of a lot of boilerplate, and is aimed at supporting SPAs I believe.

re: deployment, I am using opium to run my (dead simple, could just be static currently) site. The deployment is incredibly naive, as I'm just deploying from source and building on the target (manually, at that) It may be too naive to help, but you can see my notes here https://github.com/shonfeder/shonfeder_net/blob/master/PROJECT.org#design-notes

re: an example of serving HTML, I'm on the tail end of writing up a tutorial that includes serving HTML generated via Tyxml. The WIP work is located at https://gitlab.com/shonfeder/ocaml_webapp/tree/master. You can see the tutorial itself in the tutorial.org file (but you'll have to view the org source code, as it has a number of file includes that aren't rendered by the simplistic gitlab processing). I'm hoping to have this published properly within the next couple days. The more tutorials and material showcasing working projects the merrier! :)

re: production usage, are there any known examples of opium being used in production? If not, someone has to be the first to try, but it would be nice to be clear that this is a first, and that the users should expect to do some development and trail blazing along the way :) (I would imagine scotty already has been tested in production, tho a quick google didn't turn up anything obvious.)

[anuragsoni]

re: production usage, are there any known examples of opium being used in production? If not, someone has to be the first to try,

https://github.com/anuragsoni/routes/issues/59 This issue on my routing library indicates that there is atleast someone using Opium for their project :smile: The download numbers for opium and scotty are also comparable. But that being said I'd guess that the number of people using OCaml for this usecase is already small, and number of people using opium within that group will be a smaller subset.

[anuragsoni]

re: deployment, I am using opium to rum my (dead simple, could just be static currently) site. The deployment is incredibly naive, as I'm just deploying from source and building on the target (manually, at that) It may be too naive to help, but you can see my notes here https://github.com/shonfeder/shonfeder_net/blob/master/PROJECT.org#design-notes

Something else that can be an option for deployment is to build using docker. Platforms like Heroku, AWS, Google cloud, dokku + your own server, can make the deployment really easy. And using the multi stage docker builds the resulting images will be pretty small in size too. I have an example Dockerfile that i use at https://github.com/anuragsoni/lattice/blob/fb593cdce9608e39218dd0b3c97440eebf57ba46/Dockerfile

[smolck]

re: authentication and REST, it may be worth having a look at https://github.com/roddyyaga/ocoi. I only found this project last week, and have not used it myself, but the aim is to build up a framework that takes care of a lot of boilerplate, and is aimed at supporting SPAs I believe.

Thank you! I may look into that in the future, but as my needs are relatively simple for this project I think I'll stick to Opium and see how it goes, maybe even documenting the process in some form as you've done with shonfeder_net.

re: deployment, I am using opium to rum my (dead simple, could just be static currently) site. The deployment is incredibly naive, as I'm just deploying from source and building on the target (manually, at that) It may be too naive to help, but you can see my notes here https://github.com/shonfeder/shonfeder_net/blob/master/PROJECT.org#design-notes

This looks like a good example to me! It will give me some guidance on setting everything up. I'd especially be interested to see what you do for the "Set up ci/cd so new updates to source are propagated to server automatically" part, or what you have in mind?

re: an example of serving HTML, I'm on the tail end of writing up a tutorial that includes serving HTML generated via Tyxml. The WIP work is located at https://gitlab.com/shonfeder/ocaml_webapp/tree/master. You can see the tutorial itself in the tutorial.org file (but you'll have to view the org source code, as it has a number of file includes that aren't rendered by the simplistic gitlab processing). I'm hoping to have this published properly within the next couple days. The more tutorials and material showcasing working projects the merrier! :)

This is great! Not only for project structure but also for an example of idiomatic usage, which will again be a good reference for me. Thank you!

. . . but it would be nice to be clear that this is a first, and that the users should expect to do some development and trail blazing along the way :)

I feel that this is sometimes the case with OCaml in general, that there will be less tutorials than other mainstream options and languages. But I'm prepared for that ;)

Overall, thank you for the detailed answer @shonfeder!

[smolck]

Something else that can be an option for deployment is to build using docker. Platforms like Heroku, AWS, Google cloud, dokku + your own server, can make the deployment really easy. And using the multi stage docker builds the resulting images will be pretty small in size too. I have an example Dockerfile that i use at https://github.com/anuragsoni/lattice/blob/fb593cdce9608e39218dd0b3c97440eebf57ba46/Dockerfile

@anuragsoni Thank you for the example Dockerfile! I really need to set aside the time and learn Docker, but have yet to do so. Maybe now will be my chance to try it out! ;) It is definitely a good option.

[smolck]

. . . But that being said I'd guess that the number of people using OCaml for this usecase is already small, and number of people using opium within that group will be a smaller subset.

Yes, but there don't appear to be a whole lot of libraries in this area either, at least that I've found. So hopefully that balances things out ;)

I will say that Opium is very easy to use. You have definitely hit your design goal that "A programmer should be instantly productive when starting out."

Specifically for CORS opium doesn't add any headers by default and that's up to the individual application for now.

@anuragsoni Would CORS headers be added using Rock.Middleware.create, or something else? (From reading the README I'm guessing that CORS would be set up via middleware, is that right?)

[anuragsoni]

@anuragsoni Would CORS headers be added using Rock.Middleware.create, or something else? (From reading the README I'm guessing that CORS would be set up via middleware, is that right?)

Yes, you can create a middleware for that. Something like

let cors_middleware =
  let filter handler req =
    handler req >>| fun response -> 
      (* add cors header to response *)
  in 
  Rock.Middleware.create ~name:"middleware" ~filter

[shonfeder]

I'd especially be interested to see what you do for the "Set up ci/cd so new updates to source are propagated to server automatically" part, or what you have in mind?

@smolck, my hope here is to use https://github.com/ocurrent/ocurrent to run a daemonized job that watches the repo changes, and then either

• just pulls down the source (if only static files have changed)
• or rebuilds the app and then swaps out the systemd service (if the source code has changd)

I haven't made any steps towards getting that in place yet though.

Not only for project structure but also for an example of idiomatic usage, which will again be a good reference for me.

I should note, I'm not at all sure that my usage is idiomatic! (I'm hoping more experienced users/devs will help correct the tutorial if it isn't, once it's live.)

This issue on my routing library indicates that there is atleast someone using Opium for their project

@anuragsoni routes looks awesome! Would have been using that for stuff I have going on if I'd found it earlier, as the "stringly typed programming" aspect of naive opium routing always makes me feel a bit iffy.

[shonfeder]

I feel that this is sometimes the case with OCaml in general, that there will be less tutorials than other mainstream options and languages. But I'm prepared for that ;)

@smolck I agree! I think OCaml (the language ane ecosystem) is always a great option for devs interested in making their own solutions and/or building out existing stuff. :)

[smolck]

@anuragsoni routes looks awesome! Would have been using that for stuff I have going on if I'd found it earlier, as the "stringly typed programming" aspect of naive opium routing always makes me feel a bit iffy.

As an aside, how would one go about using routes in an Opium app? I've felt the same way about using strings, and if I can use routes instead then I'd like to.

[anuragsoni]

As an aside, how would one go about using routes in an Opium app? I've felt the same way about using strings, and if I can use routes instead then I'd like to.

I included an example for using it with opium: https://github.com/anuragsoni/routes/blob/3d7d25e11be0f13d855cad4c659944c0ebb6ec52/example/opium_example.ml

[anuragsoni]

routes looks awesome! Would have been using that for stuff I have going on if I'd found it earlier, as the "stringly typed programming" aspect of naive opium routing always makes me feel a bit iffy.

Thanks, i've been reasonably satisfied with the state its in now, but there is room for improvement for the underlying trie, and the public api :)

[smolck]

@anuragsoni Would CORS headers be added using Rock.Middleware.create, or something else? (From reading the README I'm guessing that CORS would be set up via middleware, is that right?)

Yes, you can create a middleware for that. Something like

let cors_middleware =
  let filter handler req =
    handler req >>| fun response -> 
      (* add cors header to response *)
  in 
  Rock.Middleware.create ~name:"middleware" ~filter

@anuragsoni What is the >>| operator doing in that snippet? If I try to use that same code I get an Unbound value >>| error.

EDIT: Nevermind, I've found http://rgrinberg.com/posts/middleware-intro/#examples and now know that >>| is from the Async module. I'll look into it. Thanks for the example! ;)

[anuragsoni]

EDIT: Nevermind, I've found http://rgrinberg.com/posts/middleware-intro/#examples and now know that >>| is from the Async module. I'll look into it. Thanks for the example! ;)

The post might be a little older. Opium is based around Lwt now. >>| is the same as Lwt.map (https://docs.mirage.io/lwt/Lwt/index.html#val-map)

I believe in Lwt the operator is defined as >|= Its defined in the Lwt.Infix module

[smolck]

I believe in Lwt the operator is defined as >|= Its defined in the Lwt.Infix module

Yes, that works! This is what I've come up with (based on your example) that should be the equivalent of simpleCors in Haskell's wai-cors library:

open Lwt.Infix

let cors_middleware =
  let filter handler req =
    handler req >|= fun response -> 
      let headers = 
        Cohttp.Header.add 
          (Rock.Response.headers response)
          "Access-Control-Allow-Origin"
          "*" 
      in
      { response with headers }
  in 
  Rock.Middleware.create ~name:"middleware" ~filter

Thanks again! BTW, do you want me to close this issue now since my original questions have been answered?

[anuragsoni]

Yeah, i believe this issue can be closed. But you can still comment on this or open a new issue if something comes up later :smile: