Moving to Rails

[jessieay]

I estimate that moving our application from Sinatra / Jekyll to Rails will take one developer less than 1 day.

Rails is a web framework written in Ruby. Sinatra is also a web framework written in Ruby. Sinatra is usually best for very small applications. I believe we should move to Rails because our application is growing and could benefit from pulling in some parts of Rails. While it is possible to pull in these parts one by one while still using Sinatra, it will be faster to move to Rails entirely rather than basically turning our Sinatra app into a Rails app without actually using Rails. In short: let's not reinvent the Rails wheel.

Here are some benefits of Rails over Sinatra:

• Would use same database (Postgres) and language (Ruby) that we currently use, so it would be an easy move
• Rails console for creating data - http://guides.rubyonrails.org/command_line.html#rails-console
• ActiveRecord for interacting with SQL database and adding data validations - http://guides.rubyonrails.org/active_record_basics.html
• Asset pipeline for CSS - http://guides.rubyonrails.org/asset_pipeline.html
• Improve ability to do front end tests if front end and back end are tied together. We currently do not have any tests that cover our front end and back end together.

via https://blog.engineyard.com/2014/rails-vs-sinatra

Rails we reserve for larger scale applications, anything with an expectancy for many users and greater than say 15-20 end points, or places where the application can be accessed, like a URL or login portal. Sinatra is for small projects and API pullers, something that has just a few or some limited number of users and about 5-10 end points.

Considering scale, Rails will far outstrip Sinatra. As we grow our application, assuming we started in Sinatra, we’ll see that as the number of users and end points grow, the more we are implementing parts of Rails, such as ActiveRecord, in Sinatra in order to keep up. Once that starts happening, once the Sinatra app has grown difficult to manage, it’s time to rebuild it in Rails. It may be obvious over time, but timing is crucial and knowing your app will help you determine when this switch needs to take place.

Performance can also come down to the size of the application. As a Sinatra application starts to grow, we are likely to see dips in performance. With Rails, there are many ways to offload certain aspects of an application to lighten the load and improve performance. Database indexing, creating ETLs, and background jobs for the application are much easier to implement in Rails and therefore can boost performance as the application begins to grow.

[jessieay]

An anecdotal data point:

I've written ~ 10 JSON APIs with Ruby. I started several of them with Sinatra and eventually moved all but 1 of them to Rails. The one that never moved was an API with two endpoints, which is a great use case for Sinatra since it is a lighter framework.

[pkarman]

I'm fine with this. Can we draw a line somewhere with the existing PRs so that we don't create lots of conflicts with outstanding merges?

[scottdchristian]

Thanks, this helps and regardless of what we do I realize the API URI would not change, thus our users will not be affected much by a change in either framework or language. The basic question is can rails and ruby get us to where we will need to be?

Where can I go to show Rails can support the 3-18 million submissions per year. Based on most of the data coming in during business hours 5 days per week we could be seeing anywhere from roughly 20 per minute to multiple transactions per second. At a minimum this would come from 100s of users per day, but could easily come to well over 10,000 users per day. Of these manifests we'll have multiple users commenting, changing and our system tracking each manifest's payment status.

[pkarman]

I recommend reading https://blog.engineyard.com/2014/rails-vs-sinatra

Considering scale, Rails will far outstrip Sinatra. As we grow our application, assuming we started in Sinatra, we’ll see that as the number of users and end points grow, the more we are implementing parts of Rails, such as ActiveRecord, in Sinatra in order to keep up. Once that starts happening, once the Sinatra app has grown difficult to manage, it’s time to rebuild it in Rails. It may be obvious over time, but timing is crucial and knowing your app will help you determine when this switch needs to take place.

IME, if architected correctly, a web-based application like this should always bottleneck at the storage (database, search index) layer, not the app code layer. You can always scale out the web app to multiple nodes and load balance. The storage layer has ACID constraints, which means concurrency becomes the slowdown.

I.e., Rails vs Sinatra isn't the scaling issue.

[jessieay]

+1 to what Peter said. The difference between Rails and Sinatra is relevant for developers / development time, not for scaling the application.

[brandocalrissian]

+1 to scale issues will be on the data end, not on the app side where we should be able to scale horizontally. The only issue we have today around horizontal scaling is the two-phase authentication we must go through for signing a manifest.

The e-manifest server must remember something the signing API returned in response to the user's valid username and password when submitting the user's answer to one of the canned questions. We can use distributed caches to keep a single machine failure from breaking a signing ceremony, but in a perfect world we wouldn't use the answer component at all. It's simply a second password someone else can guess.

Having a user produce something he/she knows (a username and password) and something else he/she knows (an answer to a canned question) is not two-factor authentication.

[jessieay]

@brandocalrissian to clarify: these scaling issues you mention are relevant whether we are using Sinatra or Rails, correct?

[brandocalrissian]

As for moving from Sinatra to Rails, if it'll help development and is something we'll need to move to in the near to mid term, the sooner we do the move the better.

• We're already using ActiveRecord and if we want to use some other Rails library, why should we keep getting the Sinatra-wrapped versions?
• Will we still need to use Jekyll as a templating engine even with the Rails asset pipeline?

If the Rails console helps provide utility for a developer, that's nice. Supporting end-to-end integration testing out of the box is also nice (I promise not to get on my unit tests are better than integration tests soapbox; integration tests have their own value).

The scaling issues we have today do not depend on our back-end framework, runtime environment, or any other technical concern.

[jessieay]

Closing this as we decided in the affirmative and the Rails port is nearing completion