search

MobilityData / gtfs-realtime-validator

Java-based tool that validates General Transit Feed Specification (GTFS)-realtime feeds
Other
41 stars 10 forks source link

Support a production database for running as a hosted web server instance #56

Open isabelle-dr opened 2 years ago

isabelle-dr commented 2 years ago

Issue by barbeau Nov 6, 2017 Originally opened as https://github.com/CUTR-at-USF/gtfs-realtime-validator/issues/308

Summary:

We currently have a hosted instance of the validator at http://transittools.forest.usf.edu/, but it's just a running instance of the JAR snapshot, which means that it's using Hibernate with HSQLDB. This gets unstable when running in production and inserting a lot of data. Plus, this virtualized machine only has 4GB of RAM.

There are a number of options for a permanent hosted instance of the validator. Ideally we'd get it running on a platform-as-a-service like Amazon Lamba (etc.) or Google App Engine, as there should be relatively low traffic and both of those services have a free tier. However, I believe both of the above require changes to the project to leverage Hibernate.

For now, I believe the most straight-forward path for something stable in the near term will be to leverage another internal server at CUTR for hosting the database server, which will free up transittools for just the validator.

@Suryakandukoori I believe the other CUTR virtualized machine already has MS SQL Server running on it, so let's just re-use this instead of installing a new DBMS (e.g., MySQL).

isabelle-dr commented 2 years ago

Comment by barbeau Nov 22, 2017

Link to more documentation on setting up Hibernate for MS SQL Server (and other databases too): https://github.com/CUTR-at-USF/gtfs-realtime-validator/blob/master/CONFIG.md#database

Direct link to MS SQL Server config: https://stackoverflow.com/questions/3585544/how-to-configure-hibernate-config-file-for-sql-server/3588652#3588652

isabelle-dr commented 2 years ago

Comment by barbeau Nov 22, 2017

I believe you'll also need to install the latest MS SQL Server JDBC drivers on transittools too: https://docs.microsoft.com/en-us/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server

isabelle-dr commented 2 years ago

Comment by barbeau Feb 20, 2018

As @Suryakandukoori worked on this, he discovered that several of the SQL statements behind some of the Hibernate view model objects do not work with Microsoft SQL Server. Therefore, to accomplish this task we need to abstract some of these SQL statements so they work beyond just HSQLDB.

isabelle-dr commented 2 years ago

Comment by barbeau Dec 15, 2020

There is some work started on compatibility with MS SQL Server in this PR - https://github.com/CUTR-at-USF/gtfs-realtime-validator/pull/323.

barbeau commented 2 years ago

As mentioned in https://github.com/MobilityData/gtfs-realtime-validator/issues/88, there has been some work in that issue and referenced PR towards compatibility with Microsoft SQL Server. Alternate production DBMS could also be investigated.