Question about using Mongo DB for the android app project

[bdmckean]

Khalid, it looks like the data is relatively structured and could have been put into an SQL data base (MySQL?) for queries. Why did you choose to use a NoSQL database and why MongoDB versus others?

[kalharbi]

Yes, this dataset can absolutely fit in any relational database. However, as the volume of the data increases and the structure changes, it will become much more obvious that a relational database is not the ideal choice for storing the data. NoSQL databases are designed to address new challenges and overcome the limitations of relational databases. I will highlight the ones related to our use case.

• Scalability. The data you are working on is just a very small slice of a much larger dataset (over 600K documents). When we notice that the query/insert performance degrades severely, we can easily partition the data across multiple nodes. This is also called sharding and horizontal scaling. Our dataset is now hosted on 2 commodity servers since a single one did not handle the massive amount of data we have. The 40-year-old Relational databases were not designed to scale horizontally.
• High availability. We need to plan for the worst scenario. Disk, nodes, and data center failures are common these days. NoSQL databases have native data redundancy and automatic recovery support. This is done through replica sets and master-slave replicas. They are more fault tolerance than relational dbs.
• Flexible data modeling. MongoDB is schema-less. The schema of this dataset is not stable. We started collecting this data when there were no in-app purchases, privacy statement, and developer address fields. When we noticed the addition of new fields, we were able to add them to the new documents without affecting the existing ones. In the relational world, you would need to add a new column to the table and set default value for the old records. If your table is very large, this might lock the entire db.
• Native data structure support. MongoDB is classified in a class of NoSQL databases called document-oriented databases. Document Oriented databases are well suited for storing data represented in field-value pair. MongoDB natively stores data in documents that are in JSON style format. This fits nicely with our data that is generated by a custom scraper which emits JSON files.

We will elaborate more on NoSQL databases as we progress in the course.

[TheDataLeek]

Why didn't we start this course by querying against a traditional SQL database? Given that SQL databases are the industry standard, I would have thought we would have used something like PostgreSQL, or MySQL as a "first database".

I get the hype around MonogoDB and other document based databases, but wouldn't a traditional relational database to query against have been more useful for the purpose of the course? It's hard to see the benefits of a document store database unless you've used a relational database.

[kjblakemore]

The android app data appears to be essentially read-only - once it is loaded into the database, it is analyzed, but not modified. My understanding is that NoSQL databases are better suited for large volumes of read-only data than SQL databases are. One of the reasons being that strong consistency is not required, eventual consistency is sufficient for update operations. Was this a consideration in the choice of Mongo for the android app data?