Welcome to Next_movie - a movie information portal developed by Nicolai Negru as the 4th (and final) milestone project at Code Institute's Full-stack web developer course.
The application can be accessed on https://koleaby4-next-movie.herokuapp.com/
The target group of this website consists of users who:
The initial project goal is to attract audience by providing information about movies.
It starts with two main categories:
Best Ever - movies, which historically have been recognised as the most successful movieNow Playing - movies which currently can be seen in the cinemas across the worldBoth lists are available to all users - registered and unregistered.
The project is looking to build a deeper relationship with the users by inviting them to register an account.
By signing up users will be able to:
Watched to hide them on Best Ever pageIn the future this will open opportunities for data analysis, advertising and referral fees.
Finally, the website is aiming to start generating an income stream.
For one-off fee of £9.99 registered users can become Prime Members, getting access to:
Best Ever moviesNow PlayingAvailable to everyone:
Best Ever movies (dynamically exclude movies marked as watched by the current user)Now Playing moviesAvailable to registered users only:
watched / not watched from movie detail pagewatched movies on (yes, you guessed it!) /watched pageAvailable to Prime Members only:
movie detail pageA full list of future tickets can be found here.
We assume that the user visiting our website comes in a special "movie-mood".
In order to preserve it, our designs had to be neutral and non-distracting.
As a result, the following conventional colours were chosen:
It was decided to use Open Sans Google font
because of its contemporary memorable style and friendly curves resulting in clear and functional visuals.
All user stories have been tracked using GitHub's issues section. GitHub Issues is a lightweight equivalent of Jira, which is widely used for planning and tracking software development activities.
Tickets grouping and filtering approach:
Compatibility labels were used to mark tickets related to browser / platform / screen size compatibilityBalsamiq was used to develop wireframes for this project during the requirements collection phase.
The tool allows to quickly create sketches of pages, amend them if need be and export.
To simplify and speed up development, the wireframes were embedded into user stories - see issue#49 and issue#50 as an example.
A full collection of the wireframes can also be seen in the wireframes folder
Next_movie website is using PostgreSQL for storing the data.
The structure of the database was emerging organically during the implementation.
That approach focuses on real business needs and prevents over-engineering by keeping You aren't gonna need it principle in mind.
Very often in the software development world, we have control over the data our application deals with.
Next_movie is a little bit different in that it relies primarily on the data fetched from a number of 3rd party APIs.
As the development of this project progressed, we've learned that there is a lot of inconsistent and missing data
in the responses we were receiving. A few examples to consider:
1975- as a movie's release yearimdb8.p.rapidapi.com APIs, but missing from api.themoviedb.orgTo allow our website cope with these data anomalies:
Structure of the custom tables and relationships among them is represented on this chart:
<img src="https://github.com/koleaby4/next_movie/blob/master/documentation/images/db_relationships.jpg?raw=true" alt="Database tables relationships" style="float: left; margin-right: 10px;" />
Movie modelThe Movie model within the movies app, is used to store information about individual movies.
Most of this data comes from https://imdb8.p.rapidapi.com/title/get-details end-point,
except of images field, which is populated with the data returned by a separate call to /get-images API.
Keeping in mind that network calls are expensive, it was decided to store full response payloads returned by get-details API in full_json_details field.
That approach allows us to retro-fit more fields to the Movie model
and populate them with real values from full_json_details field.
Review modelThe Review model within the movies app, was introduced to store information about movie reviews.
Each review record contains one single review. Reviews are are linked to respective movie via movie foreign key.
During the development we came across movies with a very large number of reviews.
Persisting them all would've been time- and space-consuming.
To address unnecessary pressure on these resources it was decided to persist top n (by default n=5) latest reviews.
If need be, that configuration can be changed in movies_collector > get_movie_reviews() function.
CustomUser modelThe CustomUser model within the users application was introduced to eliminate username field
from teh default user registration and authentication process.
CustomUser objects are referenced by Profile model in the profile app via foreign keys.
We also introduced paid_for_membership permission.
That permission is granted to the users who purchased Prime Membership
and indicates that these users should have access to premium features such as:
Profile modelThe Profile model within the profile app is used as a storage of information about the movies user marked as watched.
On the one side, reference to the users.CustomUser model is maintained via OneToOneField Django models field.
On the other side, we utilise ManyToManyField to connect to movies.Movie model.
Profile model also contains computationally-heavy watched_movies_years, watched_movies_genres and watched_movies_average_rating fields. Their values are dynamically recalculated by specialised profile.signals functions every time a movie is marked as watched / unwatched.
This approach ensures that:
profile pageInstall:
Register and obtain keys for the following services:
Next_movie repository by executing git clone https://github.com/koleaby4/next_movie.git in consolecd PATH_TO_THE_PROJECT_FOLDERpip install -r requirements.txtsecrets.json file with your database configuration and API keyspython3 manage.py migratepython3 manage.py createsuperuser and follow instruction in terminalpython3 manage.py runserverThe application should now be running on http://127.0.0.1:8000
pip install -r requirements.txt
python manage.py migrate
python manage.py createsuperuserProcfile (declaring application type and desired http server) and runtime.txt (declaring desired Python version) have already been created for you - please review themsecrets.json filegit add . & git commit -m "Configure Heroku" & git push to persist your changes in Git and trigger deploymentOpen App button to open the websiteThe application should now be up and running on https://koleaby4-next-movie.herokuapp.com
Testing was carried out in several iterations both manually and via automated tests.
At the lowest level (closest to the code) unit tests were implemented to verify:
I started testing using the testing frameworks provided by django.test.
Unit tests were placed into tests.py files of the respective application folders (for example users/tests.py)
and executed against a local instance of Postgresql.
Soon after promoting our project from a local database to the Heroku-hosted instance,
developing and running unit-tests became more challenging because Heroku does not allow dynamic creation / deletion of test databases.
To work around that security restriction, I continued developing and running tests against a local instance of Postgresql.
That worked for a while, but as the project was becoming more and more complex (calls to 3rd-party APIs, usage of signals to trigger post_save actions, etc)
unit tests had to either cover communication between components or to start employing python mocks to simulate parts of functionality.
On the one hand, unit tests are not supposed to be used for verifying integration among components.
On the other hand, I was reluctant using mocks because their usage in our project would lead to unnecessarily complicated tests and could hide bugs as the codebase continued evolving.
At that point I decided that the project was ready for introducing system tests.
System tests were introduced to verify end-to-end functionality of the whole website.
I decided to use cypress for system tests. Cypress is a very popular JavaScript-based testing framework which allows user-like interactions with websites and verifications of page content.
Cypress tests can be found in cypress\integration folder, while a number of helper functions are stored in cypress\support folder.
Tests are grouped by functionality in the following categories:
landing.spec - cover content of the landing page, which changes depending on whether the user is unauthenticated, logged in or registered as a Prime Membermovies.spec - verifies content of Best Ever and Now PLaying movie listsprofile.spec - checks content of profile page for both authenticated and Prime Membersearch.spec - tests for various search scenariosusers.spec - users creation as well as login/logout functionalityWhen system tests were implemented, it was decided to remove unit tests for the following reasons:
The only area not covered by the tests at that point was Stripe payment.
Automated verification of payment scenarios with 3rd party service in this case is complex for the following reasons:
To manage that risk, it was decided to continue verifying payments scenario manually on regular basis.
cypress.json and set baseUrl to the root of the websitenpm run cypress:openTo make sure that users of various browsers / platforms / screen sizes could successfully use our website,
a number of compatibility user stories were added to the requirements.
Screen sizes:
Browsers (based on statistics of browser usage in 2020):
Platforms:
Testing all permutations of above parameters would've been impractical,
so the following combinations were chosen to cover most common scenarios:
| Platform | Browser | Screen size |
|---|---|---|
| Windows 10 | FireFox | Desktop |
| Windows 10 | Edge | Desktop |
| iOS 14 | Safari | Tablet |
| iOS 14 | Safari | Mobile |
| Android 11 | Chrome | Tablet |
| Android 11 | Chrome | Mobile |
All defects identified during development and testing phase were noted.
When defect could be resolved faster than the overheads of formally documenting it, it was resolved in-place.
In situations when it was not possible to resolve defect on the spot, it was documented using the following format:
See issue 38 as an example.
All defects were marked by a 'bug' label
Keeping in mind that this website was developed for educational purposes, Stripe payments have been connected to test end-points.
As a result of this, only specific test card details will be accepted when paying for Prime Membership on our website
<img src="https://github.com/koleaby4/next_movie/blob/master/documentation/images/test_payment_details.png?raw=true" alt="Test Payment Details" style="max-width:250px; margin-right: 10px;" />
Having done an initial research of available free movies data sources, I came to conclusion
that none of them offered a full set of features required for our website.
As a result I decided to combine information from several data sources:
To compile together information about every movie, we have to make 3 calls to external APIs to fetch:
Keeping in mind that:
To address these considerations, movies information once retrieved, is persisted in our internal database. When searching for movie details, we would first check whether this information was previously retrieved. If so - we would use persisted data, otherwise we would fetch and store it.
<img src="https://github.com/koleaby4/next_movie/blob/master/documentation/images/data_flow.png?raw=true" alt="Data flow diagram" style="max-width:800px; margin-right: 10px;" />
In several parts of our codebase we used multithreading for fetching information in parallel, in order to keep our website's response time to minimum. However even with these performance optimisations the longest chain of calls to get all movie details is fairly complex and time-consuming.
Finally, to increase performance of the website on Best Ever and Now Playing pages, we introduced caching mechanisms. The first time data is retrieved from 3rd party APIs, the results are stored in top_rated_cache.json and now_playing_cache.json respectively. Every time users request these lists, the results will be fetched from the cache files. The the application then checks whether the cache files are older than 24h and if so - will spawn a separate thread to fetch the latest data and update the cache. In the meantime cached results will be returned. In this manner the system is not waiting for the latest data to be fetched, but makes sure that cache is updated and the next request will contain most up-to-date results.