SongScope

Introduction

SongScope is an AI powered music recommendation system that experiments with your spotify data such as your top tracks, liked songs , artists , etc. to recommend you new songs that you might like. The system can add a personal touch to the recommendations if you provide your mood and personal emotional data. The system can be also fine-tuned based on your liked songs and spotify recommendation system to experiment with fine-tuned models too. Other features include a function to search reviews from pitchfork or similar websites to get a better understanding of the song and artist and have a critical view. The features combined with personalized fine-tuning can experiment with a newly personalized recommendation system.

Pre-Installation Requirements

• Python 3.10

Python 3.10 is used as interpreter for the project.

• Pip

Needed to install the dependencies.

• Spotify Account

A Spotify account is needed to use the system. You can create one here.

• Spotify Developer Account

This project is built using locally, so you need to provide your own Spotify Client ID and Client Secret. Please read this documentation to create your own Spotify Developer Account and get your own Client ID and Client Secret. Getting Started with Spotify Web API

• set up a new account and login to the Spotify Developer Dashboard.

  • Create a new app and get your Client ID and Client Secret.
  • Add the redirect URI as http://localhost:5000/callback in the app settings.

• OPENAI API Key

This project uses OpenAI API to generate song lyrics. You can get your own API key by signing up here.

We will also need OPENAI ASSISTANT ID which has functions available to it. We will do this later in the project setup.

• MongoDB, ChromaDB and Nginx

We will be using MongoDB as our database, ChromaDB for storing reviews and chat history and Nginx for our web server configurations.

The details will follow in the installation section. However, install these before starting with the project setup.

Project Setup - Debian GNU/Linux 12 (bookworm)

1. Clone the repository

git clone https://github.com/parhamrahmani/SongScope.git

2. Install the dependencies

pip install -r requirements.txt

or

pip install requests python-dotenv Flask jsonify openai chromadb pymongo chainlit beautifulsoup4

3. Set up Nginx configurations

• Install Nginx

  sudo apt update
  sudo apt install nginx

• Create a new configuration file for the project in the /etc/nginx/sites-available directory.

  sudo nano /etc/nginx/sites-available/songscope

• Add the following configurations to the file.

  server {
  listen 80;
  
  server_name localhost;
  
  location / {
      proxy_pass http://127.0.0.1:5000;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
  }
  
  location /chat/ {
      proxy_pass http://127.0.0.1:8000/;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
  }
  }

• Create a symbolic link to the sites-enabled directory.

  sudo ln -s /etc/nginx/sites-available/songscope /etc/nginx/sites-enabled

• Restart the Nginx service.

  sudo systemctl restart nginx

• Check the status of the Nginx service.

  sudo systemctl status nginx

  4. Set up MongoDB

• Install MongoDB

  sudo apt update
  sudo apt install -y mongodb

• Start the MongoDB service

  sudo systemctl start mongod

• Check the status of the MongoDB service

  sudo systemctl status mongod

• Go to the conf/mongodb directory and run the mongodb_initial_setup.py to set up the MongoDB databases.
• After running the script, the databases will be set up and the collections will be populated with example data.
• Delete the example data from the collections, since it will be populated with your own data later.
• list of collections in the spotifydb database:
  • albums
    • the albums that the user has at least listened one song from. It originates from the liked songs of the user.
  • artists
    • the artists that the user has at least listened one song from. It originates from the liked songs of the user.
  • liked_songs
    • the songs that the user has liked and saved in their library.
  • recommendations
    • the songs that the Spotify recommendation system has recommended to the user.
    • it originates from when the /recommendations and /generate_random_recommendations/<int:num_recommendations>are called, and it will be saved in the database.

5. Set up ChromaDB

• Install ChromaDB

  pip install chromadb

• You don't need to set up the ChromaDB, it will be set up automatically when you run the chatbot in the first successful run. The chatbot will be used to store the chat history and reviews.

  6. Populate the MongoDB databases with your spotify data

• Run this endpoint to populate the mongodb collections with your spotify data.

  http://localhost:5000/liked_songs

• The data will be automatically inserted into the collections.
• experiment with the Spotify recommendation system using the localhost:5000/home after authorization to tweak and get new recommendations.
• experiment with the Spotify recommendation system using the localhost:5000/generate_random_recommendations/<int:num_recommendations> to get random recommendations to populate the table for using it for fine-tuning.
• pay attention to the num_recommendations parameter to get the desired number of recommendations. There are rate limits for the Spotify API.

7. Fine-Tuning (Optional)

Fine-tuning is an optional step

• You can fine-tune the recommendation system by using the recommendations table in the database.
• The more data you have in the recommendations table, the better the fine-tuning will be.
• Fine-tuning costs money and time and wouldn't necessarily improve the recommendation system. It's an experimental feature. That's why it's optional.
• Firstly run prompt.py to generate prompts based on the recommendations table. each prompt will be each iteration of a recommendation (based on its id).
• Then run the conf/finetuning/finetuning.py to fine-tune the model based on the prompts generated.
  • this will upload conf/finetuning/prompts.jsonl to the OpenAI API and will fine-tune the model based on the prompts.
• You can do this in OPENAI API Dashboard in their website as well if you want to do this with a gui environment

  8. Set up OpenAI Assistant

• Go to the OpenAI API Dashboard and create a new assistant or use the script in the project to create a new assistant.
• Get the assistant ID and set it as the OPEN_AI_ASSISTANT_ID in the .env file.
• Make sure to include the functions that you need in the assistant. Otherwise, the assistant won't work properly.
• You have to get the assistant ID from the OpenAI API Dashboard regardless of how you create the assistant.

  9. Get Tavily API Key

• Go to the Tavily API website and get your own API key.
• Set the API key as the TAVILY_KEY in the .env file.

  10. Set up the .env file

• Create a new .env file in the root directory of the project.
• Add the following environment variables to the .env file.

  SPOTIFY_CLIENT_ID=<YOUR_SPOTIFY_CLIENT_ID>
  SPOTIFY_CLIENT_SECRET=<YOUR_SPOTIFY_CLIENT_SECRET>
  REDIRECT_URI="http://localhost:5000/callback"
  FLASK_SECRET_KEY=<YOUR_SECRET_KEY> # You can set a secret as anything you want
  OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>
  CHROMADB_PATH=<PATH_TO_CHROMADB> # A path to store the chat history and reviews
  MONGO_URI="mongodb://localhost:27017/spotifydb"
  OPEN_AI_ASSISTANT_ID=<YOUR_OPENAI_ASSISTANT_ID>
  AI_MODEL=<YOUR_OPENAI_MODEL> # The model that you want to use for the AI like gpt4o etc.
  TAVILY_KEY=<YOUR_TAVILY_KEY>

  Additional Information

• This setup is done on a linux machine. You can set up the project on a Windows machine as well. For windows, you can use WSL or easily go to each requirement documentation and follow the instructions.

Running the project

Start Nginx Service

• start

  sudo systemctl start nginx

• check the status of nginx

  sudo systemctl status nginx

  Start the MongoDB Service

• start

  sudo systemctl start mongod

• check the status of mongodb

  sudo systemctl status mongod

  Start the Application

• Go to the root directory of the project and run the following command.

  python main.py

• you should see the following output

  2024-07-01 02:27:48 - Loaded .env file
  2024-07-01 02:27:48 - Anonymized telemetry enabled. See                     https://docs.trychroma.com/telemetry for more information.
  * Serving Flask app 'backend'
  * Debug mode: off
  2024-07-01 02:27:48 - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
  * Running on http://127.0.0.1:5000
  2024-07-01 02:27:48 - Press CTRL+C to quit

• The application will be running on http://localhost:5000
• After authorization, you should be redirected to localhost:8000 to chat with the chatbot.
• At this point a new chainlit server will be started you should expect the following output

  Starting Chainlit server
  2024-07-01 02:29:13 - Loaded .env file
  2024-07-01 02:29:14 - Your app is available at http://localhost:8000
  Chainlit thread started.
  2024-07-01 02:29:15 - 127.0.0.1 - - [01/Jul/2024 02:29:15] "GET /interface HTTP/1.1" 302 -
  2024-07-01 02:29:16 - Translated markdown file for en-US not found. Defaulting to chainlit.md.
  2024-07-01 02:29:18 - HTTP Request: POST https://api.openai.com/v1/threads "HTTP/1.1 200 OK"

  This means that the chatbot is running.

• write something in the chatbot, and you should see the response from the chatbot. If the response is shown in the chatbot, the chatbot is working properly.
• expect the chatbot to be slow and have patience.
• You should expect this message in terminal when the chatbot is working properly. (This is a sample)

  Entering chat function
  Thread ID:  <the thread id of openai thread>
  Message:  hi
  2024-07-01 02:32:03 - HTTP Request: POST https://api.openai.com/v1/threads/<thread_id>/messages "HTTP/1.1 200 OK"
  2024-07-01 02:32:04 - HTTP Request: POST https://api.openai.com/v1/threads/<thread_id>/runs "HTTP/1.1 200 OK"
  Run status:  queued
  2024-07-01 02:32:05 - HTTP Request: GET https://api.openai.com/v1/threads/<thread_id>/runs/run_9a7tEs4Wo5LSUuAjugVfNYrv "HTTP/1.1 200 OK"
  Run status:  in_progress
  2024-07-01 02:32:06 - HTTP Request: GET https://api.openai.com/v1/threads/<thread_id>/runs/run_9a7tEs4Wo5LSUuAjugVfNYrv "HTTP/1.1 200 OK"
  2024-07-01 02:32:06 - HTTP Request: GET https://api.openai.com/v1/threads/<thread_id>/messages "HTTP/1.1 200 OK"
  2024-07-01 02:32:07 - HTTP Request: POST https://api.openai.com/v1/threads/<thread_id>/messages "HTTP/1.1 200 OK"
  2024-07-01 02:32:07 - Collection chat_history is not created.
  Message in thread <thread_id> inserted successfully to chat history
  Verified content:  Hello! How can I assist you today?
  2024-07-01 02:32:07 - 127.0.0.1 - - [01/Jul/2024 02:32:07] "POST /chat HTTP/1.1" 200 -