[API Integration - AUDIO] Jamendo

amartya-dev commented 4 years ago

Provider API Endpoint / Documentation

Documentation: https://developer.jamendo.com/v3.0/docs The generic GET url form is the following: http[s]://api.jamendo.com/version/entity/subentity/?api_parameter=value

Provider description

On Jamendo Music, you can enjoy a wide catalog of more than 500,000 tracks shared by 40,000 artists from over 150 countries all over the world. You can stream all the music for free, download it and support the artist: become a music explorer and be a part of a great discovery experience!

Licenses Provided

Jamendo uses Creative Commons licenses to enable the free distribution of otherwise copyrighted work. CC licenses all grant 'baseline rights', such as the right to distribute the copyrighted work worldwide for non-commercial purposes, and without modification. Artists choose a license according to the conditions they want to be applied to the song. As per https://www.jamendo.com/legal/creative-commons

Provider API Technical info

The Terms of use does not as such mention a limit, there are different plans though which will be available only after signup. The plans are read-only and write-only. The API uses OAuth2 based authentication. Rate Limit: 35,000 requests per month for non-commercial apps

Checklist to complete before beginning development

No development should be done on a Provider API Script until the following info is gathered:

[x] Verify there is a way to retrieve the entire relevant portion of the provider's collection in a systematic way via their API. There is a read API that can be queried with: https://developer.jamendo.com/v3.0 as the base URL and other information can be sent along with the GET request to retrieve specific info. The API responds with 200 rows in one response and accepts a parameter offset that can be used to specify the position to start returning results from.
[x] Verify the API provides license info (license type and version; license URL provides both, and is preferred) It provides the license URL as the parameter licensecurl in the API response.
[x] Verify the API provides stable direct links to individual works. It provides the same under the key "audio" in the response. The download link is also provided under "audiodownload"
[x] Verify the API provides a stable landing page URL to individual works. The API provides a short and a share version for the landing page of the URL under the keys "shorturl" and "shareurl" respectively.
[x] Note other info the API provides, such as thumbnails, dimensions, attribution info (required if non-CC0 licenses will be kept), title, description, other meta data, tags, etc. The API provides duration, album name, album image, artist name, release date, image for the track, etc.
[x] Attach example responses to API queries that have the relevant info.

{
  "headers":{
    "status":"success",
    "code":0,
    "error_message":"",
    "warnings":"",
    "results_count":2
  },
  "results":[
    {
      "id":"1630628",
      "name":"Rock_guard",
      "duration":202,
      "artist_id":"493111",
      "artist_name":"Christian Petermann",
      "artist_idstr":"Christian_Petermann",
      "album_name":"Rock Vision",
      "album_id":"183496",
      "license_ccurl":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/3.0\/",
      "position":10,
      "releasedate":"2019-03-14",
      "album_image":"https:\/\/imgjam2.jamendo.com\/albums\/s183\/183496\/covers\/1.200.jpg",
      "audio":"https:\/\/mp3l.jamendo.com\/?trackid=1630628&format=mp31&from=app-devsite",
      "audiodownload":"https:\/\/mp3d.jamendo.com\/download\/track\/1630628\/mp32\/",
      "prourl":"https:\/\/licensing.jamendo.com\/track\/1630628",
      "shorturl":"https:\/\/jamen.do\/t\/1630628",
      "shareurl":"https:\/\/www.jamendo.com\/track\/1630628",
      "image":"https:\/\/imgjam2.jamendo.com\/albums\/s183\/183496\/covers\/1.200.jpg",
      "musicinfo":{
        "vocalinstrumental":"instrumental",
        "lang":"",
        "gender":"",
        "acousticelectric":"",
        "speed":"high",
        "tags":{
          "genres":[
            "rock"
          ],
          "instruments":[
            "electricguitar",
            "strings"
          ],
          "vartags":[
            "groovy",
            "energetic"
          ]
        }
      }
    },
    {
      "id":"1425156",
      "name":"Skitz",
      "duration":102,
      "artist_id":"497621",
      "artist_name":"Chris Bleau",
      "artist_idstr":"Chris_Bleau",
      "album_name":"San Diego State of Mind",
      "album_id":"166193",
      "license_ccurl":"http:\/\/creativecommons.org\/licenses\/by-nd\/3.0\/",
      "position":3,
      "releasedate":"2017-02-22",
      "album_image":"https:\/\/imgjam1.jamendo.com\/albums\/s166\/166193\/covers\/1.200.jpg",
      "audio":"https:\/\/mp3l.jamendo.com\/?trackid=1425156&format=mp31&from=app-devsite",
      "audiodownload":"https:\/\/mp3d.jamendo.com\/download\/track\/1425156\/mp32\/",
      "prourl":"https:\/\/licensing.jamendo.com\/track\/1425156",
      "shorturl":"https:\/\/jamen.do\/t\/1425156",
      "shareurl":"https:\/\/www.jamendo.com\/track\/1425156",
      "image":"https:\/\/imgjam1.jamendo.com\/albums\/s166\/166193\/covers\/1.200.jpg",
      "musicinfo":{
        "vocalinstrumental":"instrumental",
        "lang":"",
        "gender":"",
        "acousticelectric":"",
        "speed":"high",
        "tags":{
          "genres":[
            "rock"
          ],
          "instruments":[

          ],
          "vartags":[
            "fun",
            "groovy",
            "party"
          ]
        }
      }
    }
  ]
}

General Recommendations for implementation

The script should be in the src/cc_catalog_airflow/dags/provider_api_scripts/ directory.
The script should have a test suite in the same directory.
The script must use the ImageStore class (Import this from src/cc_catalog_airflow/dags/provider_api_scripts/common/storage/image.py).
The script should use the DelayedRequester class (Import this from src/cc_catalog_airflow/dags/provider_api_scripts/common/requester.py).
The script must not use anything from src/cc_catalog_airflow/dags/provider_api_scripts/modules/etlMods.py, since that module is deprecated.
If the provider API has can be queried by 'upload date' or something similar, the script should take a --date parameter when run as a script, giving the date for which we should collect images. The form should be YYYY-MM-DD (so, the script can be run via python my_favorite_provider.py --date 2018-01-01).
The script must provide a main function that takes the same parameters as from the CLI. In our example from above, we'd then have a main function my_favorite_provider.main(date). The main should do the same thing calling from the CLI would do.
The script must conform to PEP8. Please use pycodestyle (available via pip install pycodestyle) to check for compliance.
The script should use small, testable functions.
The test suite for the script may break PEP8 rules regarding long lines where appropriate (e.g., long strings for testing).

Examples of other Provider API Scripts

For example Provider API Scripts and accompanying test suites, please see

src/cc_catalog_airflow/dags/provider_api_scripts/flickr.py and
src/cc_catalog_airflow/dags/provider_api_scripts/test_flickr.py, or
src/cc_catalog_airflow/dags/provider_api_scripts/wikimedia_commons.py and
src/cc_catalog_airflow/dags/provider_api_scripts/test_wikimedia_commons.py.

mathemancer commented 4 years ago

Hey, for any boxes checked on the required info above, please provide references or evidence to support checking those boxes. This will help us when this issue is ready for development.

amartya-dev commented 4 years ago

@mathemancer I have updated the comment in order to include the required info

mathemancer commented 4 years ago

Okay, thanks!

cc-archive / cccatalog