search

Pilfer / ultimate-guitar-scraper

A simple scraper for Ultimate-Guitar.com's mobile API, written in Go.
81 stars 12 forks source link

`music` library produces empty `.wav` file #1

Closed qurbat closed 1 year ago

qurbat commented 1 year ago

command

ultimate-guitar-scraper wav -id 437741 -output example.wav

output for file example.wav

example.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 24 bit, stereo 44100 Hz

output for cat example.wav

RIFF,WAVEfmt D��    data
qurbat commented 1 year ago 
ultimate-guitar-scraper wav --id 1947141 --output wav.wav && file wav.wav && cat wav.wav

image

Pilfer commented 1 year ago

Hey @qurbat , sorry for the delayed response. That particular tab is in "chord" format, and not in the standard six string tab format. I should've clarified in the README that this functionality is very basic and only supports true tablature. If you're interested in writing a parser to pull out the chords and play them in order, I'd be more than happy to accept it as a PR.

lepras commented 1 year ago

Hey @qurbat , sorry for the delayed response. That particular tab is in "chord" format, and not in the standard six string tab format. I should've clarified in the README that this functionality is very basic and only supports true tablature. If you're interested in writing a parser to pull out the chords and play them in order, I'd be more than happy to accept it as a PR.

Searching the song you mentioned in the README, this comes up https://tabs.ultimate-guitar.com/tab/jeff-buckley/hallelujah-tabs-113039

so just select able tab format is pulled out.

trying others like

https://tabs.ultimate-guitar.com/tab/led-zeppelin/stairway-to-heaven-guitar-pro-223796

don't work as these are drawn in canvases.

Will look into how the data for the canvases are fetched and maybe write a parser.

Pilfer commented 1 year ago

@lepras Correct! The guitar pro tabs are in their own container format - UG supports a few different tab software types and formats.

If you dump the raw JSON from from the GetTabByIDRaw response, you'll see something like: 

{
  "artist_id": 1036,
  "artist_name": "Led Zeppelin",
  "capo": 0,
  "comments_count": 121,
  "content_urls": {
    "gp": "https://cdn.ustatik.com/storage/tab_pro/2/3/2346f2c74c6e9f4fe390a306677ed0b4.json",
    "midi": "https://cdn.ustatik.com/storage/tab_pro/2/3/2346f2c74c6e9f4fe390a306677ed0b4.mid",
    "source": "https://api.ultimate-guitar.com/api/v1/tab/download?tab_id=223796\u0026tab_access_type=public\u0026key=fbfdccac00da6b4eaf14d1e8ce05c4e9",
    "tux_guitar": "https://cdn.ustatik.com/storage/tab_pro/2/3/2346f2c74c6e9f4fe390a306677ed0b4.tg"
  },
}

If you look at the json content of the content_urls.gp file, you'll find the guitar pro project file in JSON format. The midi file is included here - if you want just the audio format for GP files, using that midi may be the best option. The v1/tab/download endpoint returns a GuitarPro5 project file.

If you feel like modifying the TabResult struct to accommodate the content_urls object, or wish to create a ProTabResult struct and do some type inference within the Scraper methods, I'd happily accept that PR! :)

lepras commented 1 year ago

How exactly do you dump that particular json? (for a particular id)

This seems to be the function, but how do I directly call it from a cli? probably has to call it from main.go? I am not really familiar with go

oh we have a .tg here, i just used tuxguitar to export to pdf, my use case is finished lmao. parsing json or midi and exporting to html seems like a tough job.

though notation software like https://musescore.org/en can export to pdf from midi

We have a cli like https://manpages.org/tuxguitar that can take tg and export pdf directly maybe we can call the cli from go itself?

lepras commented 1 year ago

I just wrote a new file in /cmd to access that function through cli, though I don't know if its best way to do that.

Pilfer commented 1 year ago

@lepras That definitely works - you can also just import the package in your own Go program and call it that way. Here's an example: 

package main

import (
    "encoding/json"
    "fmt"

    "github.com/Pilfer/ultimate-guitar-scraper/pkg/ultimateguitar"
)

func main() {
    ug := ultimateguitar.New()

    res, err := ug.GetTabByIDRaw(223796)
    if err != nil {
        panic(err)
    }

    var proTabResult ProTabResult
    err = json.Unmarshal([]byte(res), &proTabResult)
    if err != nil {
        panic(err)
    }

    fmt.Println("Gp:", proTabResult.ContentUrls.Gp)
    fmt.Println("Midi:", proTabResult.ContentUrls.Midi)
    fmt.Println("TuxGuitar:", proTabResult.ContentUrls.TuxGuitar)
    fmt.Println("Source:", proTabResult.ContentUrls.Source)
}

type ProTabResult struct {
    ID                 int     `json:"id"`
    SongID             int     `json:"song_id"`
    SongName           string  `json:"song_name"`
    ArtistID           int     `json:"artist_id"`
    ArtistName         string  `json:"artist_name"`
    Type               string  `json:"type"`
    Part               string  `json:"part"`
    Version            int     `json:"version"`
    Votes              int     `json:"votes"`
    Rating             float64 `json:"rating"`
    Date               string  `json:"date"`
    Status             string  `json:"status"`
    PresetID           int     `json:"preset_id"`
    TabAccessType      string  `json:"tab_access_type"`
    TpVersion          int     `json:"tp_version"`
    TonalityName       string  `json:"tonality_name"`
    VersionDescription string  `json:"version_description"`
    Verified           int     `json:"verified"`
    Recording          struct {
        IsAcoustic       int           `json:"is_acoustic"`
        TonalityName     string        `json:"tonality_name"`
        Performance      interface{}   `json:"performance"`
        RecordingArtists []interface{} `json:"recording_artists"`
    } `json:"recording"`
    Versions []struct {
        ID                 int         `json:"id"`
        SongID             int         `json:"song_id"`
        SongName           string      `json:"song_name"`
        ArtistID           int         `json:"artist_id"`
        ArtistName         string      `json:"artist_name"`
        Type               string      `json:"type"`
        Part               string      `json:"part"`
        Version            int         `json:"version"`
        Votes              int         `json:"votes"`
        Rating             float64     `json:"rating"`
        Date               string      `json:"date"`
        Status             string      `json:"status"`
        PresetID           int         `json:"preset_id"`
        TabAccessType      string      `json:"tab_access_type"`
        TpVersion          int         `json:"tp_version"`
        TonalityName       string      `json:"tonality_name"`
        VersionDescription interface{} `json:"version_description"`
        Verified           int         `json:"verified"`
        Recording          struct {
            IsAcoustic       int           `json:"is_acoustic"`
            TonalityName     string        `json:"tonality_name"`
            Performance      interface{}   `json:"performance"`
            RecordingArtists []interface{} `json:"recording_artists"`
        } `json:"recording"`
    } `json:"versions"`
    UserRating    int           `json:"userRating"`
    Difficulty    string        `json:"difficulty"`
    Tuning        string        `json:"tuning"`
    Capo          int           `json:"capo"`
    URLWeb        string        `json:"urlWeb"`
    Strumming     []interface{} `json:"strumming"`
    VideosCount   int           `json:"videosCount"`
    CommentsCount int           `json:"comments_count"`
    Contributor   struct {
        UserID   int    `json:"user_id"`
        Username string `json:"username"`
    } `json:"contributor"`
    ProBrother struct {
        ID                 int     `json:"id"`
        SongID             int     `json:"song_id"`
        SongName           string  `json:"song_name"`
        ArtistID           int     `json:"artist_id"`
        ArtistName         string  `json:"artist_name"`
        Type               string  `json:"type"`
        Part               string  `json:"part"`
        Version            int     `json:"version"`
        Votes              int     `json:"votes"`
        Rating             float64 `json:"rating"`
        Date               string  `json:"date"`
        Status             string  `json:"status"`
        PresetID           int     `json:"preset_id"`
        TabAccessType      string  `json:"tab_access_type"`
        TpVersion          int     `json:"tp_version"`
        TonalityName       string  `json:"tonality_name"`
        VersionDescription string  `json:"version_description"`
        Verified           int     `json:"verified"`
        Recording          struct {
            IsAcoustic       int           `json:"is_acoustic"`
            TonalityName     string        `json:"tonality_name"`
            Performance      interface{}   `json:"performance"`
            RecordingArtists []interface{} `json:"recording_artists"`
        } `json:"recording"`
    } `json:"pro_brother"`
    Recommended []struct {
        ID                 int     `json:"id"`
        SongID             int     `json:"song_id"`
        SongName           string  `json:"song_name"`
        ArtistID           int     `json:"artist_id"`
        ArtistName         string  `json:"artist_name"`
        Type               string  `json:"type"`
        Part               string  `json:"part"`
        Version            int     `json:"version"`
        Votes              int     `json:"votes"`
        Rating             float64 `json:"rating"`
        Date               string  `json:"date"`
        Status             string  `json:"status"`
        PresetID           int     `json:"preset_id"`
        TabAccessType      string  `json:"tab_access_type"`
        TpVersion          int     `json:"tp_version"`
        TonalityName       string  `json:"tonality_name"`
        VersionDescription string  `json:"version_description"`
        Verified           int     `json:"verified"`
        Recording          struct {
            IsAcoustic       int           `json:"is_acoustic"`
            TonalityName     string        `json:"tonality_name"`
            Performance      interface{}   `json:"performance"`
            RecordingArtists []interface{} `json:"recording_artists"`
        } `json:"recording"`
    } `json:"recommended"`
    ContentUrls struct {
        Source    string `json:"source"`
        Gp        string `json:"gp"`
        Midi      string `json:"midi"`
        TuxGuitar string `json:"tux_guitar"`
    } `json:"content_urls"`
    Tracking struct {
        Ctx struct {
            UserRegistered      int     `json:"user_registered"`
            URL                 string  `json:"url"`
            Referer             string  `json:"referer"`
            UtmSource           string  `json:"utm_source"`
            UtmMedium           string  `json:"utm_medium"`
            UtmCampaign         string  `json:"utm_campaign"`
            UtmContent          string  `json:"utm_content"`
            UtmTerm             string  `json:"utm_term"`
            TabType             string  `json:"tab_type"`
            TabCreated          int     `json:"tab_created"`
            TabOpenSource       string  `json:"tab_open_source"`
            TabViews            int     `json:"tab_views"`
            TabFavouritesCount  int     `json:"tab_favourites_count"`
            TabVersion          int     `json:"tab_version"`
            TabRatingsCount     int     `json:"tab_ratings_count"`
            TabRating           float64 `json:"tab_rating"`
            TabUgDifficulty     string  `json:"tab_ug_difficulty"`
            TabAuthorDifficulty string  `json:"tab_author_difficulty"`
            SongID              int     `json:"song_id"`
            SongArtistID        int     `json:"song_artist_id"`
            H                   int     `json:"h"`
            Dt                  int     `json:"dt"`
        } `json:"ctx"`
    } `json:"tracking"`
}

Which produces the output of: 

Gp: https://cdn.ustatik.com/storage/tab_pro/2/3/2346f2c74c6e9f4fe390a306677ed0b4.json
Midi: https://cdn.ustatik.com/storage/tab_pro/2/3/2346f2c74c6e9f4fe390a306677ed0b4.mid
TuxGuitar: https://cdn.ustatik.com/storage/tab_pro/2/3/2346f2c74c6e9f4fe390a306677ed0b4.tg
Source: https://api.ultimate-guitar.com/api/v1/tab/download?tab_id=223796&tab_access_type=public&key=fbfdccac00da6b4eaf14d1e8ce05c4e9

I don't use TuxGuitar, but you could pretty easily spawn a process using your Go program that calls the "export to PDF" functionality of Tux if it is exposed on the CLI. There are also a few Go html-to-pdf converters kicking around, so you can always go the template route if you feel like it. That was going to be one of my original use cases for this library when I wrote it, but life is keeping me busy :(