gregorio-project / GregoBase

http://gregobase.selapa.net/
56 stars 12 forks source link

All gabc files under scores/gabc/ ? #11

Open martindonat opened 7 years ago

martindonat commented 7 years ago

For a bigger chant book-project I would need a lot of gabc scores from Gregobase, so I don't want to download them manually. I know, it shouldn't be such a problem: Download this repo, set up a local MySQL server, run the correct php-script, etc

After some considerable amount of time and effort I have a local MariaDB server, I imported the scores, but now I have encoding problems and am fed up... Sorry, I'm just not so familiar with SQL servers, php and this stuff. So I wanted to ask for this little favour, whether you could provide all gabc files in a dir like scores/gabc/.

I guess for you it would be much easier than for me.

Thanks in advance.

igneus commented 6 years ago

Do you still need the scores?

martindonat commented 6 years ago

No, thanks. I did it the quick and dirty way: A python script which downloads file per file from the website. It looks like this:


def extract_filename(input_str, num):

    filename_pos = input_str.find("filename=")

    # "filename=" nicht gefunden
    if filename_pos == -1:

        print("Filename not found: {0}".format(num))
        return None

    # "filename=" gefunden  
    else:

        # vorne abschneiden
        filename = input_str[filename_pos + 9:]

        # evtl hinten abschneiden
        if filename.find(";") != -1:

            filename = filename[:filename.find(";")]

        return filename

for i in range(1148,9000):

    print("Download file ... {0}".format(i))

    # Request to URL
    gabc_download = urllib.request.urlopen("https://gregobase.selapa.net/download.php?id={0}&format=gabc".format(i))

    # Get Info Header
    info_str = gabc_download.info()['Content-Disposition']

    # Check if anything is found
    if info_str:

        #Extract filename
        filename = extract_filename(info_str, i)

        # Check if filename was found
        if filename:

            # Write the file
            with open("SCORES2/" + filename, "w") as f:

                f.write(gabc_download.read().decode("utf-8"))

So maybe it would be nice for other people to have the data in a more convenient way, but I have the data I needed.

igneus commented 6 years ago

OK. Should anyone in future need the data, I can provide them. And if I ever finish my toy project of an alternative read-only frontend over the GregoBase database, mass export of the scores will be part of it.

jjbauer15 commented 11 months ago

@igneus Is there any update on this? Conversely, would read-only credentials to the database itself be on the table? Thank you!

igneus commented 11 months ago

@jjbauer15 If you are looking for a GregoBase export, last December another guy asked me and I generated this http://yakub.cz/gregobase_export/ - gabc of all scores (based on the GregoBase db dump available back then), grouped by source and genre. Hope it helps.

(Should the code used for this be of interest, it's here https://github.com/igneus/adiutor/commit/2895ed61f4f24c4ed3fd028a5deac3cc7c11c707 )

On the gregoplus app I referenced in my earlier comment there is no update. It hasn't been and probably won't ever be launched, I have other priorities.

čt 16. 11. 2023 v 22:21 odesílatel jjbauer15 @.***> napsal:

@igneus https://github.com/igneus Is there any update on this? Conversely, would read-only credentials to the database itself be on the table? Thank you!

— Reply to this email directly, view it on GitHub https://github.com/gregorio-project/GregoBase/issues/11#issuecomment-1815333866, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAM63JSPP7KUKDN3Z2HE3DYEZ7WFAVCNFSM4DE3E6F2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRGUZTGMZYGY3A . You are receiving this because you were mentioned.Message ID: @.***>