r3nt0n / bopscrk

Generate smart and powerful wordlists
https://pypi.org/project/bopscrk
GNU General Public License v3.0
856 stars 109 forks source link

lyrics mode broken #6

Closed noraj closed 4 years ago

noraj commented 5 years ago
$ bopscrk -a adele
[*] Looking for Adele's lyrics...
Traceback (most recent call last):
  File "bopscrk.py", line 576, in <module>
    try: main()
  File "bopscrk.py", line 486, in main
    lyfinder = LyricsFinder(artist, False, True)
  File "/usr/share/bopscrk/lib/lyricpass.py", line 35, in __init__
    for l in self.get_lyrics(s):  # get a list of lyric lines
  File "/usr/share/bopscrk/lib/lyricpass.py", line 92, in get_lyrics
    response = requests.get(songurl)                            # Now we scrape each individual song page
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 668, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 247, in resolve_redirects
    **adapter_kwargs
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='lyrics.fandom.com', port=443): Max retries exceeded with url: /wiki/Adele:Hello (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f096a267950>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

lyricspass is working totally fine so not an upstream bug:

python2 lyricpass.py adele adele.txt
Looking for lyrics from adele and writing to file: adele.txt
Getting lyrics for http://lyrics.wikia.com/wiki/adele:Daydreamer
Found 25 lines of lyrics
Getting lyrics for http://lyrics.wikia.com/wiki/adele:Best_for_Last
Found 42 lines of lyrics
Getting lyrics for http://lyrics.wikia.com/wiki/adele:Chasing_Pavements
Found 33 lines of lyrics
Getting lyrics for http://lyrics.wikia.com/wiki/adele:Cold_Shoulder
Found 32 lines of lyrics
Getting lyrics for http://lyrics.wikia.com/wiki/adele:Crazy_for_You
Found 28 lines of lyrics
Getting lyrics for http://lyrics.wikia.com/wiki/adele:Melt_My_Heart_to_Stone
Found 31 lines of lyrics
Getting lyrics for http://lyrics.wikia.com/wiki/adele:First_Love
r3nt0n commented 4 years ago

Hello @noraj,

Thanks for your contribution. I'm going to look into it, maybe the upstream output have changed.

r3nt0n commented 4 years ago

I tried to replicate the same scenario and it works for me when I run it with your args, both master and python2-legacy-support branches. Maybe it was a problem with the specific version of python/requests/beautiful-soup that you are using. Y suggest to update it and try again.

noraj commented 4 years ago

It was one year ago so I probably don't have the same version anymore.

r3nt0n commented 4 years ago

And yet didnt work for you?

noraj commented 4 years ago

@R3nt0n I'll probably test the new version in a few days when I'll re-package it from BlackArch: https://github.com/BlackArch/blackarch/issues/2789 So I'll tell you.

noraj commented 4 years ago

https://github.com/BlackArch/blackarch/pull/2793/files

And yet didnt work for you?

Works now