shadowmoose / RedditDownloader

Scrapes Reddit to download media of your choice.
1.11k stars 99 forks source link

Metadata-generation-failed #291

Open Opening-Button-8988 opened 1 year ago

Opening-Button-8988 commented 1 year ago

Hello,

Is this project still alive and functional, and will it still work after the API changes?

Although I read the instructions I'm left with an error. The last update to this project was 2 years ago but I've seen some recent issues and the maintainer has responded in a way that implies to me that it's still working. And according to the readme.md "RMD is currently under a large-scale rewrite in a new language", which links to a discussion from 3 years ago. I don't have time to read the details, I'm guessing that's well and truly over. Please update your project readme or at least archive it if it's not working anymore.

If this project no longer works; please link me to a tool that can achieve what this tool does even after the API changes.

After following these instructions I'm left with the following error (with some paranoid redactions):

Defaulting to user installation because normal site-packages is not writeable
Collecting psaw~=0.0.12 (from -r requirements.txt (line 1))
  Using cached psaw-0.0.12-py3-none-any.whl (15 kB)
Collecting youtube_dl (from -r requirements.txt (line 2))
  Using cached youtube_dl-2021.12.17-py2.py3-none-any.whl (1.9 MB)
Collecting imgurpython==1.1.7 (from -r requirements.txt (line 3))
  Using cached imgurpython-1.1.7.tar.gz (8.4 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: requests>=2.22.0 in /usr/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (2.28.2)
Collecting sqlalchemy==1.3.23 (from -r requirements.txt (line 5))
  Using cached SQLAlchemy-1.3.23.tar.gz (6.3 MB)
  Preparing metadata (setup.py) ... done
Collecting praw==7.2.0 (from -r requirements.txt (line 6))
  Using cached praw-7.2.0-py3-none-any.whl (159 kB)
Collecting lxml==4.6.3 (from -r requirements.txt (line 7))
  Using cached lxml-4.6.3.tar.gz (3.2 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      /tmp/pip-install-redacted/lxml_redacted/setup.py:64: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
        import pkg_resources
      Building lxml version 4.6.3.
      Building without Cython.
      Error: Please make sure the libxml2 and libxslt development packages are installed.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
ghost commented 1 year ago

The typescript version may or may not be stable at any moment. I'd suggest clone the Python repository (aka the default branch) and build it yourself. It has some neat features that the latest release does not cover. Just don't forget to add the "--skip_update" flag when launching it.

Reddit said Oauth apps are able to get 1800~ requests per minute. This project might be modified so it makes the user create their own Oauth application and use it to authenticate. This may slow things down a bit but i think it's still generous as it always took long to download content.

The Typescript version is on hold until the PushShift API becomes available again, as the developer has talked about in #285 since the Reddit API is unreliable and allows only up to 1000 contents per subreddit.

Currently the regular application is unusable due to not being able to Authenticate with error "invalid client id" it seems. I looked deep into similar apps that can do the job this application did, but there is no good replacement i could find so far.

Opening-Button-8988 commented 1 year ago

@rktfier thanks for your response, I ended up using gallery-dl.