mwclient / mwclient

Python client library to interface with the MediaWiki API
https://pypi.org/project/mwclient/
MIT License
319 stars 91 forks source link

Doesn't login on private wiki #278

Closed robkam closed 1 year ago

robkam commented 1 year ago

Using Git Bash on Windows 10 with Python 3.11.1 from Microsoft Store.

$ pip install git+https://github.com/mwclient/mwclient
Collecting git+https://github.com/mwclient/mwclient
  Cloning https://github.com/mwclient/mwclient to c:\users\rob\appdata\local\temp\pip-req-build-tv63nhmh
  Running command git clone --filter=blob:none --quiet https://github.com/mwclient/mwclient 'C:\Users\rob\AppData\Local\Temp\pip-req-build-tv63nhmh'
  Resolved https://github.com/mwclient/mwclient to commit 57e9e2934c4b48dc16a019bef901ea8c37808bda
  Preparing metadata (setup.py) ... done
Requirement already satisfied: requests-oauthlib in c:\users\rob\appdata[...]local-packages\python311\site-packages (from mwclient==0.10.1) (1.3.1)
Requirement already satisfied: six in c:\users\rob\appdata[...]local-packages\python311\site-packages (from mwclient==0.10.1) (1.16.0)
Requirement already satisfied: oauthlib>=3.0.0 in c:\users\rob\appdata[...]local-packages\python311\site-packages (from requests-oauthlib->mwclient==0.10.1) (3.2.2)
Requirement already satisfied: requests>=2.0.0 in c:\users\rob\appdata[...]local-packages\python311\site-packages (from requests-oauthlib->mwclient==0.10.1) (2.28.1)
Requirement already satisfied: charset-normalizer<3,>=2 in c:\users\rob\appdata[...]local-packages\python311\site-packages (from requests>=2.0.0->requests-oauthlib->mwclient==0.10.1) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in c:\users\rob\appdata[...]local-packages\python311\site-packages (from requests>=2.0.0->requests-oauthlib->mwclient==0.10.1) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\rob\appdata[...]local-packages\python311\site-packages (from requests>=2.0.0->requests-oauthlib->mwclient==0.10.1) (1.26.13)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\rob\appdata[...]local-packages\python311\site-packages (from requests>=2.0.0->requests-oauthlib->mwclient==0.10.1) (2022.12.7)

$ dumpgenerator --user USERNAME --pass PASSWORD --cookies miraheze.org_cookies.txt --delay 0.0 --failfast --xml --xmlrevisions --images --api=https://scruffy.miraheze.org/w/api.php
Using cookies from miraheze.org_cookies.txt
Checking API... https://scruffy.miraheze.org/w/api.php
API is OK: https://scruffy.miraheze.org/w/api.php
Checking index.php... https://scruffy.miraheze.org/w/index.php
index.php is OK
No --path argument provided. Defaulting to:
  [working_directory]/[domain_prefix]-[date]-wikidump
Which expands to:
  ./scruffymirahezeorg_w-20221221-wikidump
#########################################################################
# Welcome to DumpGenerator 0.4.0-alpha by WikiTeam (GPL v3)             #
# More info at: https://github.com/elsiehupp/wikiteam3                  #
#########################################################################

#########################################################################
# Copyright (C) 2011-2022 WikiTeam developers                           #

# This program is free software: you can redistribute it and/or modify  #
# it under the terms of the GNU General Public License as published by  #
# the Free Software Foundation, either version 3 of the License, or     #
# (at your option) any later version.                                   #
#                                                                       #
# This program is distributed in the hope that it will be useful,       #
# but WITHOUT ANY WARRANTY; without even the implied warranty of        #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the         #
# GNU General Public License for more details.                          #
#                                                                       #
# You should have received a copy of the GNU General Public License     #
# along with this program.  If not, see <http://www.gnu.org/licenses/>. #
#########################################################################

Analysing https://scruffy.miraheze.org/w/api.php
Trying generating a new dump into a new directory...
Loading page titles from namespaces = all
Excluding titles from namespaces = None
28 namespaces found
    Retrieving titles in the namespace 0
Traceback (most recent call last):
  File "C:\Users\Rob\AppData\[...]\mwclient\listing.py", line 55, in __next__
    item = six.next(self._iter)
           ^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Rob\AppData\[...]\Scripts\dumpgenerator.exe\__main__.py", line 7, in <module>
  File "C:\Users\Rob\AppData\[...]\wikiteam3\dumpgenerator\__init__.py", line 26, in main
    DumpGenerator()
  File "C:\Users\Rob\AppData\[...]\wikiteam3\dumpgenerator\generator.py", line 87, in __init__
    DumpGenerator.createNewDump(config=config, other=other)
  File "C:\Users\Rob\AppData\[...]\wikiteam3\dumpgenerator\generator.py", line 98, in createNewDump
    getPageTitles(config=config, session=other["session"])
  File "C:\Users\Rob\AppData\[...]\wikiteam3\dumpgenerator\page_titles.py", line 189, in getPageTitles
    for title in titles:
  File "C:\Users\Rob\AppData\[...]\wikiteam3\dumpgenerator\page_titles.py", line 27, in getPageTitlesAPI
    for page in site.allpages(namespace=namespace):
  File "C:\Users\Rob\AppData\[...]\mwclient\listing.py", line 180, in __next__
    info = super(GeneratorList, self).__next__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Rob\AppData\[...]\mwclient\listing.py", line 61, in __next__
    self.load_chunk()
  File "C:\Users\Rob\AppData\[...]\mwclient\listing.py", line 191, in load_chunk
    return super(GeneratorList, self).load_chunk()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Rob\AppData\[...]\mwclient\listing.py", line 95, in load_chunk
    data = self.site.get(
           ^^^^^^^^^^^^^^
  File "C:\Users\Rob\AppData\[...]\mwclient\client.py", line 234, in get
    return self.api(action, 'GET', *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Rob\AppData\[...]\mwclient\client.py", line 288, in api
    if self.handle_api_result(info, sleeper=sleeper):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Rob\AppData\[...]\mwclient\client.py", line 331, in handle_api_result
    raise errors.APIError(info['error']['code'],
mwclient.errors.APIError: ('readapidenied', 'You need read permission to use this module.', 'See https://scruffy.miraheze.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/&gt; for notice of API deprecations and breaking changes.')
yzqzss commented 1 year ago

wikiteam3(mediawiki-scraper) uses its own requests.session and passes the session into the mwclient, so wikiteam3 needs to make its own logged-in session. This is not an upstream (mwclient) issue.


https://github.com/mediawiki-client-tools/mediawiki-scraper/pull/124 https://github.com/mediawiki-client-tools/mediawiki-scraper/pull/128

Already fixed, please close this issue.