After upgrade from v4.13.1 to v9.2.1 authentication to local instance is broken

[MikaMaracuja]

We have a local instance of wikibase and have upgraded wikibase-cli to the latest version V9.2.1. Since then, neigher Oauth token authentication nor password authentication works for the wikibase-cli edit commands.

for example:

~/.npm/wikidata-cli/4.13.1/package/bin> ./wd add-claim Q4227 P20 Q49
{ value: '{"entity-type":"item","numeric-id":49}',
  entity: 'Q4227',
  property: 'P20',
  snaktype: 'value',
  assert: 'user' }
{ pageinfo: { lastrevid: 47610 },
  success: 1,
  claim:
   { mainsnak:
      { snaktype: 'value',
        property: 'P20',
        datavalue: [Object],
        datatype: 'wikibase-item' },
     type: 'statement',
     id: 'Q4227$4418D1B4-548E-4990-9B28-E4434E50FA3F',
     rank: 'normal' } }

but <path>/.nvm/versions/node/v8.2.1/lib/node_modules/wikibase-cli/bin> wb add-claim Q4227 P20 Q49 This operation requires to set your crendentials for http://<hostname>:<portnumber> Which authentication mechanism would you like to use to login to http://<hostname>:<portnumber> ?: /!\ Beware that those will be stored in plain text on your computer 1 - OAuth tokens (recommended) 2 - username - password

where method 1 results in "Nonce already used error" and method 2 results in "invalid csrf token"

additional side note: our wikimedia installation does not follow the url schema https?://{lang}.{wiki}.{tld}/w/api.php, but http://{hostname}:{port}/api.php

Details on Method 1

First question:

For the Oauth registration, do you change the wpname for each myusername?

http://<hostname>:<portnumber>/wiki/Special:OAuthConsumerRegistration/propose?wpname=wikibase-cli-myusername&wpdescription=tokens%20for%20wikibase-cli%20%28https%3A%2F%2Fgithub.com%2Fmaxlath%2Fwikibase-cli%29&wpownerOnly=1

e.g. wikibase-cli-maxlath or do you keep the value: wikibase-cli-myusername?

debug output

hello request :http://<hostname>:<portnumber>/api.php?action=query&meta=tokens&format=json
parsedURL = {"protocol":"http:","slashes":true,"auth":null,"host":"<hostname>:<portnumber>","port":"<portnumber>","hostname":"<hostname>","hash":null,"search":"?action=query&meta=tokens&format=json","query":"action=query&meta=tokens&format=json","pathname":"/api.php","path":"/api.php?action=query&meta=tokens&format=json","href":"http://<hostname>:<portnumber>/api.php?action=query&meta=tokens&format=json"}
http://<hostname>:<portnumber>/api.php error response: The authorization headers in your request are not valid: Nonce already used: a0HHA7ohKqFvoo9bRwAEvV7SONpE77LK

I have used a python testscript mwtest.py for the Oauth method and spooled the debug information:

import logging
import time
import requests
from requests_oauthlib import OAuth1

logging.basicConfig(filename='mwtest.log',level=logging.DEBUG)

auth1 = OAuth1('0b6.....b0',
               '150....6',
               '0ea3......a6',
               'ca867.....92')
for i in range(100):
    response = requests.get(
        "http://<hostname>:<portnumber>/api.php",
        params={
            'action': "query",
            'meta': "userinfo",
            'format': "json"
        },
        auth=auth1
    )
    if 'error' in response.json():
        print response.json()
    time.sleep(3)

this results in

DEBUG:oauthlib.oauth1.rfc5849:Normalized params: action=query&format=json&meta=userinfo&oauth_consumer_key=0b6d....1adfb0&oauth_nonce=140726467634263487431585910400&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1585910400&oauth_token=0ea37.....7ff64a6&oauth_version=1.0
DEBUG:oauthlib.oauth1.rfc5849:Normalized URI: http://<hostname>:<portnumber>/api.php
DEBUG:oauthlib.oauth1.rfc5849:Signing: signature base string: GET&http%3A%2F%2F<hostname>%3A<portnumber>%2Fapi.php&action%3Dquery%26format%3Djson%26meta%3Duserinfo%26oauth_consumer_key%3D0b6d.......931adfb0%26oauth_nonce%3D140726467634263487431585910400%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1585910400%26oauth_token%3D0ea......64a6%26oauth_version%3D1.0
DEBUG:oauthlib.oauth1.rfc5849:Signature: 7....H++Xon....EE=
DEBUG:oauthlib.oauth1.rfc5849:Encoding URI, headers and body to utf-8.
DEBUG:requests_oauthlib.oauth1_auth:Updated url: http://<hostname>:<portnumber>/api.php?action=query&meta=userinfo&format=json
DEBUG:requests_oauthlib.oauth1_auth:Updated headers: {'Accept': '*/*', 'Connection': 'keep-alive', 'Accept-Encoding': 'gzip, deflate', 'Authorization': 'OAuth oauth_nonce="140726467634263487431585910400", oauth_timestamp="1585910400", oauth_version="1.0", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="0b6.......fb0", oauth_token="0ea......4a6", oauth_signature="74....H%2B%2BXo....YBEE%3D"', 'User-Agent': 'python-requests/2.9.1'}
DEBUG:requests_oauthlib.oauth1_auth:Updated body: None
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): <hostname>
DEBUG:requests.packages.urllib3.connectionpool:"GET /api.php?action=query&meta=userinfo&format=json HTTP/1.1" 200 217

Details on Method 2

I have added some debug output, as far as I could understand and change the code:

debug output

hello request :http://<hostname>:<portnumber>/api.php?action=login&format=json
parsedURL = {"protocol":"http:","slashes":true,"auth":null,"host":"<hostname>:<portnumber>","port":"<portnumber>","hostname":"<hostname>","hash":null,"search":"?action=login&format=json","query":"action=login&format=json","pathname":"/api.php","path":"/api.php?action=login&format=json","href":"<hostname>/api.php?action=login&format=json"}
hello request :<hostname>/api.php?action=login&format=json
parsedURL = {"protocol":"http:","slashes":true,"auth":null,"host":"<hostname>:<portnumber>","port":"<portnumber>","hostname":"<hostname>","hash":null,"search":"?action=login&format=json","query":"action=login&format=json","pathname":"/api.php","path":"/api.php?action=login&format=json","href":"http://<hostname>:<portnumber>/api.php?action=login&format=json"}
hello request :<hostname>/api.php?action=query&meta=tokens&format=json
parsedURL = {"protocol":"http:","slashes":true,"auth":null,"host":"<hostname>:<portnumber>","port":"<portnumber>","hostname":"<hostname>","hash":null,"search":"?action=query&meta=tokens&format=json","query":"action=query&meta=tokens&format=json","pathname":"/api.php","path":"/api.php?action=query&meta=tokens&format=json","href":"http://<hostname>:<portnumber>/api.php?action=query&meta=tokens&format=json"}
[get_final_token.js] +\
invalid csrf token

Did you notice, the parsedURL does include the port number in the hostname? "host":"<hostname>:<portnumber>","port":"<portnumber>"

config.json

{
  "sparql-endpoint": "http://<hostname>:<otherportnumber>/bigdata/namespace/wdq/sparql",
  "instance": "http://<hostname>:<portnumber>/api.php",
  "lang": "de",
  "username": "<botusername>",
  "password": "************",
  "clipboard": true,
  "bot": true
}

Summary

I do not think this is really an error inside of wikibase-cli, but I am a little bit stuck in the problem, and could need a hint. Did you solve similar problems already?

[MikaMaracuja]

MediaWiki Version is 1.28.0

[maxlath]

the auth code has changed quite a lot in the latest releases of wikibase-edit, and thus of wikibase-cli, did it solve your problem by any chance?

otherwise, I have been using/testing wikibase-edit/cli on MediaWiki >= 1.33, but I have little knowledge on what has change between MediaWiki releases, so no clue on if that could be an issue

[maxlath]

Closing for now: if that's still an issue with the latest version, please reopen with updated info