search

the-qa-company / WikibaseSync

Library to copy entities from one Wikibase to another and to keep them in sync
MIT License
30 stars 9 forks source link

`sudo` for Windows PC? #17

Closed ugochimobi closed 2 years ago

ugochimobi commented 2 years ago

Hi, I just want to ask, can this be used with a Windows PC? As much as I know, I don't think Windows understands this sudo command. However, I tried using Ubuntu but all the packages that I tried to install failed, throwing an Unable to fetch package error.

Any way to handle this? I appreciate your anticipated response.

D063520 commented 2 years ago

Hi,

it is a python project, so yes it can run on any platform. The general idea is to install the python dependencies and run the script. Maybe this link can help:

https://mothergeo-py.readthedocs.io/en/latest/development/how-to/venv-win.html

The instructions are more for a linux distribution. For the "Unable to fetch package" maybe you can do

sudo apt-get update

before. It might fix your problem.

Salut D063520

ugochimobi commented 2 years ago

Hi,

it is a python project, so yes it can run on any platform. The general idea is to install the python dependencies and run the script. Maybe this link can help:

https://mothergeo-py.readthedocs.io/en/latest/development/how-to/venv-win.html

The instructions are more for a linux distribution. For the "Unable to fetch package" maybe you can do

sudo apt-get update

before. It might fix your problem.

Salut D063520

Oh, right, thanks! Indeed, the instructions are more linux-based than windows; but I think I tried using that sudo apt-get update and I still got an error, although I can't recall vividly, but I'd try again; or better still, can I get a more windows-based command for those commands?

ugochimobi commented 2 years ago

Hi,

it is a python project, so yes it can run on any platform. The general idea is to install the python dependencies and run the script. Maybe this link can help:

https://mothergeo-py.readthedocs.io/en/latest/development/how-to/venv-win.html

The instructions are more for a linux distribution. For the "Unable to fetch package" maybe you can do

sudo apt-get update

before. It might fix your problem.

Salut D063520

Oh, right, thanks! Indeed, the instructions are more linux-based than windows; but I think I tried using that sudo apt-get update and I still got an error, although I can't recall vividly, but I'd try again; or better still, can I get a more windows-based command for those commands?

D063520 commented 2 years ago

Hi,

the link I send:

https://mothergeo-py.readthedocs.io/en/latest/development/how-to/venv-win.html

is to set up a virtual environment on windows.

This replaces the other commands. Please try to follow them

Salut D063520

ugochimobi commented 2 years ago

Hi,

the link I send:

https://mothergeo-py.readthedocs.io/en/latest/development/how-to/venv-win.html

is to set up a virtual environment on windows.

This replaces the other commands. Please try to follow them

Salut D063520

Many thanks, Dennis! \o/

ugochimobi commented 2 years ago

So, after a long run, I managed to set up the virtual environment, but when I run the command, for example, python import_one.py Q1 it returns Unable to create process using 'C:\Users\Myusername\AppData\Local\Programs\Python\Python310\python.exe import_one.py Q1' Any Idea of what I did wrong?

D063520 commented 2 years ago

I guess this is an issue of python on Windows. Maybe this:

https://stackoverflow.com/questions/37220055/pip-fatal-error-in-launcher-unable-to-create-process-using

?

ugochimobi commented 2 years ago

I guess this is an issue of python on Windows. Maybe this:

https://stackoverflow.com/questions/37220055/pip-fatal-error-in-launcher-unable-to-create-process-using

?

That didn't solve it either, I tried running the pip upgrade python -m pip install --upgrade pip command but it didn't work.

D063520 commented 2 years ago

Hi, this problem is not really related to the library, but to setting up python on a windows machine. I do not use windows, but this tutorial seams to be relatively complete:

https://phoenixnap.com/kb/how-to-install-python-3-windows

ugochimobi commented 2 years ago

Hi, this problem is not really related to the library, but to setting up python on a windows machine. I do not use windows, but this tutorial seams to be relatively complete:

https://phoenixnap.com/kb/how-to-install-python-3-windows

Hello Dennis, thanks and yes it not related to the library at all, maybe I'd just try following the instructions as suggested above over again but this time, with my Ubuntu.

ugochimobi commented 2 years ago

Hi Dennis, I trust you're doing great :) I managed to run it on my Desktop Computer that runs Windows 10 (64bit), but now I get an HTTP error, what do you think causes this? my network? I actually tried changing my internet connection but still returns an HTTP error, the full error below;

ERROR: Traceback (most recent call last):
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\data\api.py", line 1489, in _http_request
    response = http.request(self.site, uri=uri,
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\comms\http.py", line 232, in request
    r = fetch(baseuri, headers=headers, **kwargs)
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\comms\http.py", line 393, in fetch
    callback(response)
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\comms\http.py", line 277, in error_handling_callback
    raise ConnectionError(response)
ConnectionError: HTTPConnectionPool(host='http', port=80): Max retries exceeded with url: /w/api.php (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000020ECF7B7550>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

Traceback (most recent call last):
  File "C:\Users\USER\WikibaseSync\import_one.py", line 9, in <module>
    wikibase_repo = wikibase.data_repository()
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\site\_apisite.py", line 1151, in data_repository
    data = req.submit()
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\data\api.py", line 2068, in submit
    cached_available = self._load_cache()
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\data\api.py", line 2038, in _load_cache
    self._add_defaults()
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\data\api.py", line 1250, in _add_defaults
    and self.site.mw_version >= '1.25wmf5'):
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\site\_apisite.py", line 1111, in mw_version
    mw_ver = MediaWikiVersion(self.version())
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\site\_apisite.py", line 1084, in version
    version = self.siteinfo.get('generator', expiry=1).split(' ')[1]
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\site\_siteinfo.py", line 303, in get
    preloaded = self._get_general(key, expiry)
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\site\_siteinfo.py", line 242, in _get_general
    default_info = self._get_siteinfo(props, expiry)
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\site\_siteinfo.py", line 167, in _get_siteinfo
    data = request.submit()
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\data\api.py", line 2070, in submit
    self._data = super().submit()
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\data\api.py", line 1797, in submit
    response, use_get = self._http_request(use_get, uri, body, headers,
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\data\api.py", line 1489, in _http_request
    response = http.request(self.site, uri=uri,
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\comms\http.py", line 232, in request
    r = fetch(baseuri, headers=headers, **kwargs)
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\comms\http.py", line 393, in fetch
    callback(response)
  File "C:\Users\USER\venv\lib\site-packages\pywikibot\comms\http.py", line 277, in error_handling_callback
    raise ConnectionError(response)
ConnectionError: HTTPConnectionPool(host='http', port=80): Max retries exceeded with url: /w/api.php (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000020ECF7B7550>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))
CRITICAL: Exiting due to uncaught exception <class 'ConnectionError'>
D063520 commented 2 years ago

Hi!

On which address is your wikibase? What did you put in the config/application.config.ini under domain?

Salut D063520

ugochimobi commented 2 years ago

I put the website domain http://gratisdata.miraheze.org , please take a look at my fork here https://github.com/ugochimobi/WikibaseSync/blob/master/config/application.config.ini

ugochimobi commented 2 years ago

I also tried changing them to HTTPS still returns same error

D063520 commented 2 years ago

In:

https://github.com/the-qa-company/WikibaseSync/blob/master/config/application.config.ini

there is only the domain, not the protocol. So that part should be:

domain = gratisdata.miraheze.org protocol = http

ugochimobi commented 2 years ago

Oh, can you imagine :), I missed that Thank you very much! Now it keeps retrying and returning a 200 Status code, and a WARNING: Non-JSON response received from server my:my for url. I wonder why?

D063520 commented 2 years ago

can you paste the error message?

ugochimobi commented 2 years ago

Sure! here it is

WARNING: Non-JSON response received from server my:my for url
https://gratisdata.miraheze.org/w/api.php
The server may be down.
Status code: 200

The text message is:

MediaWiki API help - Gratisdata
document.documentElement.className="client-js";RLCONF={"wgBreakFrames":!0,"wgSeparatorTransformTable":["",""],"wgDigitTransformTable":["",""],"wgDefaultDateFormat":"dmy","wgMonthNames":["","January","February","March","April","May","June","July","August","September","October","November","December"],"wgRequestId":"_____","wgCSPNonce":!1,"wgCanonicalNamespace":"Special","wgCanonicalSpecialPageName":"ApiHelp","wgNamespaceNumber":-1,"wgPageName":"Special:ApiHelp","wgTitle":"ApiHelp","wgCurRevisionId":0,"wgRevisionId":0,"wgArticleId":0,"wgIsArticle":!1,"wgIsRedirect":!1,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":[],"wgPageContentLanguage":"en","wgPageContentModel":"wikitext","wgRelevantPageName":"Special:ApiHelp","wgRelevantArticleId":0,"wgIsProbablyEditable":!1,"wgRelevantPageIsProbablyEditable":!1,"wgVisualEditor":{"pageLanguageCode":"en","pageLanguageDir":"ltr","pageVariantFallbacks":"en"},"wgMFDisplayWikibaseDescriptions":
{"search":!1,"nearby":!1,"watchlist":!1,"tagline":!1},"wgMediaViewerOnClick":!0,"wgMediaViewerEnabledByDefault":!0,"wgNoticeProject":"all","egMapsScriptPath":"/w/extensions/Maps/","egMapsDebugJS":!1,"egMapsAvailableServices":["leaflet"],"egMapsLeafletLayersApiKeys":{"MapBox":"","MapQuestOpen":"","Thunderforest":"","GeoportailFrance":""},"wgULSCurrentAutonym":"English","wgEditSubmitButtonLabelPublish":!1,"wgCentralAuthMobileDomain":!1,"wgSiteNoticeId":"69.0"};RLSTATE={"ext.globalCssJs.user.styles":"ready","site.styles":"ready","noscript":"ready","user.styles":"ready","ext.globalCssJs.user":"ready","user":"ready","user.options":"loading","mediawiki.hlist":"ready","mediawiki.apipretty":"ready","mediawiki.skinning.interface":"ready","ext.MobileDetect.nomobile":"ready","ext.CookieWarning.styles":"ready","oojs-ui-core.styles":"ready","oojs-ui.styles.indicators":"ready","mediawiki.widgets.styles":"ready","oojs-ui-core.icons":"ready",
"ext.visualEditor.desktopArticleTarget.noscript":"ready","ext.DarkMode.styles":"ready","ext.dismissableSiteNotice.styles":"ready"};RLPAGEMODULES=["site","mediawiki.page.ready","ext.CookieWarning","ext.eventLogging","ext.visualEditor.desktopArticleTarget.init","ext.visualEditor.targetLoader","ext.urlShortener.toolbar","ext.gadget.AuthorityControl","ext.gadget.SiteIdToInterwiki","ext.gadget.ProtectionIndicators","ext.gadget.imagelinks","ext.gadget.NewSection","ext.gadget.formWizard","wikibase.ui.entitysearch","ext.centralNotice.geoIP","ext.DarkMode","ext.centralauth.centralautologin","ext.dismissableSiteNotice"];
(RLQ=window.RLQ||[]).push(function(){mw.loader.implement("user.options@1hzgi",function($,jQuery,require,module){/*@nomin*/mw.user.tokens.set({"patrolToken":"+\\","watchToken":"+\\","csrfToken":"+\\"});
});});

WARNING: Waiting 20.0 seconds before retrying.

Exactly how it comes out from my CP.

D063520 commented 2 years ago

Hi,

the above problem was an error with the protocol. It should be https and not http.

I made an account on your wikibase. I run into anther problem which is that the code is not supporting latest mediawiki installations Was not aware of this. I put the changes in the pywikibot7 branch.

The branch is configured in a way to be adapted to gratispedia. I created a new account called D063520. Apparently "normal" users cannot create properties. Can you try with your admin account? Basically start from the branch and follow the instructions to change the user, password and bot. Otherwise you can ping me at dennis.diefenbach@the-qa-company.com if you can share an admin account an I can try to set it up.

Also I cannot test fully since one need a sparql endpoint. The sparql endpoint you indicated is only local. The sparql endpoint needs to be in sync with the wikibase otherwise it is not working.

Let me know!

Salut D063520

PS: when I edit with the configuration in the pywikibot7 branch I get:

permissiondenied: You do not have the permissions needed to carry out this action. [messages: [{'name': 'wikibase-api-permissiondenied', 'parameters': [], 'html': {'': 'You do not have the permissions needed to carry out this action.'}}, {'name': 'namespaceprotected', 'parameters': ['Property', 'edit'], 'html': {'': 'You do not have permission to edit pages in the Property namespace.'}}, {'name': 'namespaceprotected', 'parameters': ['Property', 'createpage'], 'html': {'': 'You do not have permission to edit pages in the Property namespace.'}}, {'name': 'badaccess-groups', 'parameters': ['[[Gratisdata:Property creators|Property creators]], [[m:Special:MyLanguage/User groups#Administrators|Administrators]]', 2], 'html': {'': 'The action you have requested is limited to users in one of the groups: Property creators, Administrators.'}}, {'name': 'namespaceprotected', 'parameters': ['Property', 'property-create'], 'html': {'*': 'You do not have permission to edit pages in the Property namespace.'}}]; help: See https://gratisdata.miraheze.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/&gt; for notice of API deprecations and breaking changes.]

ugochimobi commented 2 years ago

Hi,

the above problem was an error with the protocol. It should be https and not http.

I made an account on your wikibase. I run into anther problem which is that the code is not supporting latest mediawiki installations Was not aware of this. I put the changes in the pywikibot7 branch.

The branch is configured in a way to be adapted to gratispedia. I created a new account called D063520. Apparently "normal" users cannot create properties. Can you try with your admin account? Basically start from the branch and follow the instructions to change the user, password and bot. Otherwise you can ping me at dennis.diefenbach@the-qa-company.com if you can share an admin account an I can try to set it up.

Also I cannot test fully since one need a sparql endpoint. The sparql endpoint you indicated is only local. The sparql endpoint needs to be in sync with the wikibase otherwise it is not working.

Let me know!

Salut D063520

PS: when I edit with the configuration in the pywikibot7 branch I get:

permissiondenied: You do not have the permissions needed to carry out this action. [messages: [{'name': 'wikibase-api-permissiondenied', 'parameters': [], 'html': {'': 'You do not have the permissions needed to carry out this action.'}}, {'name': 'namespaceprotected', 'parameters': ['Property', 'edit'], 'html': {'': 'You do not have permission to edit pages in the Property namespace.'}}, {'name': 'namespaceprotected', 'parameters': ['Property', 'createpage'], 'html': {'': 'You do not have permission to edit pages in the Property namespace.'}}, {'name': 'badaccess-groups', 'parameters': ['[[Gratisdata:Property creators|Property creators]], [[m:Special:MyLanguage/User groups#Administrators|Administrators]]', 2], 'html': {'': 'The action you have requested is limited to users in one of the groups: Property creators, Administrators.'}}, {'name': 'namespaceprotected', 'parameters': ['Property', 'property-create'], 'html': {'*': 'You do not have permission to edit pages in the Property namespace.'}}]; help: See https://gratisdata.miraheze.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/ for notice of API deprecations and breaking changes.]

Oh wow @D063520 thank you for taking time to try this out, I'm amazed \o/

Hum, the error you shared on the latter is definitely something related to permissions which is something I can do. So what if I give your account sysop permission? What do you think? I think that'd be preferrable?

I'd also try it on my own, but I'd give your account sysop permission.

D063520 commented 2 years ago

This is fine for me. I promise I will not break anything. Can you give me an example entity you would like to import? There is still the problem with the SPARQL endpoint. Is there a public one?

ugochimobi commented 2 years ago

This is fine for me. I promise I will not break anything. Can you give me an example entity you would like to import? There is still the problem with the SPARQL endpoint. Is there a public one?

Okay, nice, Hum, for the SPARQL endpoint, I think we'd have a little problem with that, cuz there's currently no endpoint for gratisdata yet, it won't work?

D063520 commented 2 years ago

No, without a SPARL endpoint it will not work. Maybe you can ping here once you set up one? You know that using the wikibase docker pipeline https://github.com/wmde/wikibase-release-pipeline you get everything for free?

ugochimobi commented 2 years ago

No, without a SPARL endpoint it will not work. Maybe you can ping here once you set up one? You know that using the wikibase docker pipeline https://github.com/wmde/wikibase-release-pipeline you get everything for free?

I have no idea about that, but at least not now again, let me check it out.

PS: I hope I didn't give the wrong account sysop :)

ugochimobi commented 2 years ago

I couldn't figure out how to get a SPARQL endpoint with that? Lol

D063520 commented 2 years ago

maybe this helps:

https://www.mediawiki.org/wiki/Wikibase/Docker

ugochimobi commented 2 years ago

I've reached out to Miraheze System Administrator (Gratisdata's host) they just told me Docker wasn't use to install Wikibase over at miraheze....

On Sun, 12 Jun 2022, 20:40 Dennis Diefenbach, @.***> wrote:

maybe this helps:

https://www.mediawiki.org/wiki/Wikibase/Docker

— Reply to this email directly, view it on GitHub https://github.com/the-qa-company/WikibaseSync/issues/17#issuecomment-1153272560, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATP5IZMLUI5SHZZDKIDU4X3VOY4JVANCNFSM5XUAYHIA . You are receiving this because you authored the thread.Message ID: @.***>

D063520 commented 2 years ago

Ah, you are not hosting it yourself .... then I think it will be difficult, I guess Miraheze is not offering all services related to wikibase.

ugochimobi commented 2 years ago

Wow, I can't believe this won't work out, so sad, anyway I'd talk to them about it as time goes on.

I am very grateful for your time that you've spent with me in this \o/

ugochimobi commented 2 years ago

Good evening Dennis, just a quick one; I was wondering since you were able to reach that part where the system denied your permission to create properties, why don't you try it again now that you have the permission let's see if it will work?

Kudos! \o/

D063520 commented 2 years ago

Hi, I'm sure it will not work. In fact without the SPARQL endpoint he will probably create duplicated entities since he cannot see the once that were already inserted .... Salut D063520

ugochimobi commented 2 years ago

I so much appreciate the time you've spent with me here! Thank you so much.

Can we probably chat in a private venue regarding Wikibase docker, I want you to advise/support me in setting up one if you don't mind. Thanks! \o/