[GSoC] Added leaderboard.py

[madhurgupta10]

This API let us fetch the uploads of a user in a particular duration - weekly, yearly and all-time

Added uploads_by_user_and_duration.py

[madhurgupta10]

Hi @urbanecm I am working on the leaderboard task for GSOC 2020, could you please deploy this file for me?

[urbanecm]

Hi @urbanecm I am working on the leaderboard task for GSOC 2020, could you please deploy this file for me?

Sure, I just left a review.

[madhurgupta10]

@urbanecm Thank you for the review, I will make the changes. I would like to deploy and test my API on https://tools.wmflabs.org/commons-android-app/tool-commons-android-app I tried to activate a web service but getting 404

I followed the instruction on https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web but getting this Could not find ~/www/python/src. Are you sure you have a proper uwsgi application in ~/www/python/src?

[urbanecm]

This has to be run using the default webserver, not uwsgi - it's not ready for that. Simply do webservice stop to halt everything you have there and then webservice --backend=gridengine lighttpd start, it should work. Not sure if the kubernetes container has support for both PHP and Python script in one tool tbh.

Maybe I should just go and rewrite this into a proper uwsgi service, but that's for a different project :).

[madhurgupta10]

This has to be run using the default webserver, not uwsgi - it's not ready for that. Simply do webservice stop to halt everything you have there and then webservice --backend=gridengine lighttpd start, it should work. Not sure if the kubernetes container has support for both PHP and Python script in one tool tbh.

Maybe I should just go and rewrite this into a proper uwsgi service, but that's for a different project :).

This one throws the following error

tools.commons-android-app@interactive:~$  webservice --backend=gridengine lighttpd start
Traceback (most recent call last):
  File "/usr/bin/webservice", line 453, in <module>
    if job.get_state() != Backend.STATE_STOPPED:
  File "/usr/lib/python2.7/dist-packages/toolsws/backends/gridengine.py", line 129, in get_state
    job = self._get_job_xml()
  File "/usr/lib/python2.7/dist-packages/toolsws/backends/gridengine.py", line 67, in _get_job_xml
    output = subprocess.check_output(["qstat", "-xml"])
  File "/usr/lib/python2.7/subprocess.py", line 216, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
tools.commons-android-app@interactive:~$

[madhurgupta10]

@urbanecm Also, I can't find my file inside the shell

tools.commons-android-app@interactive:~/public_html/tool-commons-android-app$ ls
feedback.py  README.md  uploadsbyuser.py  wikidataedits.py  wmflabs
tools.commons-android-app@interactive:~/public_html/tool-commons-android-app$ pwd
/data/project/commons-android-app/public_html/tool-commons-android-app
tools.commons-android-app@interactive:~/public_html/tool-commons-android-app$

[madhurgupta10]

@urbanecm I tried to start the service using webservice start command and I am getting the following logs for the existing file uploadsbyuser.py

    conn.query(q)
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 852, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1053, in _read_query_result
    result.read()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1336, in read
    first_packet = self.connection._read_packet()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1010, in _read_packet
    packet.check_error()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1054, u"Unknown column 'log_user' in 'where clause'")
2020-06-14 13:54:34: (server.c.1751) [note] graceful shutdown started
2020-06-14 13:54:34: (server.c.1828) server stopped by UID = 53794 PID = 11139
2020-06-14 17:45:44: (server.c.1464) server started (lighttpd/1.4.53)
Traceback (most recent call last):
  File "/data/project/commons-android-app/public_html//tool-commons-android-app/feedback.py", line 8, in <module>
    from wmflabs import db
ImportError: No module named wmflabs
Traceback (most recent call last):
  File "/data/project/commons-android-app/public_html//tool-commons-android-app/uploadsbyuser.py", line 6, in <module>
    from wmflabs import db
ImportError: No module named wmflabs
Traceback (most recent call last):
  File "/data/project/commons-android-app/public_html//tool-commons-android-app/uploadsbyuser.py", line 6, in <module>
    from wmflabs import db
ImportError: No module named wmflabs
2020-06-14 17:48:09: (server.c.2059) server stopped by UID = 0 PID = 0
2020-06-14 18:25:52: (server.c.1464) server started (lighttpd/1.4.53)
Traceback (most recent call last):
  File "/data/project/commons-android-app/public_html//tool-commons-android-app/uploadsbyuser.py", line 6, in <module>
    from wmflabs import db
ImportError: No module named wmflabs

[madhurgupta10]

I also noticed the script will fail if you pass an unexpected duration. I recommend testing that, and failing gracefully.

@urbanecm I have added the suggested changes, also renamed the file to the leaderboard, in future I will make all my changes in this particular file so no need to deploy files again and again :)

[madhurgupta10]

This has to be run using the default webserver, not uwsgi - it's not ready for that. Simply do webservice stop to halt everything you have there and then webservice --backend=gridengine lighttpd start, it should work. Not sure if the kubernetes container has support for both PHP and Python script in one tool tbh. Maybe I should just go and rewrite this into a proper uwsgi service, but that's for a different project :).

This one throws the following error

tools.commons-android-app@interactive:~$  webservice --backend=gridengine lighttpd start
Traceback (most recent call last):
  File "/usr/bin/webservice", line 453, in <module>
    if job.get_state() != Backend.STATE_STOPPED:
  File "/usr/lib/python2.7/dist-packages/toolsws/backends/gridengine.py", line 129, in get_state
    job = self._get_job_xml()
  File "/usr/lib/python2.7/dist-packages/toolsws/backends/gridengine.py", line 67, in _get_job_xml
    output = subprocess.check_output(["qstat", "-xml"])
  File "/usr/lib/python2.7/subprocess.py", line 216, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
tools.commons-android-app@interactive:~$

@urbanecm After updating the phab repo, service started with this command but it is using python2 instead of 3

[madhurgupta10]

Closing this PR for now since the test tools are now working, I will push final API to this repo :)

Sorry for the trouble @urbanecm