CXuesong / WikiClientLibrary

/*🌻*/ Wiki Client Library is an asynchronous MediaWiki API client library targeting modern .NET platforms
https://github.com/CXuesong/WikiClientLibrary/wiki
Apache License 2.0
82 stars 16 forks source link

GeoSearchGenerator does not support query continuation #64

Open CXuesong opened 4 years ago

CXuesong commented 4 years ago

Currently, GeoSearchGenerator does not support query continuation, because MW API action=query&list=geosearch does not support so. This is tracked by phab:T95241 and phab:T78703.

Example response of https://en.wikipedia.org/w/api.php?action=query&maxlag=5&list=geosearch&gsradius=10&gsprimary=primary&gslimit=2&gsbbox=32.15%7C34.75%7C32%7C34.9

{
    "batchcomplete": "",
    "query": {
        "geosearch": [
            {
                "pageid": 18328987,
                "ns": 0,
                "title": "Beit Zvi",
                "lat": 32.078408333333336,
                "lon": 34.821713888888894,
                "dist": 489.4,
                "primary": ""
            },
            {
                "pageid": 46324352,
                "ns": 0,
                "title": "HaAliya HaShniya Garden",
                "lat": 32.0697,
                "lon": 34.8148,
                "dist": 1127.4,
                "primary": ""
            }
        ]
    }
}
CXuesong commented 4 years ago

I think the current mitigation we only have is to use a larger PaginationSize (gslimit) for now. But note that this value should be no greater than 500 for users and 5000 for users with apihighlimits right.

HarelM commented 4 years ago

The only workaround I can think of is to implement an area breakdown when limit is reached in this case. i.e. Take the rectangle, split it to 4 smaller rectangles and search those until the pagination limit of 500 is not reached.

CXuesong commented 4 years ago

That should help. However, IMO this is not general enough for WCL to implement a common logic to fill this gap. This approach does not applies to the search by radius.

I've pinged in T78703 and am going to see what happens. Before that, if necessary, you may need to implement the rectangle break-down logic by yourself.

HarelM commented 4 years ago

@CXuesong thanks again for the super quick response and detailed investigation of the issue. It's a delight to converse with you and see how professionally you handle things. So thanks again! I understand your point of view and I myself wasn't sure where this temporary workaround should be implemented. As a quick fix for the issue I have I'll implement this on my end, if needed of course you're welcome to copy the relevant code. The status of my fix and relevant commits will be tracked here, in case this will interest you: https://github.com/IsraelHikingMap/Site/issues/1126