v1 API throws errors - Githubissues

tripleee commented 6 years ago

Halflife stopped working on Friday. I'm speculating this is a regression in conjunction with the v2 API rollout. Simple Metasmoke queries for me have started returning copious errors.

halflife:2017-12-18 08:45:56,130:Query posts/98146 did not return valid JSON: '<!DOCTYPE html>\n<html lang="en">\n<head>\n <meta charset="utf-8" />\n <title>Action Controller: Exception caught</title>\n <style>\n body {\n background-color: #FAFAFA;\n color: #333;\n margin: 0px;\n }\n\n body, p, ol, ul, td {\n font-family: helvetica, verdana, arial, sans-serif;\n font-size: 13px;\n line-height: 18px;\n }\n\n pre {\n font-size: 11px;\n white-space: pre-wrap;\n }\n\n pre.box {\n border: 1px solid #EEE;\n padding: 10px;\n margin: 0px;\n width: 958px;\n }\n\n header {\n color: #F0F0F0;\n background: #C52F24;\n padding: 0.5em 1.5em;\n }\n\n h1 {\n margin: 0.2em 0;\n line-height: 1.1em;\n font-size: 2em;\n }\n\n h2 {\n color: #C52F24;\n line-height: 25px;\n }\n\n .details {\n border: 1px solid #D0D0D0;\n border-radius: 4px;\n margin: 1em 0px;\n display: block;\n width: 978px;\n }\n\n .summary {\n padding: 8px 15px;\n border-bottom: 1px solid #D0D0D0;\n display: block;\n }\n\n .details pre {\n margin: 5px;\n border: none;\n }\n\n #container {\n box-sizing: border-box;\n width: 100%;\n padding: 0 1.5em;\n }\n\n .source * {\n margin: 0px;\n padding: 0px;\n }\n\n .source {\n border: 1px solid #D9D9D9;\n background: #ECECEC;\n width: 978px;\n }\n\n .source pre {\n padding: 10px 0px;\n border: none;\n }\n\n .source .data {\n font-size: 80%;\n overflow: auto;\n background-color: #FFF;\n }\n\n .info {\n padding: 0.5em;\n }\n\n

tripleee commented 6 years ago

Just to be explicit, the call is to /api/posts/98146 in this example.

tripleee commented 6 years ago

@j-f1 suggested that this might be due to updated filters, but I get this even without any filter, or with a new filter I just pieced together.

>>> r = requests.get('https://metasmoke.erwaysoftware.com/api/posts/98236',
... params={'key': my_api_key, filter: ''})
>>> r
<Response [500]>
>>> r = requests.get('https://metasmoke.erwaysoftware.com/api/posts/98236',
... params={'key': my_api_key, filter: 'IGIIOLLJFJHLJIKLKIHJNGNOMHLMHF'})
>>> r
<Response [500]>

(Different post, perhaps only to demonstrate that it's not a particular post which breaks, either.)

j-f1 commented 6 years ago

I get this even without any filter

The default filters are broken :(

or with a new filter I just pieced together

/api/filters gets you the v2-style filter. You have to work from a local clone of MS and use Filterator::V1 to get v1-style filters.

tripleee commented 6 years ago

Can you provide some additional guidance? I set up a local Docker image of the latest Metasmoke master branch but the /api/filters route gives me the same filter as the older version I tried before (AAAAAAAAAPSjgAAAAAABAA== for this set of fields).

Undo1 commented 6 years ago

@tripleee I think you've got a Python issue there, actually. This:

params={'key': my_api_key, filter: ''})

shows up in my logs as this:

Parameters: {"<built-in function filter>"=>"", "key"=>"..."}

You probably want:

params={'key': my_api_key, 'filter': ''})

Which seems to work properly.

Undo1 commented 6 years ago

I just pulled logs; the above isn't the root issue. Default filters (@ArtOfCode-) are the root issue. Parameters appear as expected for the request you posted, with filter not being present in the params dictionary.

tripleee commented 6 years ago

@Undo1 Thanks! I was thinking of the default filters myself just now -- I think the API should simply return an unfiltered result if you don't specify a filter. Should I create a separate bug report about that?

tripleee commented 6 years ago

Thanks for the hint, Halflife is back up now with 'filter': ''

tripleee commented 6 years ago

"<built-in function filter>"=>""

Is that exploitable? Can I pass in arbitrary Ruby code to run?

j-f1 commented 6 years ago

@tripleee When your code is serialized from Python, the filter key is evaluated to the builtin filter function. When stringifying the data for transport, Python calls str(filter), which returns the string you see.

ArtOfCode- commented 6 years ago

For better or worse, the V1 API is essentially dead. It's possible to fix it, but given how buggy it is even when filters do work correctly, I'm not entirely sure it's worth it.

j-f1 commented 6 years ago

Hey @ArtOfCode-! I’m trying to upgrade /pings/mods to the new API. Is there any way to get the list of users with moderator_sites from API v2?

ArtOfCode- commented 6 years ago

@j-f1 Negative, but I'll add a mod sites API - it's dead easy now :)

ArtOfCode- commented 6 years ago

f9f0501bba640bb3f59f67dc26e2caa9f9995edc 79a40c130c3ea37b3f994728513069671b2d95a0

Undo1 commented 6 years ago

@ArtOfCode- Closing this as wontfix, then? Either that or fixing default filters, right?

j-f1 commented 6 years ago

All of the user_ids are null :( @ArtOfCode-

tripleee commented 6 years ago

With the workaround, wontfix seems like the way to go with this.

tripleee commented 6 years ago

And thanks for explaining my silly "security problem"; it's obvious now, but I thought I was seeing that from Ruby.

ArtOfCode- commented 6 years ago

Yeah, this can close. @j-f1 that, however, is an issue... wanna make a new issue for it? I'm baffled how anything works if that's truly null.

Undo1 commented 6 years ago

oh, hah. @ArtOfCode- That happens because the ModeratorSite system is weird. It sometimes tries to create new records on every run... then doesn't delete the old ones, but sets the UID to null. Could use some refactoring, definitely.

Charcoal-SE / metasmoke

v1 API throws errors #301