pramsey / pgsql-http

HTTP client for PostgreSQL, retrieve a web page from inside the database.
MIT License
1.3k stars 116 forks source link

Violation check constraint "http_method_check" for request 'MKCOL' #159

Closed datenschauer closed 1 year ago

datenschauer commented 1 year ago

First of all: thank you for the great extension, which helps me a lot in my current project! 👍

But my current issue is as follows: I need to call a Nextcloud API to create a folder when I insert some data into my database. The problem is, that Nexcloud API needs to receive an MKCOL request (see their documentation). So when I do this in my code:

SELECT status::text FROM http((
        'MKCOL',
         base_url || webdav_url || nc_create_folder,
         ARRAY [http_header('Authorization', 'Basic ' || encoded_user_pw)],
         NULL,
         NULL
        )::http_request)

I get the following Error: [23514] ERROR: value for domain http_method violates check constraint "http_method_check"

As it seems MKCOL requests are not allowed. But why? Is there any way around this issue?

Really appreciate any help! :-)

pramsey commented 1 year ago

The domain definition is there to keep people from generating random HTTP commands... maybe not a check worth having?

alter domain http_method drop constraint http_method_check;

But if you want it checked...

alter domain http_method add check (
  VALUE IN ('get','post','put','delete','patch','head','mkcol'));
datenschauer commented 1 year ago

Thank you! Why didn't I think about it myself? 😅 As to the check: Imho, I'm wondering if the check is really necessary, too, since random requests would be rejected by the APIs anyway, wouldn't they?

datenschauer commented 1 year ago

But something is still strange when sending requests with different methods.

When I send a request via GET or POST to Nextcloud's so called OCS API the authorization header works fine. Here is an example:

SELECT * FROM http((
    'GET',
    'https://mydomain/ocs/v2.php/apps/files_sharing/api/v1/shares',
    ARRAY [
        http_header('OCS-APIRequest', 'true'),
        http_header('Authorization', 'Basic ' || encode('admin:password'::bytea, 'base64'))
        ],
    NULL,
    NULL
    )::http_request);

But unfortunately this doesn't work with their Webdav API where I have to send the 'MKCOL' request:

SELECT * FROM http((
    'MKCOL',
    'https://mydomain/remote.php/dav/files/nextcloud-admin/neeew',
    ARRAY [
    http_header('Authorization', 'Basic ' || encode('admin:password'::bytea, 'base64'))
        ],
    NULL,
    NULL
    )::http_request);

When I do this I get an exception:

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>File with name //neeew could not be located</s:message>
</d:error>

And my Nextcloud logs say:

OC\Authentication\Exceptions\InvalidTokenException:
Token is too short for a generated token, should be the password during basic auth at <<closure>>

But I am sending the same authorization header as above (ok, actually without the 'OCS-APIRequest' header, but that's for OCS only, not for Webdav). Also the corresponding curl command works just fine (and also a test with node.js):

curl "https://mydomain/remote.php/dav/files/nextcloud-admin/neeew" -X MKCOL -u admin:password

Just wanted to check if there is something different behind the scenes, when I send an none GET, POST, PUT etc. request.

pramsey commented 1 year ago

If you read through the http_request() function you can see what's going on, but I don't see anything much different for a content-less request like this. And the error is odd too. You could turn on the client_min_messages to debug4 and see more messages in the pgsql-http side and up the verbosity on curl and see if you can see where the two diverge.

robe2 commented 1 year ago

@pramsey It looks like you end up shoving all unsupported into the GET bucket.

See: https://github.com/pramsey/pgsql-http/blob/master/http.c#L427

and https://github.com/pramsey/pgsql-http/blob/master/http.c#L1185

So I think MKCOL gets rewritten as GET. I have a nextcloud instance with Webdav enabled I can test. You do too :) the nextcloud.osgeo.org one.

I'll take a stab at fixing.

and I agree we should probably get rid of that domain check once we have the internal code fixed and assume the user knows what they are doing.

robe2 commented 1 year ago

@pramsey I took a stab at it and with my changes, I can do this now and see the folder get created in my nextcloud home folder

    SELECT h.content::xml FROM http((
    'MKCOL',
    'https://nextcloud.osgeo.org/remote.php/dav/files/robe/test',
    ARRAY [
        http_header('Authorization', 'Basic ' || encode('robe:somepassword'::bytea, 'base64'))
        ],
    NULL,
    NULL
    )::http_request) AS h;

It doesn't return an output so maybe I need to do something else to handle return. If I repeat the above again, I get an error as expected.

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\MethodNotAllowed</s:exception>
  <s:message>The resource you tried to create already exists</s:message>
</d:error>

I know you tried to add MKCOL as an option, but there are other options for WebDav, like the commonly used PROPFIND method. So I think it's best to just not bother checking.

I also tested:

    SELECT h.content::xml FROM http((
    'PROPFIND',
    'https://nextcloud.osgeo.org/remote.php/dav/files/robe',
    ARRAY [
        http_header('Authorization', 'Basic ' || encode('robe:somepassword'::bytea, 'base64'))
        ],
    NULL,
    NULL
    )::http_request) AS h;

and it returned in xml the list of documents in my nextcloud home folder.

The only thing I am unsure of is for other methods if we need to set other stuff in case there is a body. I suspect so. For that I just set the method, cause I wasn't sure what else needed to be done.

robe2 commented 1 year ago

For reference here is a listing of all https://webconcepts.info/concepts/http-method/ as you can see there are quite a few just for WebDav