search

nodeSolidServer / node-solid-server

Solid server on top of the file-system in NodeJS
https://solidproject.org/for-developers/pod-server
Other
1.78k stars 303 forks source link

Unable to create Containers #699

Closed markwilkinson closed 6 years ago

markwilkinson commented 6 years ago

I believe this is a bug, though I might be missing something...??

node-solid-server-4.0.14 (from the tar.gz release)

Curl-ing a message to create a container seems to result in the creation of a Resource instead. Please see the transactions below:

markw@markw ~ $ cat testcontainer.ttl 
@prefix ldp: <http://www.w3.org/ns/ldp#> .

<> a ldp:Container, ldp:BasicContainer .

markw@markw ~ $ curl -k -i -X POST "https://localhost:8443/" -H "Slug: testcontainer" -H "Link: <http://www.w3.org/ns/ldp/BasicContainer>; rel="type"" -d @testcontainer.ttl
HTTP/1.1 201 Created
X-Powered-By: solid-server
Vary: Origin
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate
Allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
Link: <.acl>; rel="acl", <.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Container>; rel="type", <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"
Location: /testcontainer
Content-Type: text/plain; charset=utf-8
Content-Length: 7
ETag: W/"7-rM9AyJuqT6iOan/xHh+AW+7K/T8"
Date: Thu, 07 Jun 2018 07:33:27 GMT
Connection: keep-alive

Created

markw@markw ~ $ curl -k -i -X GET "https://localhost:8443/"
HTTP/1.1 200 OK
X-Powered-By: solid-server
Vary: Origin
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate
Allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
Link: <.acl>; rel="acl", <.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Container>; rel="type", <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"
MS-Author-Via: SPARQL
Updates-Via: wss://localhost:8443
Content-Type: text/turtle
Date: Thu, 07 Jun 2018 07:33:51 GMT
Connection: keep-alive
Transfer-Encoding: chunked

@prefix : <#>.
@prefix n0: <>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix terms: <http://purl.org/dc/terms/>.
@prefix XML: <http://www.w3.org/2001/XMLSchema#>.
@prefix st: <http://www.w3.org/ns/posix/stat#>.
@prefix html: <http://www.w3.org/ns/iana/media-types/text/html#>.

n0:
    a ldp:BasicContainer, ldp:Container;
    terms:modified "2018-06-07T07:33:27Z"^^XML:dateTime;
    ldp:contains <https://localhost:8443//index.html>, n0:testcontainer;
    st:mtime 1528356807.401;
    st:size 4096.
<https://localhost:8443//index.html>
    a html:Resource, ldp:Resource;
    terms:modified "2018-06-05T11:42:35Z"^^XML:dateTime;
    st:mtime 1528198955.5;
    st:size 942.
n0:testcontainer
    a ldp:Resource;
    terms:modified "2018-06-07T07:33:27Z"^^XML:dateTime;
    st:mtime 1528356807.401;
    st:size 83.
dmitrizagidulin commented 6 years ago

Hmm, yeah, that doesn’t seem right. Oh, you know what, could be a content-type issue. Try adding a content-type: text/turtle to the create container POST

markwilkinson commented 6 years ago

Unfortunately, almost the same result: (the POST includes the content-type header now, this is just the result for POSTing testcontainer2 versus testcontainer without that header (above) )

n0:testcontainer
    a ldp:Resource;
    terms:modified "2018-06-07T07:33:27Z"^^XML:dateTime;
    st:mtime 1528356807.401;
    st:size 83.
<https://localhost:8443//testcontainer2.ttl>
    a tur:Resource, ldp:Resource;
    terms:modified "2018-06-07T12:56:13Z"^^XML:dateTime;
    st:mtime 1528376173.189;
    st:size 83.

It's actually a bit surprising to see the degree to which the result differs with/without the Content-Type header. I saw the conversation (including TBL) about adding file type extensions even if the Slug doesn't have them, but... I stopped using Content-Type specifically because I wanted to control when there was an extension and when there was not. I don't like machines to second-guess me when they don't have to ;-)

dmitrizagidulin commented 6 years ago

Oh weird. Ok.

So, something's definitely off here. I know that creating containers via POST is working (see this integration test for example). What is different in your case?

dmitrizagidulin commented 6 years ago

Also, can you repeat this test on the latest develop branch? I vaguely recall that some of the issues (like the double // in https://localhost:8443//testcontainer2.ttl) have been recently fixed? Maybe not though.

markwilkinson commented 6 years ago

Hmmmm.... I am not able to test the devel version because it will not accept a self-signed certificate.

markwilkinson commented 6 years ago

b.t.w. I am running it as a docker image... if that makes a difference.

markwilkinson commented 6 years ago

OK, I cloned the current "develop" repo branch onto my production server that has a certificate. Configured everything identically to my localhost test environment. Started it as a docker image (and restarted after configuring the json and moving the cert and key into to the image).

The server responds, but it cannot find the root Container:


$ curl -k -i -X GET "https://linkeddata.systems:8443/"
HTTP/1.1 404 Not Found
X-Powered-By: solid-server
Vary: Origin
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Upda
tes-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate
Allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
Link: <.acl>; rel="acl", <.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Container>; rel="type", <http://www.w3.org/ns/l
dp#BasicContainer>; rel="type"
MS-Author-Via: SPARQL
Updates-Via: wss://linkeddata.systems:8443
Content-Type: text/html; charset=utf-8
Content-Length: 52
ETag: W/"34-dfZxo4myzCTnUawMXJ8ddFPSszk"
Date: Fri, 08 Jun 2018 13:07:15 GMT
Connection: keep-alive
Can't find file requested: data/linkeddata.systems/

config.json

{
  "root": "./data",
  "port": "8443",
  "serverUri": "https://linkeddata.systems:8443",
  "webid": false,
  "mount": "/",
  "configPath": "./config",
  "dbPath": "./.db",
  "sslKey": "./cert.key",
  "sslCert": "./cert.pem",
  "multiuser": true,
  "corsProxy": "/proxy"
}
dmitrizagidulin commented 6 years ago

Interesting. And what's the contents (like if you do ls) of your ./data dir?

markwilkinson commented 6 years ago

Not very informative :-)

$ sudo docker run -it node-solid-server /bin/bash root@569bf0c94b72:/usr/src/app# ls data/ root@569bf0c94b72:/usr/src/app#

dmitrizagidulin commented 6 years ago

@markwilkinson hmmm that’s not right. it should be creating a dir named ‘linkeddata.systems’ upon startup, inside ./data. That’s what’s causing the 404.

markwilkinson commented 6 years ago

I just did a docker restart to be 100% sure - nope, it isn't created. Is it creating that folder it based on the info in the config file, or is it looking somewhere else for that information?

dmitrizagidulin commented 6 years ago

So I guess the next question is why the dir is not being created. Is the app running as root, and if not, does the os user have write permission?

dmitrizagidulin commented 6 years ago

Oh also, about the self-signed certificate - try running it via solid-test start, that disables the cert checking

markwilkinson commented 6 years ago

Ummmm.... the permissions are whatever is inside of the Docker container... I start the docker image as root, for sure. Inside, I don't know what the permissions are.

markwilkinson commented 6 years ago

solid-test start? I can do that with a docker container?

dmitrizagidulin commented 6 years ago

Ah, I don’t know, I’m not familiar with our docker setup. Will take a look later.

Meanwhile, just to diagnose if it’s an fs permission, try starting it in single user mode (set multiuser to false in config) and try creating container.

markwilkinson commented 6 years ago

ha! Single user mode worked!

Not sure what single vs multi user really means, but... at least I am able to get back to my primary task now :-)

$ curl -k -i -X GET "https://linkeddata.systems:8443/"
HTTP/1.1 200 OK
X-Powered-By: solid-server
Vary: Origin
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Upda
tes-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate
Allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
Link: <.acl>; rel="acl", <.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Container>; rel="type", <http://www.w3.org/ns/l
dp#BasicContainer>; rel="type"
MS-Author-Via: SPARQL
Updates-Via: wss://linkeddata.systems:8443
Content-Type: text/turtle
Date: Mon, 11 Jun 2018 12:02:49 GMT
Connection: keep-alive
Transfer-Encoding: chunked
@prefix : <#>.
@prefix n0: <>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix terms: <http://purl.org/dc/terms/>.
@prefix XML: <http://www.w3.org/2001/XMLSchema#>.
@prefix loc: <https://linkeddata.systems:8443//localhost/>.
@prefix st: <http://www.w3.org/ns/posix/stat#>.
n0:
    a ldp:BasicContainer, ldp:Container;
    terms:modified "2018-06-08T12:47:17Z"^^XML:dateTime;
    ldp:contains <https://linkeddata.systems:8443//.gitkeep>, loc:;
    st:mtime 1528462037.282;
    st:size 4096.
<https://linkeddata.systems:8443//.gitkeep>
    a ldp:Resource;
    terms:modified "2018-06-08T12:04:34Z"^^XML:dateTime;
    st:mtime 1528459474;
    st:size 0.
loc:
    a ldp:BasicContainer, ldp:Container, ldp:Resource;
    terms:modified "2018-06-08T12:47:17Z"^^XML:dateTime;
    st:mtime 1528462037.282;
    st:size 4096.

(note that it still has duplicate //'s in various places)

Anyway, I'm happy!

Thanks!

markwilkinson commented 6 years ago

So, back to the original bug report - the behavior has not changed (this is a checkout of the develop branch four days ago).

POSTing the message described above (with or without a Content-Type header) still results in the creation of a Resource, rather than a Container.

???

dmitrizagidulin commented 6 years ago

Ah, damn, ok. (btw, the Content-type is currently required.)

dmitrizagidulin commented 6 years ago

And I'm assuming you're still passing along the right Link: header, right

markwilkinson commented 6 years ago

Command I am using is copy/paste from the original bug report:

$ curl -k -i -X POST "https://linkeddata.systems:8443/" -H "Slug: testcontainer" -H "Content-Type: text/turtle" -H 'Link: <http://www.w3.org/ns/ldp/BasicContainer>; rel="type"' -d @testcontainer.ttl

dmitrizagidulin commented 6 years ago

Heh heh I suspect it's a one-character typo that's causing your issue :)

The Link should be: <http://www.w3.org/ns/ldp#BasicContainer> - notice the # instead of / :)

markwilkinson commented 6 years ago

Goddammit! I'm so sorry for wasting your time!

Yes, that was the problem. (note that it is still generating some odd '//'s but that's not a blocker for the demo that I want to give next week)

Sorry again!

dmitrizagidulin commented 6 years ago

Hahahah no problem! :) Glad we sorted it out.

(Yeah, hmm, the '//' part is not good, I thought that got fixed recently...)