search

owncloud / ocis

:atom_symbol: ownCloud Infinite Scale Stack
https://doc.owncloud.com/ocis/next/
Apache License 2.0
1.38k stars 181 forks source link

Test the parallel deployment #2387

Closed butonic closed 2 years ago

butonic commented 3 years ago

We are working on a docker-compose file for a parallel deployment of oc10 and ocis: https://github.com/owncloud/ocis/pull/2302

While manual testing has revealed a few bugs the overall approach is getting more mature. Now, we have arrived at a stage where we want to make sure the corner cases also work. Switching between oc10 and ocis can be done per request by sending a owncloud-selector cookie:

With this in mind I propose to use the existing test suite against this docker-compose scenario with a small twist: When more than one user is involved (sharing) make the sharing user use oc10 and any recipient use ocis. This mimics the case where most users will be using oc10 and early adopters will be using ocis.

The testsuite will have to provision users using ldap though.

User Story

As a vendor i need to test my customer setup to guarantee stability

Acceptance Criteria

kiranparajuli589 commented 2 years ago

need-help

with the hosts and default .env configuration:

is there some configuration problem or error inside the request?

wkloucek commented 2 years ago

That's a bug in the basic auth authentication middleware (fix here: https://github.com/owncloud/ocis/pull/2755).

Please note also that owncloud-selector is not a header but a cookie. You also need to enable basic auth in that example since it is enabled by default. You can do so by adding PROXY_ENABLE_BASIC_AUTH: "true" to oCIS.

We should also set up a meeting next week to discuss how the testing setup can look like in CI.

wkloucek commented 2 years ago

@kiranparajuli589 the fix is merged, could you please try again :-)

Please note also that owncloud-selector is not a header but a cookie. You also need to enable basic auth in that example since it is enabled by default. You can do so by adding PROXY_ENABLE_BASIC_AUTH: "true" to oCIS.

kiranparajuli589 commented 2 years ago

Please note also that owncloud-selector is not a header but a cookie. You also need to enable basic auth in that example since it is enabled by default. You can do so by adding PROXY_ENABLE_BASIC_AUTH: "true" to oCIS.

@wkloucek sorry, the issue description says its a header so i was using it like that. i've also enable basic auth in ocis environment, but still propfind request for a file is failing like:

PROPFIND response 
  ❯ curl -X PROPFIND -u einstein:relativity -b 'owncloud-selector=ocis;path=/;' https://cloud.owncloud.test/remote.php/webdav/lorem1.txt -k -v
  *   Trying 127.0.0.1:443...
  * TCP_NODELAY set
  * Connected to cloud.owncloud.test (127.0.0.1) port 443 (#0)
  * ALPN, offering h2
  * ALPN, offering http/1.1
  * successfully set certificate verify locations:
  *   CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: /etc/ssl/certs
  * TLSv1.3 (OUT), TLS handshake, Client hello (1):
  * TLSv1.3 (IN), TLS handshake, Server hello (2):
  * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
  * TLSv1.3 (IN), TLS handshake, Certificate (11):
  * TLSv1.3 (IN), TLS handshake, CERT verify (15):
  * TLSv1.3 (IN), TLS handshake, Finished (20):
  * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
  * TLSv1.3 (OUT), TLS handshake, Finished (20):
  * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
  * ALPN, server accepted to use h2
  * Server certificate:
  *  subject: CN=TRAEFIK DEFAULT CERT
  *  start date: Nov 16 10:05:14 2021 GMT
  *  expire date: Nov 16 10:05:14 2022 GMT
  *  issuer: CN=TRAEFIK DEFAULT CERT
  *  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
  * Using HTTP2, server supports multi-use
  * Connection state changed (HTTP/2 confirmed)
  * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
  * Server auth using Basic with user 'einstein'
  * Using Stream ID: 1 (easy handle 0x5631ad5e8e10)
  > PROPFIND /remote.php/webdav/lorem1.txt HTTP/2
  > Host: cloud.owncloud.test
  > authorization: Basic ZWluc3RlaW46cmVsYXRpdml0eQ==
  > user-agent: curl/7.68.0
  > accept: */*
  > cookie: owncloud-selector=ocis;path=/;
  > 
  * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
  * Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
  < HTTP/2 500 
  < date: Tue, 16 Nov 2021 10:06:21 GMT
  < content-length: 0
  < 
  * Connection #0 to host cloud.owncloud.test left intact
wkloucek commented 2 years ago

Hmm... works on my side. My steps: (assuming basic auth enabled and pwd is the oCIS repo root)

    * Connected to cloud.owncloud.test (127.0.0.1) port 443 (#0)
    * ALPN, offering h2
    * ALPN, offering http/1.1
    * successfully set certificate verify locations:
    *  CAfile: /etc/ssl/certs/ca-certificates.crt
    *  CApath: none
    } [5 bytes data]
    * TLSv1.3 (OUT), TLS handshake, Client hello (1):
    } [512 bytes data]
    * TLSv1.3 (IN), TLS handshake, Server hello (2):
    { [122 bytes data]
    * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
    { [15 bytes data]
    * TLSv1.3 (IN), TLS handshake, Certificate (11):
    { [878 bytes data]
    * TLSv1.3 (IN), TLS handshake, CERT verify (15):
    { [264 bytes data]
    * TLSv1.3 (IN), TLS handshake, Finished (20):
    { [36 bytes data]
    * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
    } [1 bytes data]
    * TLSv1.3 (OUT), TLS handshake, Finished (20):
    } [36 bytes data]
    * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
    * ALPN, server accepted to use h2
    * Server certificate:
    *  subject: CN=TRAEFIK DEFAULT CERT
    *  start date: Nov 16 11:40:31 2021 GMT
    *  expire date: Nov 16 11:40:31 2022 GMT
    *  issuer: CN=TRAEFIK DEFAULT CERT
    *  SSL certificate verify result: self signed certificate (18), continuing anyway.
    * Using HTTP2, server supports multiplexing
    * Connection state changed (HTTP/2 confirmed)
    * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
    } [5 bytes data]
    * Server auth using Basic with user 'einstein'
    * Using Stream ID: 1 (easy handle 0x5597cfb7d9a0)
    } [5 bytes data]
    > PROPFIND /remote.php/webdav/ HTTP/2
    > Host: cloud.owncloud.test
    > authorization: Basic ZWluc3RlaW46cmVsYXRpdml0eQ==
    > user-agent: curl/7.79.1
    > accept: */*
    > cookie: owncloud-selector=ocis;path=/;
    > 
    { [5 bytes data]
    * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
    { [130 bytes data]
    * Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
    } [5 bytes data]
    < HTTP/2 207 
    < access-control-allow-origin: *
    < access-control-expose-headers: Tus-Resumable, Tus-Version, Tus-Extension
    < content-security-policy: default-src 'none';
    < content-type: application/xml; charset=utf-8
    < date: Tue, 16 Nov 2021 11:47:16 GMT
    < dav: 1, 3, extended-mkcol
    < set-cookie: owncloud-selector=ocis; Path=/
    < tus-extension: creation,creation-with-upload,checksum,expiration
    < tus-resumable: 1.0.0
    < tus-version: 1.0.0
    < vary: Origin
    < x-content-type-options: nosniff
    < x-download-options: noopen
    < x-frame-options: SAMEORIGIN
    < x-permitted-cross-domain-policies: none
    < x-robots-tag: none
    < x-xss-protection: 1; mode=block
    < 
    { [5 bytes data]
    100  2636    0  2636    0     0   7941      0 --:--:-- --:--:-- --:--:--  7963
    * Connection #0 to host cloud.owncloud.test left intact
    <?xml version="1.0" encoding="utf-8"?>
    <d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
      <d:response>
        <d:href>/remote.php/webdav/</d:href>
        <d:propstat>
          <d:prop>
            <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjU=</oc:id>
            <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjU=</oc:fileid>
            <d:getetag>"87eec814115952638b7ebe42121f7355"</d:getetag>
            <oc:permissions>RDNVCK</oc:permissions>
            <d:resourcetype>
              <d:collection/>
            </d:resourcetype>
            <oc:size>7394063</oc:size>
            <d:getlastmodified>Tue, 16 Nov 2021 10:37:56 GMT</d:getlastmodified>
            <oc:favorite>0</oc:favorite>
          </d:prop>
          <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
      </d:response>
      <d:response>
        <d:href>/remote.php/webdav/Documents/</d:href>
        <d:propstat>
          <d:prop>
            <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI0</oc:id>
            <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI0</oc:fileid>
            <d:getetag>"61938a03afb6b"</d:getetag>
            <oc:permissions>RDNVCK</oc:permissions>
            <d:resourcetype>
              <d:collection/>
            </d:resourcetype>
            <oc:size>36227</oc:size>
            <d:getlastmodified>Tue, 16 Nov 2021 10:37:55 GMT</d:getlastmodified>
            <oc:favorite>0</oc:favorite>
          </d:prop>
          <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
      </d:response>
      <d:response>
        <d:href>/remote.php/webdav/Photos/</d:href>
        <d:propstat>
          <d:prop>
            <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI2</oc:id>
            <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI2</oc:fileid>
            <d:getetag>"61938a0422b81"</d:getetag>
            <oc:permissions>RDNVCK</oc:permissions>
            <d:resourcetype>
              <d:collection/>
            </d:resourcetype>
            <oc:size>1011464</oc:size>
            <d:getlastmodified>Tue, 16 Nov 2021 10:37:56 GMT</d:getlastmodified>
            <oc:favorite>0</oc:favorite>
          </d:prop>
          <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
      </d:response>
      <d:response>
        <d:href>/remote.php/webdav/ownCloud%20Manual.pdf</d:href>
        <d:propstat>
          <d:prop>
            <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjIz</oc:id>
            <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjIz</oc:fileid>
            <d:getetag>"0dd548631bb572b4a69522d3bf9edd98"</d:getetag>
            <oc:permissions>RDNVW</oc:permissions>
            <d:resourcetype/>
            <d:getcontentlength>6346372</d:getcontentlength>
            <d:getcontenttype>application/pdf</d:getcontenttype>
            <d:getlastmodified>Tue, 16 Nov 2021 10:37:55 GMT</d:getlastmodified>
            <oc:checksums>
              <oc:checksum>SHA1:5f0037da44c98a32fce800996a1b23e860bb03aa MD5:b60258ec4cd04800b82e294a05448e93 ADLER32:36051af0 MD5:b60258ec4cd04800b82e294a05448e93 ADLER32:36051af0</oc:checksum>
            </oc:checksums>
            <oc:favorite>0</oc:favorite>
          </d:prop>
          <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
      </d:response>
    </d:multistatus>

Though I noticed that basic auth requests currently are ALWAYS routed to oC10... Fix is here: https://github.com/owncloud/ocis/pull/2779

kiranparajuli589 commented 2 years ago

@wkloucek thanks a lot. It's working now. The problem was due to not using the latest code. I had to pull the docker-compose. waiting for #2779 to be merged

individual-it commented 2 years ago

@wkloucek

Converting the setup to Starlark is also not nice

what is the issue with that?

When to run this? Nighly?

Nightly

Where to have the tests? oCIS repo? separate Repo?

I would suggest to keep it in ocis

wkloucek commented 2 years ago

@wkloucek

Converting the setup to Starlark is also not nice

We will have two setups then... one you can run locally and one that is run in CI. As far as I overlook it, this is only because we cannot start docker-compose stacks in CI. I didn't have time yet to think about different possibilities (eg. using an external server where the docker-compose stack runs). Are there any requirements from the test suite?

individual-it commented 2 years ago

@wkloucek I would suggest to let @kiranparajuli589 make it run in CI with starlark now, the same as we do with other CI runs to make some progress here and then when we have a better idea we can change it over

kiranparajuli589 commented 2 years ago

Update: I'm trying to check the fix PR. So far,

My concerns:

cc @wkloucek

wkloucek commented 2 years ago

The responses look different:

oC10: curl 'https://cloud.owncloud.test/remote.php/webdav/' -X 'PROPFIND' --insecure -b 'owncloud-selector=oc10;path=/;' -u einstein:relativity | xmllint -format -

<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
  <d:response>
    <d:href>/remote.php/webdav/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:20 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>7394063</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"5b8f9dcc45341f65093a811151f5710e"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:19 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>36227</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"6196624fa66c7"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:getcontentlength/>
        <d:getcontenttype/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:20 GMT</d:getlastmodified>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <d:quota-used-bytes>1011464</d:quota-used-bytes>
        <d:quota-available-bytes>-3</d:quota-available-bytes>
        <d:getetag>"6196625027839"</d:getetag>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:getcontentlength/>
        <d:getcontenttype/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/ownCloud%20Manual.pdf</d:href>
    <d:propstat>
      <d:prop>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:19 GMT</d:getlastmodified>
        <d:getcontentlength>6346372</d:getcontentlength>
        <d:resourcetype/>
        <d:getetag>"a0c9189ff5e96c6187f788cf9a254036"</d:getetag>
        <d:getcontenttype>application/pdf</d:getcontenttype>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
    <d:propstat>
      <d:prop>
        <d:quota-used-bytes/>
        <d:quota-available-bytes/>
      </d:prop>
      <d:status>HTTP/1.1 404 Not Found</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>

oCIS: curl 'https://cloud.owncloud.test/remote.php/webdav/' -X 'PROPFIND' --insecure -b 'owncloud-selector=ocis;path=/;' -u einstein:relativity | xmllint -format -

<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
  <d:response>
    <d:href>/remote.php/webdav/</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjU=</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjU=</oc:fileid>
        <d:getetag>"5b8f9dcc45341f65093a811151f5710e"</d:getetag>
        <oc:permissions>RDNVCK</oc:permissions>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <oc:size>7394063</oc:size>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:20 GMT</d:getlastmodified>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Documents/</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI3MA==</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI3MA==</oc:fileid>
        <d:getetag>"6196624fa66c7"</d:getetag>
        <oc:permissions>RDNVCK</oc:permissions>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <oc:size>36227</oc:size>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:19 GMT</d:getlastmodified>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/Photos/</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI3Mg==</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI3Mg==</oc:fileid>
        <d:getetag>"6196625027839"</d:getetag>
        <oc:permissions>RDNVCK</oc:permissions>
        <d:resourcetype>
          <d:collection/>
        </d:resourcetype>
        <oc:size>1011464</oc:size>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:20 GMT</d:getlastmodified>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/webdav/ownCloud%20Manual.pdf</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI2OQ==</oc:id>
        <oc:fileid>MTI4NGQyMzgtYWE5Mi00MmNlLWJkYzQtMGIwMDAwMDA5MTU3OjI2OQ==</oc:fileid>
        <d:getetag>"a0c9189ff5e96c6187f788cf9a254036"</d:getetag>
        <oc:permissions>RDNVW</oc:permissions>
        <d:resourcetype/>
        <d:getcontentlength>6346372</d:getcontentlength>
        <d:getcontenttype>application/pdf</d:getcontenttype>
        <d:getlastmodified>Thu, 18 Nov 2021 14:25:19 GMT</d:getlastmodified>
        <oc:checksums>
          <oc:checksum>SHA1:5f0037da44c98a32fce800996a1b23e860bb03aa MD5:b60258ec4cd04800b82e294a05448e93 ADLER32:36051af0 MD5:b60258ec4cd04800b82e294a05448e93 ADLER32:36051af0</oc:checksum>
        </oc:checksums>
        <oc:favorite>0</oc:favorite>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>

If you really want to make sure that your requests is answered by oCIS you could stop oC10 by running docker-compose stop oc10. The other way round does not work since all requests need to pass through the oCIS proxy.

saw-jan commented 2 years ago

Yeah, I can also confirm that the responses are now different with different selectors.

saw-jan commented 2 years ago

I cannot create user with:

curl 'https://cloud.owncloud.test/ocs/v2.php/cloud/users' -XPOST -u admin:admin \
-b 'owncloud-selector=oc10;path=/;' \
-H"content-type=application/x-www-form-urlencoded" \
-d"userid=Alice" -d"password=1234" -d"displayname=Alice Hansen" -d"username=Alice" -d"email=alice@example.com" \
-vk | xmllint --format -

response:

> POST /ocs/v2.php/cloud/users HTTP/2
> Host: cloud.owncloud.test
> authorization: Basic YWRtaW46YWRtaW4=
> user-agent: curl/7.68.0
> accept: */*
> cookie: owncloud-selector=oc10;path=/;
> content-length: 90
> content-type: application/x-www-form-urlencoded
> 
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [130 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
} [5 bytes data]
* We are completely uploaded and fine
{ [5 bytes data]
< HTTP/2 401 
< date: Mon, 22 Nov 2021 06:45:13 GMT
< www-authenticate: Bearer realm="cloud.owncloud.test", charset="UTF-8"
* Authentication problem. Ignoring this.
< www-authenticate: Basic realm="cloud.owncloud.test", charset="UTF-8"
< content-length: 0
< 
{ [0 bytes data]
100    90    0     0  100    90      0   1428 --:--:-- --:--:-- --:--:--  1428
* Connection #0 to host cloud.owncloud.test left intact
-:1: parser error : Document is empty

Also tried with einstein user, response:

< HTTP/2 401 
< cache-control: no-store, no-cache, must-revalidate
< content-security-policy: default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *
< content-type: text/xml; charset=UTF-8
< date: Mon, 22 Nov 2021 06:48:59 GMT
< expires: Thu, 19 Nov 1981 08:52:00 GMT
< pragma: no-cache
< server: Apache
< set-cookie: owncloud-selector=oc10; Path=/
< set-cookie: oc2kwg7gksih=d0bhgoidnnop4oltpfaqclrn5t; path=/; secure; HttpOnly; SameSite=Strict
< set-cookie: oc_sessionPassphrase=Oh4Z5ou%2BFTGJaekrelKbZ%2FLwN7oWWHSd1MPNXY46ff58I%2F7UGgISrtF3P5zu7N2E4xqJA%2FR1ynRprV6KUaNpzoiJJuZr%2By9%2F3j0uGwE83y%2Bw3HsVyp9%2B%2B03SL%2BWuGWqd; path=/; secure; HttpOnly; SameSite=Strict
< set-cookie: oc2kwg7gksih=vppua4rjlcddv44bf170o1qokk; path=/; secure; HttpOnly; SameSite=Strict
< set-cookie: cookie_test=test; expires=Mon, 22-Nov-2021 07:48:59 GMT; Max-Age=3600
< set-cookie: oc_username=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; secure; HttpOnly
< set-cookie: oc_token=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; secure; HttpOnly
< set-cookie: oc_remember_login=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; secure; HttpOnly
< set-cookie: oc_username=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure; HttpOnly
< set-cookie: oc_token=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure; HttpOnly
< set-cookie: oc_remember_login=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; secure; HttpOnly
< set-cookie: oc2kwg7gksih=crmjfn3glrqleae9i533f041nt; path=/; secure; HttpOnly; SameSite=Strict
< set-cookie: oc2kwg7gksih=7q5opkk6a2juu99fa190uvhm71; path=/; secure; HttpOnly; SameSite=Strict
* Authentication problem. Ignoring this.
< www-authenticate: Basic realm="Authorisation Required"
< x-content-type-options: nosniff
< x-download-options: noopen
< x-frame-options: SAMEORIGIN
< x-permitted-cross-domain-policies: none
< x-robots-tag: none
< x-xss-protection: 0
< content-length: 153
< 
{ [5 bytes data]
100   243  100   153  100    90    529    311 --:--:-- --:--:-- --:--:--   840
* Connection #0 to host cloud.owncloud.test left intact
<?xml version="1.0"?>
<ocs>
  <meta>
    <status>failure</status>
    <statuscode>997</statuscode>
    <message>Unauthorised</message>
  </meta>
  <data/>
</ocs>

But I can do PROPFIND and other actions such as creating files with user einstein but not with admin.

Q: How can I create other users? (needed for tests) Q: Am I missing out on something or some configuration?

CC @wkloucek

saw-jan commented 2 years ago

Also, I have encountered an issue that I can create a file with owncloud-selector=oc10 but not with owncloud-selector=ocis.

With oc10

curl 'https://cloud.owncloud.test/remote.php/webdav/textfile.txt' -XPUT \
-b 'owncloud-selector=oc10;path=/;' -u marie:radioactivity -vk | xmllint --format -

response:

< HTTP/2 201 
< cache-control: no-store, no-cache, must-revalidate
< content-security-policy: default-src 'none';
< content-type: text/html; charset=UTF-8
< date: Mon, 22 Nov 2021 06:57:27 GMT
< etag: "e64130e817dcafaf1e4c5dabedc37ece"
< expires: Thu, 19 Nov 1981 08:52:00 GMT
< oc-etag: "e64130e817dcafaf1e4c5dabedc37ece"
< oc-fileid: 00002845oc2kwg7gksih
< pragma: no-cache
< server: Apache
< set-cookie: owncloud-selector=oc10; Path=/
< set-cookie: oc2kwg7gksih=3u4a1fkmguoedf3ant0skoecra; path=/; secure; HttpOnly; SameSite=Strict
< set-cookie: oc_sessionPassphrase=AKSn%2F4TWQV4nF2mUGElQ923SOpEF4B3t86Q26LePzkSSC56BjBt7hHPsCZ5k6tNNF2CXyKLhV7uPCtXTRtO0V7GKZuWhx64DfJTdf8uEMkzoTZqA6%2FlWudPUDTfpKVAx; path=/; secure; HttpOnly; SameSite=Strict
< set-cookie: oc2kwg7gksih=67bsjeha28t9n3fu0bpa03v39d; path=/; secure; HttpOnly; SameSite=Strict
< set-cookie: cookie_test=test; expires=Mon, 22-Nov-2021 07:57:27 GMT; Max-Age=3600
< x-content-type-options: nosniff
< x-download-options: noopen
< x-frame-options: SAMEORIGIN
< x-permitted-cross-domain-policies: none
< x-robots-tag: none
< x-xss-protection: 0
< content-length: 0

With ocis

curl 'https://cloud.owncloud.test/remote.php/webdav/textfile.txt' -XPUT \
-b 'owncloud-selector=ocis;path=/;' -u marie:radioactivity -vk | xmllint --format -

response:

< HTTP/2 500 
< access-control-allow-origin: *
< content-security-policy: default-src 'none';
< date: Mon, 22 Nov 2021 06:58:19 GMT
< set-cookie: owncloud-selector=ocis; Path=/
< vary: Origin
< x-content-type-options: nosniff
< x-download-options: noopen
< x-frame-options: SAMEORIGIN
< x-permitted-cross-domain-policies: none
< x-robots-tag: none
< x-xss-protection: 1; mode=block
< content-length: 0

This is the log from ocis service:

ocis_1          | {"level":"error","service":"storage","error":"unable to register services: rgrpc: grpc service storageprovider could not be started,: error connecting to the database: dial tcp 172.22.0.4:3306: connect: connection refused","time":"2021-11-22T07:45:00Z","message":"error starting the grpc server"}
ocis_1          | 2021/11/22 07:45:10 dialing:dial tcp 127.0.0.1:9250: connect: connection refused
ocis_1          | process idp terminatedprocess glauth terminatedprocess accounts terminatedprocess proxy terminated2021-11-22 07:45:31  file=v4@v4.4.0/service.go:206 level=info Starting [service] com.owncloud.web.proxy
ocis_1          | 2021-11-22 07:45:31  file=v4@v4.0.0-20211028090348-ed690ed838cc/http.go:255 level=info Listening on [::]:9200
ocis_1          | 2021-11-22 07:45:31  file=v4@v4.0.0-20211028090348-ed690ed838cc/http.go:169 level=info Registering node: com.owncloud.web.proxy-c0788ec0-92c2-40f8-a798-934bdfd50ed4
ocis_1          | {"level":"error","error":"Error 1366: Incorrect integer value: '' for column `owncloud`.`oc_filecache`.`storage_mtime` at row 1","time":"2021-11-22T07:46:09Z","message":"could not store filecache item"}
ocis_1          | {"level":"error","service":"storage","pkg":"rgrpc","error":"Error 1366: Incorrect integer value: '' for column `owncloud`.`oc_filecache`.`storage_mtime` at row 1","time":"2021-11-22T07:46:09Z","caller":"/home/sawjan/go/pkg/mod/github.com/cs3org/reva@v1.15.1-0.20211110160042-a6eb9d5a7cd2/internal/grpc/services/storageprovider/storageprovider.go:361","message":"error getting upload id: path:\"/home/textfile.txt\" "}
ocis_1          | {"level":"error","service":"storage","pkg":"rhttp","path":"/home/textfile.txt","status":{"code":15,"message":"error getting upload id: path:\"/home/textfile.txt\" ","trace":"00000000000000000000000000000000"},"time":"2021-11-22T07:46:09Z","message":"grpc request failed"}
ocis_1          | {"level":"error","service":"storage","pkg":"rhttp","host":"127.0.0.1","method":"PUT","uri":"/remote.php/webdav/textfile.txt","url":"/remote.php/webdav/textfile.txt","proto":"HTTP/1.1","status":500,"size":0,"start":"22/Nov/2021:07:46:09 +0000","end":"22/Nov/2021:07:46:09 +0000","time_ns":31611143,"time":"2021-11-22T07:46:09Z","message":"http"}
wkloucek commented 2 years ago

Q: How can I create other users? (needed for tests)

Users in that deployment example are coming from the LDAP server. The initial users are imported from this file: https://github.com/owncloud/ocis/blob/master/deployments/examples/oc10_ocis_parallel/config/ldap/ldif/20_users.ldif.

You can use the LDAP user cn=admin,dc=owncloud,dc=com with password admin to create new users.

Q: Am I missing out on something or some configuration?

Please also have a look at https://owncloud.dev/ocis/deployment/oc10_ocis_parallel/, since it describes the setup. If you got more questions, I'm also happy to add it to the docs.

But I can do PROPFIND and other actions such as creating files with user einstein but not with admin.

You're right, admin does not exist in the user base. I will add them for consistency reasons (the deployment examples claim, that one can log in with the demo users)

Also, I have encountered an issue that I can create a file with owncloud-selector=oc10 but not with owncloud-selector=ocis

Looks like the mtime is missing, but I will need to look into that. (Providing the mtime -H 'x-oc-mtime: 1636993082.336' didn't help). oC Web is able to upload files though...

wkloucek commented 2 years ago

Sharing is not yet implemented (in a way that it is usable from both oC10 and oCIS). Stay tuned for https://github.com/owncloud/ocis/pull/2232

phil-davis commented 2 years ago

OK - so we can start with basic single-user scenarios - Alice uploads a file on one system and can download the file from the other system...

wkloucek commented 2 years ago

Update to the mtime problems encountered in https://github.com/owncloud/ocis/issues/2387#issuecomment-975184346:

Uploading a file in general works:

curl -k 'https://cloud.owncloud.test/remote.php/webdav/Nyan-Cat-GIF-source.gif' -X 'PUT' -H 'content-length: 75416' -H 'x-oc-mtime: 1638179645' -H 'cookie: owncloud-selector=ocis' -u einstein:relativity -v -T ~/Desktop/oCIS-Team/Nyan-Cat-GIF-source.gif

-> gives me HTTP 201 / 204

But uploading an empty file:

curl -k 'https://cloud.owncloud.test/remote.php/webdav/zero' -X 'PUT' -H 'content-length: 0' -H 'x-oc-mtime: 1638179645' -H 'cookie: owncloud-selector=ocis' -u einstein:relativity -v

-> gives me a HTTP 500

I think this is related to https://github.com/owncloud/ocis/issues/2609

saw-jan commented 2 years ago

I have created this draft PR to automate tests for parallel deployment, but I could not get the tests to pass.

Issue: The requested user could not be found while getting user Alice using ocis:9200 with owncloud-selector=oc10;path=/;. CI job: https://drone.owncloud.com/owncloud/ocis/8170/1/19 ocis log: https://drone.owncloud.com/owncloud/ocis/8170/1/15 oc10 log: https://drone.owncloud.com/owncloud/ocis/8170/1/17

Checked:

Service Stack:

CC @wkloucek @phil-davis

saw-jan commented 2 years ago

I have created this draft PR to automate tests for parallel deployment, but I could not get the tests to pass.

Issue: The requested user could not be found while getting user Alice using ocis:9200 with owncloud-selector=oc10;path=/;. CI job: https://drone.owncloud.com/owncloud/ocis/8170/1/19 ocis log: https://drone.owncloud.com/owncloud/ocis/8170/1/15 oc10 log: https://drone.owncloud.com/owncloud/ocis/8170/1/17

Checked:

  • [x] user Alice exists in ldap
  • [x] can GET user Alice directly from oc10:8080 server

Service Stack:

  • oc10 (pulled from owncloud/latest)
  • mariadb for oc10
  • ocis (build from the branch)
  • ldap
  • keycloak
  • postgres for keycloak
  • redis

CC @wkloucek @phil-davis

Now, the tests are passing. The real reason for this issue was: PROXY_CONFIG_FILE config option no longer exists. So the configs must be in one of these three locations https://owncloud.dev/ocis/config/#expected-loading-locations Thanks to @wkloucek

individual-it commented 2 years ago

@phil-davis @saw-jan can we close this?

phil-davis commented 2 years ago

@phil-davis @saw-jan can we close this?

There are only a few scenarios running. We need to review the scenarios and decide what "user workflows" to test, to verify that various changes done on the oC10 side are visible on the oCIS and vice-versa.

wkloucek commented 2 years ago

@phil-davis @saw-jan can we close this?

There are only a few scenarios running. We need to review the scenarios and decide what "user workflows" to test, to verify that various changes done on the oC10 side are visible on the oCIS and vice-versa.

We're still waiting for sharing to work. That will definitively enable some more tests. Let's keep this ticket open until then.

individual-it commented 2 years ago

I've created a new issue to run ALL tests with the parallel setup https://github.com/owncloud/ocis/issues/2932

wkloucek commented 2 years ago

@saw-jan @kiranparajuli589 we just merged https://github.com/owncloud/ocis/pull/3052, which brings back the parallel deployment on master and also has now a working sharing integration between oCIS and oC10. There is already one sharing test, which is currently failing. Could you please have a look at this single test and add more tests? :-)

saw-jan commented 2 years ago

Ticket for investigating sharing test fail https://github.com/owncloud/ocis/issues/3069

individual-it commented 2 years ago

closing this, remaining work will be tracked in follow up issues