search

xenon-middleware / xenon

A middleware abstraction library that provides a simple programming interface to various compute and storage resources.
http://xenon-middleware.github.io/xenon/
Apache License 2.0
34 stars 17 forks source link

Webdav Adaptor fails to connect to /webdav/workingdir #622

Closed bpmweel closed 6 years ago

bpmweel commented 6 years ago

I have a webdav server running at localhost/webdav/, but the webdav connector thinks that /webdav is the working directory.

jmaassen commented 6 years ago

From the definition of URI, the "webdav" part is part of the path. So if you are using:

http://localhost:8080/webdav/workdir

Then the /webdav/workdir part of the URI is the path.

The webdav adaptor will assume the whole thing to be the working directory. This does not necessarily mean all subpaths are accessible (like /webdav). Note that this is also true for other adaptors like FTP, SFTP or even local)

Can you specify a bit more what problem you ran into, so I can try to replicate it in a test?

bpmweel commented 6 years ago

The problem lies in the fact that the webdav connector tries to connect to the base url: http://localhost:8080/ however this is not a webdav endpoint (that is actually running the frontend of my application) so it fails to connect. The webdav is running at http://localhost:8080/webdav/, however I supply a subdir within this as a working dir to the adaptor, for instance http://localhost:8080/webdav/input.

sverhoeven commented 6 years ago

The xenon webdav Docker image also does not have / as an webdav endpoint. Not working:

xenon filesystem webdav --location http://172.17.0.2/ list  .

Working:

xenon filesystem webdav --location http://172.17.0.2/downloads list .
xenon filesystem webdav --location http://172.17.0.2/ list downloads/filesystem-test-fixture
xenon filesystem webdav --location http://172.17.0.2/downloads list filesystem-test-fixture

What connection/command is failing for you?

bpmweel commented 6 years ago

After testing a bit more I found that without credentials this works just fine indeed, however when I put basic authentication on my webdav server (I tested with a http webdav docker) I get the following error:

nl.esciencecenter.xenon.XenonException: webdav adaptor: Failed to check existence of directory: /webdav/input/echo.cwl
    at nl.esciencecenter.xenon.adaptors.filesystems.webdav.WebdavFileSystem.exists(WebdavFileSystem.java:248)
    at nl.esciencecenter.xenon.filesystems.FileSystem.assertPathExists(FileSystem.java:1622)
    at nl.esciencecenter.xenon.filesystems.FileSystem.assertFileExists(FileSystem.java:1674)
    at nl.esciencecenter.xenon.adaptors.filesystems.webdav.WebdavFileSystem.readFromFile(WebdavFileSystem.java:256)
    at nl.esciencecenter.computeservice.rest.service.staging.StagingManifestFactory.createStagingManifest(StagingManifestFactory.java:46)
    at nl.esciencecenter.computeservice.rest.service.tasks.CwlStageInTask.run(CwlStageInTask.java:59)
    at nl.esciencecenter.computeservice.rest.service.tasks.XenonMonitoringTask.startStageInForSubmittedJobs(XenonMonitoringTask.java:327)
    at nl.esciencecenter.computeservice.rest.service.tasks.XenonMonitoringTask.update(XenonMonitoringTask.java:117)
    at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
bpmweel commented 6 years ago

Looking at it again this may be an issue of Sardine as it issues a HEAD request to see if the url exists, but perhaps does not supply the Authorization header.

I know the OPTIONS http request should be possible without authorization header (to allow for preflight requests), do you know if HEAD requires the same?

edit: Adding the following to the apache config makes it work:

<LimitExcept OPTIONS HEAD PROPFIND>
    Require valid-user
</LimitExcept>

But i'm not sure if I feel comfortable allowing non-valid users to get all the metadata for a file.

sverhoeven commented 6 years ago

Always send credentials in 6130e6bfceb305ba5ab1c75413e5e9695860668c

bpmweel commented 6 years ago

Tried the new build, but it does not seem to fix the issue.

The webdav server returns a 401 Unauthorized header, with the correct WWW-Authenticate header set.

sverhoeven commented 6 years ago

The https://www.w3.org/TR/cors/#cross-origin-request-with-preflight-0 says that a client should exclude the user credentials headers.

bpmweel commented 6 years ago

We finally figured out that the problem was with a base64 encoding I used somewhere in my application that meant I supplied the wrong password to the fileadaptor.

So in the end there was no bug with xenon or sardine, but with my config.

Sorry for the hassle :)

jmaassen commented 6 years ago

No worries.