put(String, InputStream) puts empty File

[sunchezz89]

• Sardine version: 5.8
• Java 1.8
• in a Spring Boot 2.1.2 project (relevant for any dependency overwrites?)
• Tomcat9.0.14

I have seen #163 and #84. Both seem not to apply fully (but mixed).

I can connect to my Server (Nextcloud) via Sardine. Also the put has an effect to the server, as it places the expected File into the correct directory. But as stated in the title, the file is empty (0byte). Also there is no client Exception.

working with put(String, byte[]) works as expected and the File on the Server is completly working. But I don't want to be limited to maximum fileSize of byte[Integer.MAX-5].

Following Code is working which means, it's not about ssl, or any other connection related things

File file = new File(filePath);
byte[] data = new byte[(int) file.length()];
FileInputStream stream = new FileInputStream(file);
int readedBytes = stream.read(data);
if (readedBytes == file.length()) {
   client.put(remotePath, data);
}

This Code is NOT working

File file = new File(filePath);
client.put(remotePath, new FileInputStream(file));

On the Nextcloud Server i get following Log Entry:

"Exception":"Sabre\\DAV\\Exception\\BadRequest","Message":"expected filesize 240823 got 0"

To me it looks like something is going wrong when copying the inputstream content to the actual http out stream. As the headers seem to havae the correct byte size as we see in the Log of the server, but the actual content is zero.

Please investigate on this. If you need anything else from me, let me know.

[lookfirst]

Correct, you should be streaming the file, otherwise you're loading all the bytes into the heap and if your JVM doesn't have enough memory, you run out.

The code that 'isn't working', looks fine. My guess is that you're getting an exception on the client side. Does it complete successfully? Any exceptions there? I see you're running through Tomcat, are you sure there is enough OS permissions for Tomcat to read the file?

[sunchezz89]

Thats why i also added the working code. If i 'just' use the other Sardine method its working correctly, which means, no permission problems on client, or server side. And as i said, there is an exception on the server side, but it is only complaining about an empty upload, nothing else.

[lookfirst]

Without some actual code and full logs, I am just guessing at the issue. My suggestion is to really enable as much debugging as you can and look at everything.

Since this isn't likely an issue with Sardine itself, I'm going to close this issue.

[sunchezz89]

I am very sure that this is not on the server side, I have looked into every possible log file, without results.

Which options do you see, if the one method with an byte array is working correctly? And if the second approach using the stream is also accepted on the server side, with the only difference that the resulting file has no size?

What can the difference be? Isn't the resulting tcp package the same? I would guess so?

So assuming that... The issue must lie in the sardine library or in one of the dependencies, like apaches httpclient?

I will later today try to capture the tcp packages and will have a look how they differ.

The last possible option I can imagine, is that nextcloud can't handle a tcp stream and does not reporting it the logs?

I will post results.

[lookfirst]

Take a look in SardineImpl.java, you'll see that all the put() methods just call this.put().

[lookfirst]

You have the source code, run it in a debugger and step through it.

Turn up logging too.

[sunchezz89]

I now saw that the issue indeed is not within Sardine itself.

But i would like to ask you for help, because i do not have another webDAV server to test against, and i still can not find any issue on my server side and i would like to help to solve this (no matter which part has the issue)

Therefore i setup a little test project where you can just insert server config. https://github.com/sunchezz89/StreamWebDAVIssue Also: There are two public testserver listed in the Class WebdavConnector which didn't worked.

For more details: I could track it down to the class org.apache.http.entity.InputStreamEntity. If the initial length of the entity is -1, the writeTo method behaves different. I debugged it multiple times and watched, how the content was written into the outstream in line 136.

// consume util EOF
while ((l = instream.read(buffer)) != -1) {
     outstream.write(buffer, 0, l);
}

i had the feeling, that it sometimes got interrupted and proceeded with the finally block. But this would also not explain a 0byte result on server, right?

The interesting part here is, using the put(targetDir, stream, null, false, file.length());, results in the InputStreamEntity.writeTo method in the else-block, which ALWAYS succeeds with a correct filesize on the Server. But to me it doesnt look like there is any difference in the resulting outputstream or the content which should be send to the server?!

Can you reproduce this somehow?

[lookfirst]

Impossible for me to reproduce without your server, but I also don't have much time to spend on this.

Turn on trace debugging at the org.apache.http level? Turn on low level logging on the server?

It also does sound like you have a workaround... go with that.

[a-schild]

We see this problem too in our nextcloud api

https://github.com/a-schild/nextcloud-java-api/issues/20

[a-schild]

Probably this problem

https://trac.cyberduck.io/wiki/help/en/howto/mount/issues/fastcgi#ZerobytefiletruncateissuewithNextcloudandownClouddeployedwithFastCGI

[kkofler]

The interesting part here is, using the put(targetDir, stream, null, false, file.length());, results in the InputStreamEntity.writeTo method in the else-block, which ALWAYS succeeds with a correct filesize on the Server. But to me it doesnt look like there is any difference in the resulting outputstream or the content which should be send to the server?!

The secret is probably not in InputStreamEntity.writeTo, but that that overload allows Sardine to tell Nextcloud in advance what the file size will be, just like the (much less efficient) byte[] overload.

[kkofler]

I can confirm that using put(targetDir, stream, null, true, file.length()); (I used the default expectContinue = true rather than false as you did, so the only thing I changed is adding the size, the other two added arguments are what the short overload defaults to anyway, I had to add them only because there is no overload taking the size without those two) works around the issue. This is much more efficient than the initially suggested workaround of reading the whole file into a byte[] and uploading that.