Uploaded file has 0 B size

[intest]

@ptrthomas It's a continuation of this issue https://github.com/intuit/karate/issues/1605, you asked for a prove that it works for me with P*stman. So the cURL is:

curl --location --request POST 'https://my-website/importjobs?storageId=9bb306d1221c39b6847c7393ad5ef0b0&externalIdColumn=Customer%20number' \
--header 'Authorization: Basic Zm_some_base64_string_here_MSe=' \
--form 'file=@"/C:/path/to/my/features/some_subdirectory/fink.xlsx"'

And here is the file on S3:

As you can see - it works. And it worked the same with K0.9.6. And I would be grateful if you didn't offend me ;) writing "(...) it does sound like tools like P0stman are simpler and possibly a better fit for you". I actually migrated hundrets of tests from JMeter to Karate and it's a cool tool, but sometimes I come across such traps and I think is quite normal to ask for help or raise a bug :) (I use P*stman from time to time just for debugging) Peace! :)

[intest]

Maybe this will help - content-length: 0

17:27:34.104 request:
3 > POST https://my-website/importjobs?storageId=4c940863202e39ca82d64a111edf457c&externalIdColumn=Customer+number
3 > Content-Type: multipart/form-data; boundary=5e91265b34c1ca93
3 > Accept: application/json
3 > X-Credential-Username: xxxxxxxxxx
3 > Authorization: Basic zzzzzzzzzzzzzz
3 > Content-Length: 187
3 > Host: api.my-website
3 > Connection: Keep-Alive
3 > User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_271)
3 > Accept-Encoding: gzip,deflate

content-disposition: form-data; name="file"; filename="fink.xlsx"
content-type: application/octet-stream; charset=UTF-8
content-length: 0
Completed: true
IsInMemory: true

[ptrthomas]

just for the record, this doesn't prove anything unless I can run code locally and see for myself that it works :)

but thanks for the cURL command. it just makes sure that we are dealing with a normal multipart and the field-name is file. and now we can make some progress !

we are going to try to be constructive here and we can turn this thread into a definitive resource for troubleshooting karate file-upload. we will create a server that handles a curl command to upload a file and "prove" it works. we'll also print as much diagnostic information possible.

and then - in theory, if you can point your existing test to this mock server, you should be able to spot what your broken test is doing differently. so here goes !

P.S. also note that https://httpbin.org can be used to quickly check what the structure of the multipart request being sent is: https://stackoverflow.com/a/78547136/143475

step 0

make sure java is installed and in your PATH etc

step 1

download the latest karate binary from the releases page, right now it is karate-1.1.0.RC3.jar

step 2

rename it to karate.jar for convenience, and move it to some folder you choose

step 3

create the mock: mock.feature. note that we assume that file is the name of the "part" in your file-upload, the name in the --form part of your cURL command

here it is below:

Feature:

Scenario: pathMatches('/upload')
* print 'method:', requestMethod
* print 'headers:', requestHeaders
* print 'requestParts:', requestParts
* def file = requestParts['file'][0]
* def length = file.value.length
* print 'bytes received:', length
* def response = { bytes: '#(length)' }

step 4

we need a file to upload. so have a test.xslx in the same folder

so this is what things should look like:

step 5

start the server ! you can add -p 8080 to change the port but it will default to 8080

java -jar karate.jar -m mock.feature 

you should see the server start on the console. since this terminal is running it will be "blocked", so for the next step you need to open a new one, and you may want to use the same folder for convenience

step 6

fire your curl command in a new terminal ! note how I am using your curl command "as is" without changes to "prove" things. use the full, absolute path of the file if needed. here please refer to the cURL documentation if in doubt.

curl --location --request POST 'http://localhost:8080/upload' \
--header 'Authorization: Basic Zm_some_base64_string_here_MSe=' \
--form 'file=@"test.xlsx"' 

and you should see this result ! which confirms that the file went to server, bytes and all. this is what the last line of mock.feature does of course.

{"bytes":8638}

what you see on the server log also will be quite informative:

21:16:41.451 [armeria-common-worker-nio-2-1]  DEBUG com.intuit.karate - scenario matched at line 3: pathMatches('/upload')
21:16:41.470 [armeria-common-worker-nio-2-1]  INFO  com.intuit.karate - [print] method: POST 
21:16:41.478 [armeria-common-worker-nio-2-1]  INFO  com.intuit.karate - [print] headers: {
  "authorization": [
    "Basic Zm_some_base64_string_here_MSe="
  ],
  "content-length": [
    "8839"
  ],
  "host": [
    "localhost:8080"
  ],
  "content-type": [
    "multipart/form-data; boundary=------------------------c3a7ac1cb57e4978"
  ],
  "user-agent": [
    "curl/7.64.1"
  ],
  "accept": [
    "*/*"
  ]
}

21:16:41.502 [armeria-common-worker-nio-2-1]  INFO  com.intuit.karate - [print] requestParts: {
  "file": [
    {
      "charset": "UTF-8",
      "filename": "test.xlsx",
      "transferEncoding": "7bit",
      "name": "file",
      "contentType": "application/octet-stream",
      "value": "[B@39854b0d"
    }
  ]
}

21:16:41.515 [armeria-common-worker-nio-2-1]  INFO  com.intuit.karate - [print] bytes received: 8638 

step 7

see if a karate test has the same effect. here is test.feature, in the same folder of course. refer the docs for multipart file if needed. note that for normal "string" fields, form field can be mixed with multipart file, but multipart file gives you full control over the individual "part" Content-Type etc.

Feature:

Scenario:
* url 'http://localhost:8080/upload'
* multipart file file = { read: 'test.xlsx', filename: 'my-file.xlsx', contentType: 'text/csv' }
* method post
* status 200
* match response == { bytes: '#number' }

step 8

run the karate test !

java -jar karate.jar test.feature 

voila !

so now point your test to http://localhost:8080/upload and see what happens.

happy testing :)

[intest]

Thanks @ptrthomas for providing above instruction. I followed it and I got the same result as you. Seems like locally the upload works. So.. I don't have a clue why it doesn't work in my tests. Really no idea. It looks like I need to live with that :D

[ptrthomas]

I'm pretty sure you are reading the file wrong which others also tried telling you on stack overflow. but yes, you will have to live with 0.9.X or perhaps its time to switch to P0stman :) closing this ticket

[intest]

What do you mean that "I'm reading the file wrong"? I do it the same way in 1.X.X like in 0.9.6.

PS. A note about Postman is not necessary here.

[ptrthomas]

@intest it is possible we fixed bugs in the file-reading that changed behavior - and you were depending on the buggy behavior :)

ok you can use JMeter then

[intest]

OK, so how my Karate request should look like to make it work?

PS. I don't get your comments about p..man or jmeter.. I never said that they are better tools or whatever. I really like Karate and I just switched to it from jmeter. What is your point while commenting like that? I like your tool and I would expect that you want people to use your tool. Only that I mentioned those 2 tools doesn't mean I consider them as better than Karate. I just wanted to mention them to clarify that it was not an issue with server or anything else.

[ptrthomas]

OK, so how my Karate request should look like to make it work?

@intest I have to see EXACTLY how your project is set up to determine that - which apparently is not possible ! and a working example is provided above, take it or leave it.

I'm locking this thread now. questions can be asked on stack overflow.

[ptrthomas]

@intest we got a bug report today that most likely is the same problem you faced, see if it helps: https://github.com/intuit/karate/issues/1650

[meridicolos]

Hello! It helped a lot! Thank yo so much, Peter!