error "Invalid path part "" on uploaded coverage

[zanona]

Hey, guys I am having issues while uploading coverage locally. After cc-test-reporter after-build the coverage is successfully uploaded but I do get an error under code climate mentioning the coverage is invalid.

error: (formatted for readability)

{
  "message": "Invalid path part \"\"",
  "file_document": "{
    \"_id\"=>BSON::ObjectId('5bec2222e586504995006bd8'), 
    \"type\"=>\"test_file_reports\",
    \"blob_id\"=>\"8f2152d437a409b348c438b3d72059e8bbba1262\", 
    \"coverage\"=>\"[1,1]\",
    \"covered_percent\"=>100, 
    \"covered_strength\"=>1, 
    \"line_counts\"=>   {\"missed\"=>0, \"covered\"=>2, \"total\"=>2},
    \"path\"=>\"/mnt/c/Users/Marcus/Developer/project/lib/utils.ts\",
    \"test_report_id\"=>BSON::ObjectId('5bec22210fef2e304f004f73')
   }"
}

my test command looks like:

npx nyc ava-ts
npx nyc report --reporter lcov
cc-test-reporter after-build

any ideas?

[wfleming]

Hi @zanona,

The error message here is a bit confusing, and I'm actually surprised the test-reporter client didn't catch this locally. It looks to me like the problem is that the path being sent up is an absolute path, but our test coverage code always expects paths to be relative to the root of the repository (e.g. lib/utils.ts) in this case. The string escaping makes the error a bit hard to read, but it's actually saying Invalid path part "", because the validation code is splitting the path into segments and validating each segment, and doing that on a root path results in an empty segment at the beginning.

I don't know what details of your environment led to the coverage being generated in that way. It looks like you're running the test reporter from a linux VM on windows or perhaps using the Bash subsystem for Windows? What's the working directory when you're running the tests and the test-reporter?

[zanona]

Thanks for following up, Will. So, my environment is as you mentioned indeed. I am running WSL as Ubuntu 16.04.5 LTS.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial

$ bash --version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. 

$ node -v
v9.11.2

I hope this can be helpful. Please let me know I can provide you with any other info.

Thanks

[zanona]

Just some additional debug info:

$: cc-test-reporter after-build -h

 Locate, parse, and re-format supported coverage sources. Upload pre-formatted coverage payloads to Code Climate servers.

Usage:
  cc-test-reporter after-build [flags]

Flags:
  -s, --batch-size int               batch size for source files (default 500)
  -e, --coverage-endpoint string     endpoint to upload coverage information to (default "https://api.codeclimate.com/v1/test_reports")
  -t, --coverage-input-type string   type of input source to use [clover, cobertura, coverage.py, excoveralls, gcov, gocov, jacoco, lcov, simplecov]
      --exit-code int                exit code of the test run
  -r, --id string                    reporter identifier (default "8a3d27964529d7cfdcfc57723b022b43539a7490a170cfc5f51fc5208e67f90e")
      --insecure                     send coverage insecurely (without HTTPS)
  -p, --prefix string                the root directory where the coverage analysis was performed
                                     (default "/home/zanona/Developer/project")

Global Flags:
  -d, --debug   run in debug mode

As you can see the --prefix options default already uses an absolute path to my repo rather than relative, and even with calling cc-test-reporter after-build --prefix ./ or cc-test-reporter after-build --prefix /home/zanona/Developer/project or generates the same invalid report on code climate where the path displayed in the error is /mnt/c/Users/Marcus/Developer/project/lib/utils.ts

Perhaps there's something to do with WSL since /mtn/c/Users/Marcus/ is the real path of files however $HOME is set to /home/zanona so I am not quite sure how it's getting the real path? I don't believe it should be the case.

[wfleming]

The default --prefix is the working directory where you're running the command from. Why that is different than the path Istanbul seemed to decided to use is odd, and I don't know why it would do that.

I'd suggest trying to understand why the istanbul report is using absolute paths at all, and seeing if you can configure it to output relative paths itself. That should fix the usage with Code Climate, and I is also likely to save you some headaches in the future, since the report with absolute paths is likely to run into similar issues if you ever want to use it with any other 3rd party tools or services.

That said, if you just want to move forward with getting the report to work on Code Climate to work, you can use the --prefix option to do that. That option is used to strip excessive pieces off of path names off of some coverage reports in cases like this, but the mismatch between your actual working path and the path that Istanbul seems to be seeing is the problem. If you used --prefix /mnt/c/Users/Marcus/Developer/project/, it should strip the paths to be correctly relative to the project root.

[zanona]

Thanks again for having a look at this, Will. So, I tried to use the prefix option as you have suggested and it worked - thanks for that 👍

After analysing further, I could spot the difference between nyc when analysing the cwd and cc-test-reporter it may have something to do with retrieving the real path by reading the link to the directory rather than the system pwd?

have a look:

pwd

$ pwd
/home/zanona/Developer/project

readlink

$ readlink -f .
/mnt/c/Users/Marcus/Developer/project

I believe that the second example routes the original path of the file instead of the alias that is created. I believe that perhaps, regardless of the configuration, readlink is something that would always work, whether pwd may have its drawbacks?

As a workaround, I am now calling cc-test-reporter after-build --prefix $(readlink -f .)

What do you think? And, do you believe that perhaps using something like readlink to retrieve the current directory could perhaps work as a replacement for the actual way of doing it?

Thanks again.

[wfleming]

Ah. Symlinks. Good call.

I don't think we're going to want to change the determination of "default" --prefix on our end, at least not without more evidence that a lot more coverage tools are fully canonicalizing paths by following symlinks, as Istanbul seems to be doing. This is one of those situations where I fully expect some tools use the canonicalized path, and some coverage tools use the "apparent" (symlinked) path, so if we changed this to fix this case I fully expect it would break some other configurations, and I think either choice is a defensible one, so I'm inclined to not change the existing behavior in our test-reporter.

That said I would consider a change on our end for the test reporter to validate file paths aren't still absolute after prefix operations have occurred, which would at least have caught this problem earlier and could have provided a more logical error message.

I'm going to go ahead and close this since we discovered what was affecting your configuration and found a config that works for you, but feel free to still comment here if you have further questions or want to discuss more.

[zanona]

Thanks, Will That makes sense. I'm glad we have the --prefix option which can be overridden. All the best