The current implementation sets up the test environment (specifically the configuration files to be used during tests) as part of the Travis CI environment setup. These tests doesn't work when call on a differently setup test environment (e.g. During the test stage of a CPAN based installation
An alternative workaround/shortcut may be to define environment dependent tests as optional.