postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.88k stars 1.16k forks source link

Newman produces no output in cron, produces output from cli #1452

Closed JaimeLynSchatz closed 6 years ago

JaimeLynSchatz commented 6 years ago

The following line in a bash script: newman run /home/teamname/tests/vm_smoke_tests.json -e /home/teamname/tests/devint_env.json --reporters html,cli > results.txt

Produces output when run from the command line, but does not produce output when run in cron.

The bash script also contains other output such as: echo "I have just run the Newman tests and want to write this to a file" > /home/teamname/results/$today/text-$ts.html and post_to_slack "Newman Hates Cron" "INFO" "All tests ran but I can't get Newman to generate the proof from crontab. http://hostname/$today/text-$ts.html" "PASSING"

but Newman will not produce results when run from cron. Newman will only generate (or modify) files when run directly from the command line.

This script used to work when run from cron. (Last known success was in the fall.)

  1. Newman Version (can be found via newman -v): 3.9.3

  2. OS details (type, version, and architecture): Ubuntu 17.10 GNU/Linux 4.13.0-32-generic x86_64, Bash

  3. Are you using Newman as a library, or via the CLI? CLI

  4. Did you encounter this recently, or has this bug always been there: Newly encountered (last successful attempt was in the fall, haven't tried since Februrary when observed failing)

  5. Expected behaviour: Line in cron (crontab -e) * * * * * /home/teamname/tests/run_and_post_smoke_test_results.sh > new_cron_errors.txt Newman command should generate the file in cron the same as it does when run in the command line

  6. Command / script used to run Newman: newman run /home/teamname/tests/vm_smoke_tests.json -e /home/teamname/tests/devint_env.json --reporters html,cli > results.txt

  7. Sample collection, and auxiliary files (minus the sensitive details): I've attached a sample collection, environment, etc and all applicable results. The collection is taken from Postman -- it's not the one I'm running for my project. Same results. (Newman produces output from command line, not from cron.) newman_example.zip

  8. Screenshots (if applicable): n/a

rvolz commented 6 years ago

May I add, that the same behaviour shows up when run in Gitlab (enterprise instance)? My clients run newman as a library (see below), which works fine locally. When run on Gitlab the output via util.log appears, the jobs exits with success, but newman doesn't seem to work -- no output for cli, html or json. The job's execution time is far too short, so I guess that the collection is not run.

Differences to the settings mentioned above:

  1. OS details (type, version, and architecture): Alpine 3.6, docker image https://github.com/nodejs/docker-node/blob/a79c59b7b3e81c1fedb419531fce01f1a4726a39/9/alpine/Dockerfile
  2. Are you using Newman as a library, or via the CLI? Library
  3. Did you encounter this recently, or has this bug always been there: Newly encountered, new project
  4. Expected behaviour: test reports via cli, html and json
  5. Command / script used to run Newman: NodeJS 9
    
    const util = require('util');
    const newman = require('newman');

util.log('API test starting ...');

const options = { collection: 'https://api.getpostman.com/collections/...', environment: 'https://api.getpostman.com/environments/...', iterationCount: 1, iterationData: 'data/staging-company-313353.json', reporters: ['cli','html','json'] };

newman.run(options);

util.log('API test ended.');

rvolz commented 6 years ago

Addendum: using the newman cli utility (installed locally, via ./node_modules/.bin/newman) in the configuration mentioned above works. The output appears in Gitlab/Docker. However, using the newman library would be much easier for us.

dhrubesh commented 6 years ago

This issue is someway similar to #97. @JaimeLynSchatz take a look at this. Also, take a look at this, you might find it useful.

JaimeLynSchatz commented 6 years ago

97 is similar but not the same issue. Was this closed because the issue was resolved? Or closed because this functionality is no longer supported?