`code:` isn't evaluated from the correct directory

[beccasaurus]

Steps to reproduce:

• In a test: - code: print(os.getcwd())

Expected result:

• The chdir: location should be printed

Actual:

• The directory that I invoked sample-tester from is printed out.

If I cd around my system, the value of os.getcwd() changes as I cd where I invoke sample-tester from. This should remain static and should always be the working directory specified in the test's chdir:

---

Use case example (how I ran into this)

• I'm using write_file: to save file_name: ["output.mp3"]
• I need to assert that the file was output and isn't empty
• os.path.exists("output.mp3") returns false (even though output.mp3 was saved OK)
• If I print os.getcwd() is show the directory that I'm currently invoking sample-tester FROM (rather than the chdir: location)

This means that my tests fail unless I cd into a very particular directory (which is already configured as the chdir:)

From texttospeech/ library root, where sample tests are typically run from:

$ sample-tester samples/v1/test/*

RUNNING: Test environment: ""
  RUNNING: Test suite: "Synthesizing Voice Audio"
    FAILED: Test case: "Converting text to synthetic voice audio"
      | 
      | ### Test case SETUP
      | 
      | ### Test case TEST
      | 
      | # Calling: python3 ./v1/tts_synthesize_text.py 
      | # FAILED ASSERTION: output.mp3 file expected but not found
      | 
      | ### Test case TEARDOWN
      | 

Tests failed

Change directory into texttospeech/samples/ (shouldn't need to be in this directory)

$ sample-tester v1/test/*

RUNNING: Test environment: ""
  RUNNING: Test suite: "Synthesizing Voice Audio"
    PASSED: Test case: "Converting text to synthetic voice audio"

Tests passed

---

Implementation thought:

• In Ruby, we evaluate the code inside a block:

• Dir.chdir("some-dir") do
    # code goes here
  end

• Presumably Python has something similar for evaling code in a specific directory 😄

[vchudnov-g]

It sounds like the issue is that, as captured in the doc, chdir changes to the specified directory before executing the sample, not before running the test. chdir is a property of the sample, after all, in the manifest file. So with this behavior, it's expected that printing os.getcwd() would give you the directory from which the sample-tester is run, whereas call() should switch to the chdir directory before calling the sample. (If you had the sample printout os.getcwd(), it should always give the chdir directory if one was provided.)

Sounds like what's needed is a way to have the test plan extract a particular sample's chdir and then cd to it.

[beccasaurus]

+1

Where "have the test plan extract a particular sample's chdir and then cd to it." can use a language built-in to switch the running context safely to that dir (and switch back to the main dir after code evaluation)

[beccasaurus]

If there's no idiomatic block-ish way to eval a bit of code chdir'd, looks like the recommended approach is just to try/finally

program_dir = os.getcwd()
sample_chdir = sample.chdir

try:
  os.chdir(sample_chdir)
  # eval the things
finally:
  os.chdir(program_dir)