search

google-deepmind / optax

Optax is a gradient processing and optimization library for JAX.
https://optax.readthedocs.io
Apache License 2.0
1.71k stars 194 forks source link

Tidy up test.sh and make it clean up properly. #1074

Closed carlosgmartin closed 2 months ago

carlosgmartin commented 2 months ago

Partially addresses #1067. Currently, test.sh leaves a wheel file optax-0.2.4.dev0-py3-none-any.whl in the root directory of the repo. It also doesn't run cleanup when an intermediate failure occurs.

carlosgmartin commented 2 months ago

why are you saying that this PR only partially solves the issues of #1067?

Sorry for the unclear wording. I just meant that #1067 was also addressed by #1068 and #1071.

On my machine, test.sh creates a file called docs/sg_execution_times.rst:

``` (venv) $ sh test.sh ------------------------------------------------------------------- Your code has been rated at 10.00/10 (previous run: 1.41/10, +8.59) ------------------------------------ Your code has been rated at 10.00/10 * Creating isolated environment: venv+pip... * Installing packages in isolated environment: - flit_core >=3.2,<4 * Getting build dependencies for sdist... * Building sdist... Version number normalised: '0.2.4.dev' -> '0.2.4.dev0' (see PEP 440) * Building wheel from sdist * Creating isolated environment: venv+pip... * Installing packages in isolated environment: - flit_core >=3.2,<4 * Getting build dependencies for wheel... * Building wheel... Version number normalised: '0.2.4.dev' -> '0.2.4.dev0' (see PEP 440) Successfully built optax-0.2.4.dev0.tar.gz and optax-0.2.4.dev0-py3-none-any.whl Processing ./dist/optax-0.2.4.dev0.tar.gz Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: optax Building wheel for optax (pyproject.toml) ... done Created wheel for optax: filename=optax-0.2.4.dev0-py3-none-any.whl size=303574 sha256=fc038fe0d99da6cde0fbdc6f6637eca69d61e149385de98a42319d81724565e4 Stored in directory: /Users/carlos/Library/Caches/pip/wheels/12/8b/12/f73f3c3c2e327a62f3b987cf82f38ac6de00e4d01f1a02e9cd Successfully built optax Computing dependencies Analyzing 110 sources with 0 local dependencies ninja: Entering directory `.pytype' [110/110] check optax.losses._self_supervised_test Leaving directory '.pytype' Success: no errors found =============================== test session starts ================================ platform darwin -- Python 3.12.3, pytest-8.3.3, pluggy-1.5.0 rootdir: /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT plugins: xdist-3.6.1 16 workers [3564 items] ss......ss........s....s.......s.......s.s......................s..s........ [ 2%] .....s.s...s..s.....s...............................s..s.....s....s.......s. [ 4%] .....s..s......s.s...s....s..s......s.s....s.s.s.....s.............s....s.s. [ 6%] s....s.....s.......ss...s...s..s...s.......s.....s.s....s..s...s.s.........s [ 8%] .....s.s.s....s.s.....s..s..s.....s....s...s..s...s.......s.......s.s....s.. [ 10%] .s...s......s..s...s.....s.s...s..s.........ss....s..s...s....s..s.......... [ 12%] s.......s..s.....s..s...s.....s...ss...s..s..s.....s....s....s....s.......s. [ 14%] .....s.......s....s...s..s.s.....s.s.........sss....s.......s..s...s....s... [ 17%] ..s.s...s..s.........s...s...s...s.......s..s...s.........s..s...s.....ss.s. [ 19%] .......s....s.s......s...........s.....s.....s...s..s..s...s....s..ss....... [ 21%] .s..s.....ss.......s.......s.s........s.s............s......s...s...s....... [ 23%] s..s..s....s....s......s..s.........s....s......s......s.s.......s...s..s... [ 25%] s........s...s...s.......s.....s..s...s...........s.....s....s........ss.s.s [ 27%] ..ss.......s....s....s.s............s..s.........s.........s..............s. [ 29%] ...s...s.......s...s......s.......s.........s....s...s......s.s......s...... [ 31%] s......s.....s......s.....s.......s..........s.......s.....s................ [ 34%] .s.....s.......s............s..s.........s.........................s.s...s.s [ 36%] .......................s......s.s..............s...........s................ [ 38%] s..s..............s......s.............s...s......s...s.....s......s...s.... [ 40%] .....s.................s.......s...s....s.........s.........s..s............ [ 42%] s...s............ss.......s.....s....s.....s..s.....s.........s.......s...s. [ 44%] .s....................s.....s...s.s........s.s....s.....s......s........s... [ 46%] .s..s.........s...s.s..s.s...s....ss.....s.......s.....s...s....s....s..s... [ 49%] ...s.....s..........s...s..s......s........s...s.s...s.s.......s...........s [ 51%] ....s..........s.s..s..s....s..s...s........s.........s..s....s.......ss.... [ 53%] ...s.s........s.s.s.........s....s..s.s.....s.........s.....s....s.......... [ 55%] s.s.....s...s..s...........s......s...................s.........s..s........ [ 57%] s...s....s......s.....s..........s.....s....s....s.....s......s..s.....s.... [ 59%] ....s.......s.....s.s.s.....s.......s.s......s.....s..s....s............s... [ 61%] .s..........s....s....s..........s.......s.s..........ss....s....s...s....s. [ 63%] ..s.ssss.....s.......s.......s....s....s......s..........s..s.s..........s.. [ 66%] ....s...s.....s.....s....s.s........s..s....s.......s..s.s.......s...ss...s. [ 68%] ...s...s..s........s.........s.....s...s..s.......s.....................s... [ 70%] ......s....s........s.....s.....s.....s..s....s......s.....s..s......s...... [ 72%] .....s.s...s..........s....s...s.....s..s.........s......s.s.s....s.......s. [ 74%] ...s...s....s....s.s...s.s..............s..s......s......s......s......s.... [ 76%] s.............s.....s...s.......s.s.....s.s.s....s...s......s.........s..... [ 78%] ..s.........s..s......s..s.......s...s.s..s.s........s..s.......s.s....s.... [ 81%] s...........s..s...s........ss..s...s.......s.....s.....ss...........s....s. [ 83%] ..s...s..s.s..s.....s......s......s......s........s.s.s..........s......ss.. [ 85%] .....s....s....s...s........s......s.........s.s.s..s.s........ss..........s [ 87%] ...s....s.s.s......s...s.............s........s....s................s....s.s [ 89%] ..s...ss..............s.....s.s..ss.s.............s.....s.s.......s..s...... [ 91%] .s.........s..s.....s.........s.s.......s...s......s.s.s.s....s..s..s.s.s... [ 93%] ..s.........s.s....ss....................ss....s........s..s....s.s....s.... [ 95%] ............................s......s....................................s... [ 98%] .......s..s........s............................s........s...s....s. [100%] ================================= warnings summary ================================= venv/lib/python3.12/site-packages/optax/_src/alias_test.py: 6 warnings venv/lib/python3.12/site-packages/optax/contrib/_schedule_free_test.py: 4 warnings /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages/jax/_src/numpy/array_methods.py:118: DeprecationWarning: Casting from complex to real dtypes will soon raise a ValueError. Please first use jnp.real or jnp.imag to take the real/imaginary component of your input. return lax_numpy.astype(self, dtype, copy=copy, device=device) venv/lib/python3.12/site-packages/optax/_src/alias_test.py: 16 warnings /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages/optax/_src/transform.py:1521: UserWarning: Explicitly requested dtype float64 requested in zeros is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more. lambda leaf: jnp.zeros((memory_size,) + leaf.shape, dtype=leaf.dtype), venv/lib/python3.12/site-packages/optax/contrib/_common_test.py::ContribTest::test_optimizers_can_be_wrapped_in_inject_hyperparams_(opt_name='momo_adam', opt_kwargs={'learning_rate': 0.1}, wrapper_name=None, wrapper_kwargs=None)__without_device /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages/optax/contrib/_momo.py:301: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more. bc2 = jnp.asarray(1 - b2 ** count_inc, dtype=barf.dtype) venv/lib/python3.12/site-packages/optax/contrib/_common_test.py::ContribTest::test_optimizers_can_be_wrapped_in_inject_hyperparams_(opt_name='momo_adam', opt_kwargs={'learning_rate': 0.1}, wrapper_name=None, wrapper_kwargs=None)__without_device /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages/optax/contrib/_momo.py:310: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more. bc1 = jnp.asarray(1 - b1 ** count_inc, dtype=barf.dtype) venv/lib/python3.12/site-packages/optax/tree_utils/_tree_math_test.py::TreeUtilsTest::test_tree_add_scalar_mul /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages/jax/_src/numpy/array_methods.py:118: UserWarning: Explicitly requested dtype complex128 requested in astype is not available, and will be truncated to dtype complex64. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more. return lax_numpy.astype(self, dtype, copy=copy, device=device) venv/lib/python3.12/site-packages/optax/tree_utils/_tree_math_test.py::TreeUtilsTest::test_tree_add_scalar_mul /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages/jax/_src/numpy/array_methods.py:118: UserWarning: Explicitly requested dtype float64 requested in astype is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more. return lax_numpy.astype(self, dtype, copy=copy, device=device) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================= 2957 passed, 607 skipped, 30 warnings in 35.70s ================== WARNING: Generating metadata for package sphinx-collections produced metadata for project name sphinxcontrib-collections. Fix your #egg=sphinx-collections fragments. Using pip 24.2 from /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages/pip (python 3.12) Requirement already satisfied: typing_extensions in /private/var/folders/9l/175rd1wj3mbbbpppdyh788680000gn/T/tmp.95WKOqtieT/venv/lib/python3.12/site-packages (4.12.2) Running Sphinx v8.0.2 loading translations [en]... done Using Sphinx-Gallery to convert rst text blocks to markdown for .ipynb files. Read in collections ... examples: Initialised Clean collections ... Executing collections ... examples: (CopyFolderDriver) Copy folder... Converting `source_suffix = ['.rst', '.md', '.ipynb']` to `source_suffix = {'.rst': 'restructuredtext', '.md': 'restructuredtext', '.ipynb': 'restructuredtext'}`. loading pickled environment... done [autosummary] generating autosummary for: _collections/examples/README.md, _collections/examples/adversarial_training.ipynb, _collections/examples/cifar10_resnet.ipynb, _collections/examples/contrib/README.md, _collections/examples/contrib/differentially_private_sgd.ipynb, _collections/examples/contrib/reduce_on_plateau.ipynb, _collections/examples/contrib/sam.ipynb, _collections/examples/flax_example.ipynb, _collections/examples/gradient_accumulation.ipynb, _collections/examples/lbfgs.ipynb, ..., api/optimizer_wrappers.rst, api/optimizers.rst, api/projections.rst, api/stochastic_gradient_estimators.rst, api/transformations.rst, api/utilities.rst, development.md, gallery.rst, getting_started.ipynb, index.rst myst v4.0.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions={'dollarmath', 'amsmath'}, disable_syntax=[], all_links_external=False, links_external_new_tab=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, fence_as_directive=set(), number_code_blocks=[], title_to_header=False, heading_anchors=0, heading_slug_func=None, html_meta={}, footnote_sort=True, footnote_transition=True, words_per_minute=200, substitutions={}, linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', enable_checkboxes=False, suppress_warnings=[], highlight_code_blocks=True) myst-nb v1.1.2: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, eval_name_regex='^[a-zA-Z_][a-zA-Z0-9_]*$', execution_mode='force', execution_cache_path='', execution_excludepatterns=['nanolm.ipynb', 'cifar10_resnet.ipynb', 'adversarial_training.ipynb', 'reduce_on_plateau.ipynb', 'differentially_private_sgd.ipynb'], execution_timeout=30, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, code_prompt_show='Show code cell {type}', code_prompt_hide='Hide code cell {type}', number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False) Using jupyter-cache at: /Users/carlos/Desktop/optax/docs/_build/.jupyter_cache generating gallery... generating gallery for _collections/generated_examples... generating gallery for _collections/generated_examples/contrib... computation time summary: building [mo]: targets for 0 po files that are out of date writing output... building [html]: targets for 36 source files that are out of date updating environment: 0 added, 1 changed, 0 removed reading sources... [100%] sg_execution_times looking for now-outdated files... none found pickling environment... done checking consistency... done preparing documents... done copying assets... copying downloadable files... [100%] _collections/generated_examples/generated_examples_python.zip copying static files... done copying extra files... done copying assets: done writing output... [100%] sg_execution_times generating indices... genindex done writing additional pages... search done copying images... [ 4%] _build/jupyter_execute/2db2ef768b28720d9668d5b49a01d5db7d44copying images... [ 7%] _build/jupyter_execute/4cfb16fc489800e657c95ac17ce286d1dd1ccopying images... [ 11%] _build/jupyter_execute/13f8d35808986e574b77dc833722e07b865ccopying images... [ 14%] _build/jupyter_execute/b331292ac25cf6cee9e33d2475388111e76ccopying images... [ 18%] _build/jupyter_execute/7dbf8ca6be25e44d63e604e8f102a946e70fcopying images... [ 21%] _build/jupyter_execute/f2cfc35febcaff6cb995ac675582c1b3f397copying images... [ 25%] _build/jupyter_execute/4551da1a5f466a35bb9cb93e0829ba6e373ccopying images... [ 29%] _build/jupyter_execute/ca81746dc88288f76c549ab154ce1ff04b85copying images... [ 32%] _build/jupyter_execute/41af00b14d2cfcc8735c6f9bee66ddccb697copying images... [ 36%] _build/jupyter_execute/0bd7fdee1feedf83b46d8793bd5acdb3c144copying images... [ 39%] _build/jupyter_execute/00f7bf90403b6b86b923474fb2641b2637ebcopying images... [ 43%] _build/jupyter_execute/b33b3fa627f8cc2f50fdfd81f97c410a2efdcopying images... [ 46%] _build/jupyter_execute/1d7f7eb7a6473fe989ae3d1e3bad743e014ecopying images... [ 50%] _build/jupyter_execute/602c2c65a082a421446798c67857612178f0copying images... [ 54%] _build/jupyter_execute/7ef13b68c1504170c28f773e6655848b35f2copying images... [ 57%] _build/jupyter_execute/44394d62237b861d5ef5e9377a8e0b77ea65copying images... [ 61%] _build/jupyter_execute/4eeaacf737971254a0094c2699a2009c8ea2copying images... [100%] images/examples/contrib/sam.png dumping search index in English (code: en)... done dumping object inventory... done Sphinx-Gallery successfully executed 0 out of 0 files subselected by: gallery_conf["filename_pattern"] = '/plot' gallery_conf["ignore_pattern"] = '_test\\.py' after excluding 0 files that had previously been run (based on MD5). embedding documentation hyperlinks... embedding documentation hyperlinks for _collections/generated_examples/... [ 25%] sgembedding documentation hyperlinks for _collections/generated_examples/... [ 50%] inembedding documentation hyperlinks for _collections/generated_examples/... [ 75%] sgembedding documentation hyperlinks for _collections/generated_examples/... [100%] index.html Final clean of collections ... examples: (CopyFolderDriver) Folder deleted: /Users/carlos/Desktop/optax/docs/_collections/examples build succeeded. The HTML pages are in _build/html. Running Sphinx v8.0.2 loading translations [en]... done Using Sphinx-Gallery to convert rst text blocks to markdown for .ipynb files. Read in collections ... examples: Initialised Clean collections ... Executing collections ... examples: (CopyFolderDriver) Copy folder... Converting `source_suffix = ['.rst', '.md', '.ipynb']` to `source_suffix = {'.rst': 'restructuredtext', '.md': 'restructuredtext', '.ipynb': 'restructuredtext'}`. loading pickled environment... done [autosummary] generating autosummary for: _collections/examples/README.md, _collections/examples/adversarial_training.ipynb, _collections/examples/cifar10_resnet.ipynb, _collections/examples/contrib/README.md, _collections/examples/contrib/differentially_private_sgd.ipynb, _collections/examples/contrib/reduce_on_plateau.ipynb, _collections/examples/contrib/sam.ipynb, _collections/examples/flax_example.ipynb, _collections/examples/gradient_accumulation.ipynb, _collections/examples/lbfgs.ipynb, ..., api/optimizers.rst, api/projections.rst, api/stochastic_gradient_estimators.rst, api/transformations.rst, api/utilities.rst, development.md, gallery.rst, getting_started.ipynb, index.rst, sg_execution_times.rst myst v4.0.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions={'dollarmath', 'amsmath'}, disable_syntax=[], all_links_external=False, links_external_new_tab=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, fence_as_directive=set(), number_code_blocks=[], title_to_header=False, heading_anchors=0, heading_slug_func=None, html_meta={}, footnote_sort=True, footnote_transition=True, words_per_minute=200, substitutions={}, linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', enable_checkboxes=False, suppress_warnings=[], highlight_code_blocks=True) myst-nb v1.1.2: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, eval_name_regex='^[a-zA-Z_][a-zA-Z0-9_]*$', execution_mode='force', execution_cache_path='', execution_excludepatterns=['nanolm.ipynb', 'cifar10_resnet.ipynb', 'adversarial_training.ipynb', 'reduce_on_plateau.ipynb', 'differentially_private_sgd.ipynb'], execution_timeout=30, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, code_prompt_show='Show code cell {type}', code_prompt_hide='Hide code cell {type}', number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False) Using jupyter-cache at: /Users/carlos/Desktop/optax/docs/_build/.jupyter_cache generating gallery... generating gallery for _collections/generated_examples... generating gallery for _collections/generated_examples/contrib... computation time summary: building [mo]: targets for 0 po files that are out of date writing output... building [doctest]: targets for 36 source files that are out of date updating environment: 0 added, 0 changed, 0 removed reading sources... looking for now-outdated files... none found running tests... Document: api/optimizer_schedules --------------------------------- 1 items passed all tests: 12 tests in default 12 tests in 1 items. 12 passed and 0 failed. Test passed. Document: api/combining_optimizers ---------------------------------- 1 items passed all tests: 8 tests in default 8 tests in 1 items. 8 passed and 0 failed. Test passed. Document: api/losses -------------------- 1 items passed all tests: 12 tests in default 12 tests in 1 items. 12 passed and 0 failed. Test passed. Document: api/transformations ----------------------------- 1 items passed all tests: 41 tests in default 41 tests in 1 items. 41 passed and 0 failed. Test passed. Document: api/contrib --------------------- 1 items passed all tests: 56 tests in default 56 tests in 1 items. 56 passed and 0 failed. Test passed. Document: api/utilities ----------------------- 1 items passed all tests: 91 tests in default 91 tests in 1 items. 91 passed and 0 failed. Test passed. Document: api/optimizers ------------------------ 1 items passed all tests: 244 tests in default 244 tests in 1 items. 244 passed and 0 failed. Test passed. Doctest summary =============== 464 tests 0 failures in tests 0 failures in setup code 0 failures in cleanup code Sphinx-Gallery successfully executed 0 out of 0 files subselected by: gallery_conf["filename_pattern"] = '/plot' gallery_conf["ignore_pattern"] = '_test\\.py' after excluding 0 files that had previously been run (based on MD5). Final clean of collections ... examples: (CopyFolderDriver) Folder deleted: /Users/carlos/Desktop/optax/docs/_collections/examples build succeeded. Testing of doctests in the sources finished, look at the results in _build/doctest/output.txt. All tests passed. Congrats! (venv) $ git status On branch tidy_test_sh Your branch is up to date with 'origin/tidy_test_sh'. Untracked files: (use "git add ..." to include in what will be committed) docs/sg_execution_times.rst nothing added to commit but untracked files present (use "git add" to track) (venv) $ ```

Does this happen on your machine? How would you prefer to address it?

vroulet commented 2 months ago

Yes, I had seen it as I was doing the review. It's a file created by sphinx gallery. There does not seem to be an option for sphinx gallery to simply not create that file. I would simply add it to the clean up operations, i.e., modify your cleanup function as follows

function cleanup {
  deactivate
  rm -r "${TEMP_DIR}"
  # sphinx gallery creates a log of execution times
  # we delete it manually by lack of better mechanism
  rm -f docs/sg_execution_times.rst
}

If you have a better solution, I'll take it but I think it's ok to simply clean it up at the end. (Best thing to do would be to open an issue on sphinx-gallery but it's out of scope of this PR).

Thank you again for taking care of this. We are used to run tests on google's side and it will really help users to be able to actually run tests locally properly.

carlosgmartin commented 2 months ago

@vroulet We could also add it to .gitignore. Might be useful to keep the file for debugging/inspection purposes. What do you think?

carlosgmartin commented 2 months ago

Come to think of it, the same might be true for the virtual env created and used by the test script (keeping it in a gitignored folder, e.g. tests_venv, rather than deleting it on exit, to allow for debugging/inspection).

vroulet commented 2 months ago

We could also add it to .gitignore.

That's the right solution. I don't think we will ever need these logs but I'd prefer to keep test.sh as clean as possible without weird exceptions like this one.

the same might be true for the virtual env

To be honest I don't know why we had this clean up logic in the first place (I'm not one of the original authors of the package). But I don't think either of us are eager to understand why if we start using a gitignore and finding bugs. Your PR is quite clean and I think it solves well the issue you raised so just add docs/sg_execution_times.rst to the gitignore file and we're good I think.

carlosgmartin commented 2 months ago

@vroulet Done.