search

MESAHub / mesa_test

Command line tool for running the MESA test suite and uploading results to MESATestHub
MIT License
2 stars 3 forks source link

Handle missing testhub.yml #23

Closed jschwab closed 3 years ago

jschwab commented 3 years ago

In extreme cases (i.e., a failure in const, utils, or math), the install process may fail to generate the file testhub.yml.

mesa_test should know how to handle the absence of this file. It should report a compilation failure.

At present, it dies with

lux: ~$ mesa_test submit --empty
Email, password, and computer name accepted
This MESA installation doesn't seem to be compiled properly. Submitting a compilation failure to MESATestHub.
Traceback (most recent call last):
        10: from /home/jwschwab/modules/ruby-2.6.4/bin/mesa_test:23:in `<main>'
         9: from /home/jwschwab/modules/ruby-2.6.4/bin/mesa_test:23:in `load'
         8: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/mesa_test-1.0.0/bin/mesa_test:386:in `<top (required)>'
         7: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
         6: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
         5: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
         4: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
         3: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/mesa_test-1.0.0/bin/mesa_test:144:in `submit'
         2: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/mesa_test-1.0.0/lib/mesa_test.rb:320:in `submit_commit'
         1: from /home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/mesa_test-1.0.0/lib/mesa_test.rb:245:in `commit_params'
/home/jwschwab/modules/ruby-2.6.4/lib/ruby/gems/2.6.0/gems/mesa_test-1.0.0/lib/mesa_test.rb:561:in `compiler_hash': Could not find file testhub.yml in /data/groups/ramirez-ruiz/jwschwab/.mesa_test/work. (MesaDirError)
wmwolf commented 3 years ago

Is there a particular commit SHA I can test this out on?

Edit: Per slack conversation, it seems 9a0f870 will do nicely.

wmwolf commented 3 years ago

Plan is to insert "Unknown" for the various compiler information that would normally be in testhub.yml. The downside here is that this now couples mesa_test to knowing what is supposed to be in this file. I don't really see another way about it right now, but it's something we should keep in mind.

wmwolf commented 3 years ago

I have this working now, and it will submit a compilation failure after attempting to install 9a0f870. However, it will now happily submit a compilation failure even if you never even tried to run install. That is, a user could do mesa_test checkout && mesa_test submit --empty, and it would submit as a compilation failure.

I wonder if we should have the first thing install does is touch testhub.yml, and then if that file is found, but empty, submit a compilation failure.

jschwab commented 3 years ago

Yes, that's a great idea. Done in bda98f93.

wmwolf commented 3 years ago

This is now implemented in mesa_test v1.0.4. If no testhub.yml file is found, it barks at you and doesn't even try to submit. If one is found, but nothing is in it, it populates the compiler, compiler version, etc. with "Unknown" and carries on with submitting a compilation failure. For example, see Rummy's submission of 9a0f870, which I artificially made work by creating an empty testhub.yml file.

Such tricks should no longer be necessary after bda98f93, which should always succeed in creating testhub.yml at the beginning of installation.