search

Psyop / Cryptomatte

Cryptomatte Nuke plugin, Fusion plugin, sample images, and specification
BSD 3-Clause "New" or "Revised" License
637 stars 152 forks source link

Crowdsourcing - Run test suite on different OSes and Nuke versions #126

Open jonahfriedman opened 4 years ago

jonahfriedman commented 4 years ago

I am currently only testing Nuke 12.0v3 on Windows 10. Cryptomatte does work around some Nuke idiosyncrasy which has been seen in the past to vary from version to version, so I'm attempting to crowdsource a bit here, If you would like to contribute and have some OSes and Nuke version combinations handy that you don't see results for here, please run the test suite and report back the results.

  1. Open Nuke with a Cryptomatte installation (1.2.6 or 1.3.0-beta2)
  2. Open a script editor in Nuke, clear it, and run the Python command below. See also, docs on tests
  3. Report back results. You should see the Cryptomatte version, Nuke version and OS in the results (in 1.2.6 and 1.3.0-beta2). If the result looks like the sample result below in the script editor, that's a pass and you don't need to post the full results.
  4. Also check the console for expression warnings between the results.

Python command:

import cryptomatte_utilities as cu
cu.tests.run_nuke_tests()

Cryptomatte 1.2.6, Nuke 12.0v3, Windows-10-10.0.18362:

# Result: test_csv_round_trip (cryptomatte_utilities_tests.CSVParsing) ... ok
test_mm3hash_float (cryptomatte_utilities_tests.CryptoHashing) ... ok
test_big_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_bug_still_exists (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug still exists. We don't want to be running the fix if we don't have to. ... ok
test_paste_with_channelmerge (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug has been fixed. ... ok
test_bad_names_in_nuke_layers (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_blendery_names_encryptomatte (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that names with nuke-unfriendly layer names are mangled property by Encryptomatte. ... ok
test_blendery_names_in_metadata (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that names with nuke-unfriendly layer names are mangled property. ... ok
test_channel_identification (cryptomatte_utilities_tests.CryptomatteNukeTests)
Makes sure multiple Cryptomattes can have sub-channels identified properly, ... ok
test_clear_button (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_crypto_channel_knobs_type (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_basic (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests both basic decryptomatte, as well as ensuring hash_channel() is ... ok
test_decrypto_custom_channel (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_matteonly_unpremul (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_rmchannels_alpha (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_decrypto_rmchannels_customlayer (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_bogus_inputs (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that when setting up layers, entering the name before pressing "setup layers" ... ok
test_encrypt_bogus_manifest (cryptomatte_utilities_tests.CryptomatteNukeTests) ... Cryptomatte: Unable to parse manifest. (Expecting property name: line 1 column 2 (char 1)).
ok
test_encrypt_channel_knobs_type (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_fresh_keyable (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests fresh Encryptomatte setup where there is no input constant. ... ok
test_encrypt_fresh_roundtrip (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_layer_forced_update_btn (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_layer_lock (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_layerselection (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_manifest (cryptomatte_utilities_tests.CryptomatteNukeTests)
Gets it into a weird state where it has a manifest but no cryptomatte. ... ok
test_encrypt_matte_name_autofill2 (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_merge_operations (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_roundtrip_keyable (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_roundtrip_setup (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_roundtrip_without_prefix (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_encrypt_setup_layers_numbers (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests that when setting up layers, entering the name before pressing "setup layers" ... ok
test_file_open_doesnt_update_nodes (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_gizmo_version (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_blank_matteList (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_clear (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_manual (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_multiselect (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_nothing (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_partial_black (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_single_selection (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_stop_auto_update (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_with_removechannels (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_without_prefix (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_keying_without_preview_channels (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_bogus_manifest (cryptomatte_utilities_tests.CryptomatteNukeTests) ... Cryptomatte: Unable to parse manifest. (Expecting property name: line 1 column 2 (char 1)).
Cryptomatte: Unable to parse manifest. (Expecting property name: line 1 column 2 (char 1)).
ok
test_layer_forced_update_btn (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_forced_update_func (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_lock (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_selection (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_layer_selection_after_keying (cryptomatte_utilities_tests.CryptomatteNukeTests)
Tests all layer selection options are still available after keying. ... ok
test_load_manifests_lazy (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_load_manifests_nonlazy (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_manifests (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_matte_list_name_modifications (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_matte_list_numeric (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_output_preview (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_output_preview_disabled (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok
test_output_preview_multi (cryptomatte_utilities_tests.CryptomatteNukeTests) ... ok

----------------------------------------------------------------------
Ran 59 tests in 6.151s

OK
---------
Cryptomatte 1.2.6, Nuke 12.0v3, Windows-10-10.0.18362
---------
Testing passed: 0 failed, 0 errors. (59 test cases.)
mybikeislost commented 4 years ago 
----------------------------------------------------------------------
Ran 59 tests in 6.636s

OK
---------
Cryptomatte 1.2.6, Nuke 11.3v6, Darwin-18.7.0-x86_64-i386-64bit
---------
Testing passed: 0 failed, 0 errors. (59 test cases.)
jacksonspencer commented 4 years ago 
# Result: test_csv_round_trip (cryptomatte_utilities_tests.CSVParsing) ... ok
test_mm3hash_float (cryptomatte_utilities_tests.CryptoHashing) ... ok
test_big_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_csv_through_gizmo (cryptomatte_utilities_tests.CSVParsingNuke) ... ok
test_bug_still_exists (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug still exists. We don't want to be running the fix if we don't have to. ... ok
test_paste_with_channelmerge (cryptomatte_utilities_tests.CryptomatteNodePasting)
Tests this bug has been fixed. ... ok
ERROR

======================================================================
ERROR: setUpClass (cryptomatte_utilities_tests.CryptomatteNukeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:/Program Files/Nuke12.2v2/plugins\cryptomatte_utilities_tests.py", line 290, in setUpClass
    (file_path, SAMPLES_IMAGES_DIR_ENVIRON))
IOError: Could not find: C:/Program Files/Nuke12.2v2/sample_images/bunny_CryptoObject.exr. Sample image dir can be defined env variable, CRYPTOMATTE_TESTING_SAMPLES

----------------------------------------------------------------------
Ran 6 tests in 0.138s

FAILED (errors=1)
---------
Cryptomatte 1.2.8, Nuke 12.2v2, Windows-10-10.0.18362
---------
Error: _ErrorHolder.

Traceback (most recent call last):
  File "C:/Program Files/Nuke12.2v2/plugins\cryptomatte_utilities_tests.py", line 290, in setUpClass
    (file_path, SAMPLES_IMAGES_DIR_ENVIRON))
IOError: Could not find: C:/Program Files/Nuke12.2v2/sample_images/bunny_CryptoObject.exr. Sample image dir can be defined env variable, CRYPTOMATTE_TESTING_SAMPLES

---------
TESTING FAILED: 0 failed, 1 errors. (60 test cases.)
jacksonspencer commented 4 years ago

I'm not sure if this has anything to do with what it shows in the test I posted above or if I'm doing something incorrectly, but using the picker doesn't highlight the objects in yellow and fully create the matte, but it does select the correct identifier and add it to the list.

jacksonspencer commented 4 years ago

Posting back here again to say that after running tests, using Cryptomatte in Nuke with my set up listed above works correctly when the Cryptomatte options in RenderMan for Maya 23.3 are set to create mattes based on object name, and does not work when set to create mattes based on applied materials. In Nuke, the object name identifier shows up as "identifier_obect#" and the material identifier shows up as "user___materialid#"