search

golemfactory / clay

Golem is creating a global market for computing power.
https://golem.network
GNU General Public License v3.0
2.91k stars 286 forks source link

Verification failure (wlochaty3.blend) #3617

Closed shadeofblue closed 5 years ago

shadeofblue commented 5 years ago

Version: 0.18.2+dev295.gfc68f0a System: Ubuntu 18.04

While running the test: `scripts/node_integration_tests/tests/test_golem.py::GolemNodeTest::test_jpeg``

task dict:

{'type': 'Blender', 'name': 'test task', 'timeout': '0:10:00', 'subtask_timeout': '0:09:50', 'subtasks_count': 1, 'bid': 1.0, 'resources': ['/home/blue/PycharmProjects/golem/scripts/node_integration_tests/tasks/test_task_1/wlochaty3.blend', '/home/blue/PycharmProjects/golem/scripts/node_integration_tests/tasks/test_task_1/puchacz_canopy.jpg'], 'options': {'output_path': '/tmp/tmpjd43z6y3', 'format': 'JPEG', 'resolution': [320, 240]}}

verification failure is reported

2018-11-29 15:27:05 DEBUG    apps.blender                        METRIC: Subtask: 'c7471a26-f3e2-11e8-83a8-e06bc7d8f875' crop no: 0, frame 0 scene /golem/resources/wlochaty3.blend 
requestor 'e06bc7d8f8754b42d117c86e186b0705438569817ade71bedc96ff60657cd34e107ea1dbf9ff66a37841abb26b103ac83b74132c30954b38cb5a24faaee12874'
provider 'b8b876594902e575388e1132dd189ecbe079dd936097bedc524a1dcb132449dc60da41eb3ca2d84faff90f008da9671cb55a319d38ae1eccfefe0ba593fdc42c' metrics {0: {0: {'Label': 'FALSE', 'comp_edge_factor': 18.01953125, 'cr
op_resolution': '32x24', 'edge_difference': 3.6644965277777777, 'histograms_correlation': 0.9988992723765092, 'image_variance': 5.121476915147573, 'max_x_mass_center_distance': 0.00015838865049355721, 'max_y_mas
s_center_distance': 9.637079637581092e-05, 'psnr': 49.82995382282539, 'ref_edge_factor': 17.960069444444443, 'reference_variance': 3.5793762207031223, 'ssim': 0.9936472101164285, 'variance_difference': 1.5421006
944444509, 'wavelet_db4_base': 6.317704386260237, 'wavelet_db4_high': 0.44221926848543436, 'wavelet_db4_low': 6.317704386260237, 'wavelet_db4_mid': 6.317704386260237, 'wavelet_haar_base': 3.2421875000005826, 'wa
velet_haar_freq_x1': 0.13888888888888906, 'wavelet_haar_freq_x2': 0.4513888888888845, 'wavelet_haar_freq_x3': 0.9166666666666584, 'wavelet_haar_high': 1.1756944444444417, 'wavelet_haar_low': 1.7213541666667211, 
'wavelet_haar_mid': 19.296874999999968, 'wavelet_sym2_base': 74.01946493992185, 'wavelet_sym2_high': 0.44329167591234075, 'wavelet_sym2_low': 21.75956027822425, 'wavelet_sym2_mid': 3.1080987668540674}}, 1: {0: {
'Label': 'TRUE', 'comp_edge_factor': 17.71875, 'crop_resolution': '32x24', 'edge_difference': 0.0, 'histograms_correlation': 1.0, 'image_variance': 0.0, 'max_x_mass_center_distance': 0, 'max_y_mass_center_distan
ce': 0, 'psnr': 3.4028234663852886e+38, 'ref_edge_factor': 17.71875, 'reference_variance': 0.0, 'ssim': 1.0, 'variance_difference': 0.0, 'wavelet_db4_base': 0.0, 'wavelet_db4_high': 0.0, 'wavelet_db4_low': 0.0, 
'wavelet_db4_mid': 0.0, 'wavelet_haar_base': 0.0, 'wavelet_haar_freq_x1': 0.0, 'wavelet_haar_freq_x2': 0.0, 'wavelet_haar_freq_x3': 0.0, 'wavelet_haar_high': 0.0, 'wavelet_haar_low': 0.0, 'wavelet_haar_mid': 0.0
, 'wavelet_sym2_base': 0.0, 'wavelet_sym2_high': 0.0, 'wavelet_sym2_low': 0.0, 'wavelet_sym2_mid': 0.0}}, 2: {0: {'Label': 'TRUE', 'comp_edge_factor': 21.73220486111111, 'crop_resolution': '32x24', 'edge_differe
nce': 133.76649305555554, 'histograms_correlation': 0.3339002857019339, 'image_variance': 1453.105141533746, 'max_x_mass_center_distance': 0.0005397902674758592, 'max_y_mass_center_distance': 0.00139095388042043
13, 'psnr': 37.657987313527386, 'ref_edge_factor': 21.94140625, 'reference_variance': 1437.1118384467234, 'ssim': 0.9542230129088348, 'variance_difference': 15.993303087022696, 'wavelet_db4_base': 98.55965061965
21, 'wavelet_db4_high': 9.62300995873549, 'wavelet_db4_low': 98.5596506196521, 'wavelet_db4_mid': 98.5596506196521, 'wavelet_haar_base': 338.8437499999982, 'wavelet_haar_freq_x1': 0.4131944444444448, 'wavelet_ha
ar_freq_x2': 2.4236111111111156, 'wavelet_haar_freq_x3': 1.5000000000000127, 'wavelet_haar_high': 26.44375000000001, 'wavelet_haar_low': 76.01562500000087, 'wavelet_haar_mid': 147.9730902777774, 'wavelet_sym2_ba
se': 644.7550451104072, 'wavelet_sym2_high': 14.65286929202725, 'wavelet_sym2_low': 231.20599200584377, 'wavelet_sym2_mid': 83.05144775470332}}}
2018-11-29 15:27:05 DEBUG    raven.base.Client                   Sending message of length 1925 to https://talkback.golem.network/api/5/store/
2018-11-29 15:27:05 WARNING  apps.blender                        Subtask 'c7471a26-f3e2-11e8-83a8-e06bc7d8f875' verified. Result: REJECT
2018-11-29 15:27:05 WARNING  apps.blender                        Failure callback <twisted.python.failure.Failure builtins.bool: False>
2018-11-29 15:27:05 WARNING  apps.blender.verification           Finishing verification with fail
magdasta commented 5 years ago

It's because of JPEG compression distorting small crops differently than big images (locally generated reference vs. provider's result). We can't do much about it, except discouraging users from using this format or dropping its support. In the future we might want to add a new parameter defining compression level, but most probably JPEG is rarely used by blender artists, so I wouldn't consider it a big problem, at least for now.