haesleinhuepf / human-eval-bia

Benchmarking Large Language Models for Bio-Image Analysis Code Generation
MIT License
20 stars 13 forks source link

New OME XML test case not working #144

Open haesleinhuepf opened 5 days ago

haesleinhuepf commented 5 days ago

Hi Riccardo @rmassei ,

in #142, this notebook doesn't work on my computer (the assert fails). There is a special character in the xml, which might be read differently on my Windows. I removed the notebook from your submission and if you can make it work again, a follow-up PR would be welcome.

read_ome_xml_from_ome_tiff.ipynb.txt

Error:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 check(read_ome_metadata_from_ome_tiff)

Cell In[2], line 11, in check(candidate)
      8 ome_string_result = ome_string_result.strip()
      9 ome_string_expected = ome_string_expected.strip()
---> 11 assert ome_string_result == ome_string_expected

AssertionError: 

Thanks!

Best, Robert

rmassei commented 4 days ago

Really strange, it seems that the additional character is in the expected string

Number of characters in results: 2866
Number of characters in results: 2867

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[6], line 1
----> 1 check(read_ome_metadata_from_ome_tiff)

Cell In[5], line 14, in check(candidate)
     11 print(f"Number of characters in results: {len(ome_string_result)}")
     12 print(f"Number of characters in results: {len(ome_string_expected)}")
---> 14 assert ome_string_result == ome_string_expected

AssertionError: 

I will dig into it

rmassei commented 4 days ago

found it, the code was reading "µm" as "µm" due to the missing utf-8 encoding. :) Easy fix, new version is coming in the next PR :)