Closed will-moore closed 4 years ago
Travis integration test failing with:
> from numpy import fromfunction, int16
E ModuleNotFoundError: No module named 'numpy'
Need to require numpy in test dependencies
@will-moore : I pushed a commit which moves omero-test-infra further, but it's still failing locally with:
robj = run_script(client, id, args, "New_Figure")
> ann = robj.getValue()
E AttributeError: 'NoneType' object has no attribute 'getValue'
test/integration/test_figure_scripts.py:79: AttributeError
Running omero-test-infra
locally failed with
flake8.main.application MainProcess 4505 INFO Found a total of 107 violations and reported 2
./build/lib/omero_figure/views.py:489:23: E999 SyntaxError: invalid syntax
./build/lib/omero_figure/scripts/omero/figure_scripts/Figure_To_Pdf.py:814:45: E999 SyntaxError: invalid syntax
OK, so rm -rf ./build/
fixed the flake8 errors (old cached files there).
There are just 2 tests failing on travis:
test/integration/test_figure_files.py . [ 16%]
test/integration/test_figure_scripts.py F.F.. [100%]
Both of these are exporting as PDF:
The script stdout has:
b'Markdown not installed. See https://pypi.python.org/pypi/Markdown\nReportlab not installed.\nTraceback (most recent call last):\n File "./script", line 2327, in <module>\n run_script()\n File "./script", line 2313, in run_script\n file_annotation = export_figure(conn, script_params)\n File "./script", line 2269, in export_figure\n return fig_export.build_figure()\n File "./script", line 943, in build_figure\n self.create_figure()\n File "./script", line 1790, in create_figure\n "Need to install https://bitbucket.org/rptlab/reportlab")\nImportError: Need to install https://bitbucket.org/rptlab/reportlab\n'
I see this:
+ yum -y install python-reportlab python-markdown app-srv
Does that mean that reportlab is being installed on the OMERO.server docker or on the OMERO.web docker? (needs to be on the server).
Green now. Should be included in the next build.
Note - testing suspended, as Figure does not work on py3-ci
Conflicting PR. Removed from build OMERO-plugins-push#113. See the console output for more details. Possible conflicts:
--conflicts Conflict resolved in build OMERO-plugins-push#127. See the console output for more details.
Conflicting PR. Removed from build OMERO-plugins-push#114. See the console output for more details. Possible conflicts:
--conflicts Conflict resolved in build OMERO-plugins-push#128. See the console output for more details.
Tested with some unicode only i.e. მიკროსკოპის პონი
I end up with _.pdf
Removing the unicode completely is, I don't think, the way forward
Note: I have installed in virtualenv reportlab and markdown
There is some difference with Unicode handling in python3 on idr1-slot2
vv locally.
For example, if I add this into my script, it works fine locally:
fname = "µTest"
with open(fname, 'w') as out:
out.write(fname)
print(fname, os.path.exists(fname))
but fails when run in a script on that server:
Traceback (most recent call last):
File "./script", line 2333, in <module>
run_script()
File "./script", line 2319, in run_script
file_annotation = export_figure(conn, script_params)
File "./script", line 2275, in export_figure
return fig_export.build_figure()
File "./script", line 914, in build_figure
with open(fname, 'w') as out:
UnicodeEncodeError: 'ascii' codec can't encode character '\xb5' in position 0: ordinal not in range(128)
Any ideas?
Reading https://stackoverflow.com/questions/20022115/python3-and-encoding-different-on-linux-and-on-osx
I tried this locally and on py3 server:
>>> import sys
>>> sys.stdout.encoding
>>> os.environ.get('LANG')
# locally:
UTF-8
en_US.UTF-8
# py3 server
ANSI_X3.4-1968
None
that is probably the source of the problem
I don't know where to fix that, but hopefully @joshmoore or @manics could help?
Something like update-locale LANG=C.UTF-8
should set what we want
in the container py3ci_omero_1
on idr1-slot3
I got
. omero-virtualenv/bin/activate
import sys
>>> sys.stdout.encoding
'UTF-8'
import os
>>> os.environ.get('LANG')
'en_US.UTF-8'
This is required to fix the issue on py3-ci-devspace.openmicroscopy.org https://github.com/ome/omero-py/pull/150 but may also need changes to https://github.com/ome/omero-server-docker/blob/master/Dockerfile
With that last commit, and the script uploaded to py3-ci server, I can export a figure with unicode in the name (connecting from local omero-web since figure app is not yet working on py3-ci).
Tested on py3-ci, after the hint in https://openmicroscopy.slack.com/archives/CLHEW13NX/p1578329831029100
All works as expected. Tried to export a figure with unicode chars in name. Also, a figure with 2 panels, both export as tiff.
Also, I tried to rename an image with a unicode char in it (micrometer) Then, I added this image to the figure as a panel. Then exported the figure as tiff. This had as a result that the files in the "tiff with images" had still the micrometer in the name of the file I renamed in OMERO. Maybe I do not fully understand the header instructions of this PR ? These suggest that the unicode chars will be reomoved from the names in the exported objects. I did not observe any such removal. Instead, all is there as expected (which I like).
@pwalczysko it was initially removed. That was one of the problems since we could end up with a file with no name and only the extension e.g. _.pdf
good to merge then fmpov
WFM, but to clarify, the description and name no longer really match what's in the PR, correct?
correct
Might be worth updating to help create changelogs later. Otherwise, :+1: for merging.
Updated description and name
The unicode figure name issue was eventually addressed by changes to ci (see below). The only remaining changes to this PR are a tiny bug fix.
To test unicode figure export:
File names should be unchanged.
Previous errors:
show full error
``` Traceback (most recent call last): File "./script", line 118, in compress zip_file.write(fullpath, archive_name) File "/opt/rh/rh-python36/root/lib64/python3.6/zipfile.py", line 1622, in write with open(filename, "rb") as src, self.open(zinfo, 'w') as dest: File "/opt/rh/rh-python36/root/lib64/python3.6/zipfile.py", line 1355, in open return self._open_to_write(zinfo, force_zip64=force_zip64) File "/opt/rh/rh-python36/root/lib64/python3.6/zipfile.py", line 1468, in _open_to_write self.fp.write(zinfo.FileHeader(zip64)) File "/opt/rh/rh-python36/root/lib64/python3.6/zipfile.py", line 723, in write n = self.fp.write(data) AttributeError: 'bytes' object has no attribute 'write' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./script", line 2333, inWithout the line:
I now get:
show full error
``` Traceback (most recent call last): File "./script", line 2333, in