wolph / numpy-stl

Simple library to make working with STL files (and 3D objects in general) fast and easy.
http://numpy-stl.readthedocs.org/
BSD 3-Clause "New" or "Revised" License
624 stars 105 forks source link

Support writing to io.BytesIO() psuedo files. #148

Closed bwoodsend closed 3 years ago

bwoodsend commented 3 years ago

The only change needed for this is to conditionally swap array.tofile(fh) for the marginally slower fh.write(array.data) when fh is not a real file.

I've also added a test for it, although it may not be in the correct place.

I was getting a very strange issue before this change where _write_binary() returned on self.data.tofile(fh) meaning that it never reached the assertion below which should have caught that numpy hadn't written to my fh.

coveralls commented 3 years ago

Coverage Status

Coverage decreased (-100.0%) to 0.0% when pulling 7ed56fc66563d4380a810615ceaf57cdb88cf49b on bwoodsend:support-write-to-io-buffers into 93f194684f1f208fd333e430b430a7f22d657816 on WoLpH:develop.

coveralls commented 3 years ago

Coverage Status

Coverage decreased (-100.0%) to 0.0% when pulling 8d16d55fe826875f02c677b41d6250e0fe77ab66 on bwoodsend:support-write-to-io-buffers into cded7b37be912fb3bb3dc32322c3b77cd124e5e4 on WoLpH:develop.

wolph commented 3 years ago

I don't know how I missed this... I'm really sorry about that. I just have 1 little comment about the try/except that seems to silently ignore errors. Am I missing something?

bwoodsend commented 3 years ago

OK, I've tidied up the conditional logic, removed the # pragma: no cover and provided some proper tests for them.

bwoodsend commented 3 years ago

Prod prod.

wolph commented 3 years ago

Not sure how, but your pull request is causing a segfault here ;)

I'm investigating and will release a version as soon as I've fixed and merged it :)

# py.test                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 /Volumes/workspace/numpy-stl
=========================================================================
platform darwin -- Python 3.8.1, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /Volumes/workspace/numpy-stl, configfile: pytest.ini
plugins: flake8-1.0.6, cov-2.8.1
collected 129 items

plotly.py s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            [  0%]
setup.py s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             [  1%]
test.py .                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [  2%]
stl/__init__.py .                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      [  3%]
stl/__about__.py .                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     [  3%]
stl/base.py ..                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         [  5%]
stl/main.py s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          [  6%]
stl/mesh.py s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          [  6%]
stl/stl.py .                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           [  7%]
stl/utils.py s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         [  8%]
tests/__init__.py .                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    [  9%]
tests/conftest.py s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    [ 10%]
tests/stl_corruption.py s..............                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                [ 21%]
tests/test_ascii.py .....ssFatal Python error: Segmentation fault

Current thread 0x000000010f62adc0 (most recent call first):
  File "numpy-stl/stl/stl.py", line 291 in _write_ascii
  File "numpy-stl/stl/stl.py", line 282 in save
  File "numpy-stl/tests/test_ascii.py", line 123 in test_ascii_io
  File "site-packages/_pytest/python.py", line 184 in pytest_pyfunc_call
  File "site-packages/pluggy/callers.py", line 187 in _multicall
  File "site-packages/pluggy/manager.py", line 84 in <lambda>
  File "site-packages/pluggy/manager.py", line 93 in _hookexec
  File "site-packages/pluggy/hooks.py", line 286 in __call__
  File "site-packages/_pytest/python.py", line 1627 in runtest
  File "site-packages/_pytest/runner.py", line 163 in pytest_runtest_call
  File "site-packages/pluggy/callers.py", line 187 in _multicall
  File "site-packages/pluggy/manager.py", line 84 in <lambda>
  File "site-packages/pluggy/manager.py", line 93 in _hookexec
  File "site-packages/pluggy/hooks.py", line 286 in __call__
  File "site-packages/_pytest/runner.py", line 256 in <lambda>
  File "site-packages/_pytest/runner.py", line 310 in from_call
  File "site-packages/_pytest/runner.py", line 255 in call_runtest_hook
  File "site-packages/_pytest/runner.py", line 216 in call_and_report
  File "site-packages/_pytest/runner.py", line 127 in runtestprotocol
  File "site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
  File "site-packages/pluggy/callers.py", line 187 in _multicall
  File "site-packages/pluggy/manager.py", line 84 in <lambda>
  File "site-packages/pluggy/manager.py", line 93 in _hookexec
  File "site-packages/pluggy/hooks.py", line 286 in __call__
  File "site-packages/_pytest/main.py", line 338 in pytest_runtestloop
  File "site-packages/pluggy/callers.py", line 187 in _multicall
  File "site-packages/pluggy/manager.py", line 84 in <lambda>
  File "site-packages/pluggy/manager.py", line 93 in _hookexec
  File "site-packages/pluggy/hooks.py", line 286 in __call__
  File "site-packages/_pytest/main.py", line 313 in _main
  File "site-packages/_pytest/main.py", line 257 in wrap_session
  File "site-packages/_pytest/main.py", line 306 in pytest_cmdline_main
  File "site-packages/pluggy/callers.py", line 187 in _multicall
  File "site-packages/pluggy/manager.py", line 84 in <lambda>
  File "site-packages/pluggy/manager.py", line 93 in _hookexec
  File "site-packages/pluggy/hooks.py", line 286 in __call__
  File "site-packages/_pytest/config/__init__.py", line 164 in main
  File "site-packages/_pytest/config/__init__.py", line 187 in console_main
  File "numpy-stl/bin/py.test", line 8 in <module>
zsh: segmentation fault  py.test
wolph commented 3 years ago

Can you tell me which Python version you are running? Because the tests appear to be segfaulting both for me and the CI system

bwoodsend commented 3 years ago

Hmm, everything works fine for me using any of 3.5.10, 3.6.0, 3.7.9, 3.9.0 on Linux installed using pyenv but if I run with 3.5.10 in the manylinux1 docker image I also get the seg fault. I'll keep digging...

Edit: I wasn't using the speedups. After building the extension module I get the segfault for everything.

wolph commented 3 years ago

I found the culprit. The BytesIO object has no fileno() which is crashing the speedups module :)

I think I've fixed it now

bwoodsend commented 3 years ago

Ah, you've worked it out. You're able to push your fix to this branch I believe?

wolph commented 3 years ago

I can merge it to develop so no worries