Error when executing example

[KilianHohm]

When I try to run the first MarkDown example (https://github.com/thombashi/pytablewriter#write-tables) I get the following error:

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

So I added the import statement

from multiprocessing import freeze_support

and moved the code into the main part:

if __name__ == '__main__':
    freeze_support()
    ...

This solved the initial problem and prints the table as markdown on the console. However now I get the following error:

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\process.py", line 101, in _python_exit
    thread_wakeup.wakeup()
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\process.py", line 89, in wakeup
    self._writer.send_bytes(b"")
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\multiprocessing\connection.py", line 183, in send_bytes
    self._check_closed()
  File "C:\Users\hohm\AppData\Local\Programs\Python\Python37\lib\multiprocessing\connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

I am running the example on a Windows 10 machine, with Python version 3.7.3 and pytablewriter version 0.46.1

[thombashi]

@KilianHohm Thank you for your report.

I could successfully execute the example at my environment (same with yours: Windows 10, Python 3.7.3 and pytablewriter 0.46.1). Could you provide me the following information to understand the situation:

1. The entire source code that you executed
2. Installed dependency package versions of pytablewriter
   • Following is my environment

pytablewriter==0.46.1
  DataProperty==0.43.1
  mbstrdecoder==0.8.1
  msgfy==0.0.7
  pathvalidate==0.29.0
  six==1.12.0
  tabledata==0.9.1
  typepy==0.6.0

[KilianHohm]

@thombashi Thanks for the quick response.

The first error appears with this code (started from VS Code):

from pytablewriter import MarkdownTableWriter

writer = MarkdownTableWriter()
writer.table_name = "example_table"
writer.headers = ["int", "float", "str", "bool", "mix", "time"]
writer.value_matrix = [
    [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
    [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 45:01:23+0900"],
    [3,   0,        "bar",  "true",  "inf", "2017-03-03 33:44:55+0900"],
    [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
]

writer.write_table()

The second warning is reproducible with this code (again executed from within VS Code):

from pytablewriter import MarkdownTableWriter
from multiprocessing import freeze_support

if __name__ == '__main__':
    freeze_support()
    writer = MarkdownTableWriter()
    writer.table_name = "example_table"
    writer.headers = ["int", "float", "str", "bool", "mix", "time"]
    writer.value_matrix = [
        [0,   0.1,      "hoge", True,   0,      "2017-01-01 03:04:05+0900"],
        [2,   "-2.23",  "foo",  False,  None,   "2017-12-23 45:01:23+0900"],
        [3,   0,        "bar",  "true",  "inf", "2017-03-03 33:44:55+0900"],
        [-10, -9.9,     "",     "FALSE", "nan", "2017-01-01 00:00:00+0900"],
    ]

    writer.write_table()

During writing this comment I noticed that I never tried to execute this outside VS Code. So I tried it in Windows Powershell. There I only get the first error (missing main and freeze_support()), the second program code runs through without any warning.

Here is the dependecy tree generated with pipdeptree:

pytablewriter==0.46.1
  - DataProperty [required: >=0.43.1,<1.0.0, installed: 0.43.1]
    - mbstrdecoder [required: >=0.8.0,<1.0.0, installed: 0.8.1]
      - setuptools [required: >=38.3.0, installed: 40.8.0]
    - setuptools [required: >=38.3.0, installed: 40.8.0]
    - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]
    - typepy [required: >=0.6.0,<1.0.0, installed: 0.6.0]
      - mbstrdecoder [required: >=0.8.0,<1.0.0, installed: 0.8.1]
        - setuptools [required: >=38.3.0, installed: 40.8.0]
      - setuptools [required: >=38.3.0, installed: 40.8.0]
      - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]
  - mbstrdecoder [required: >=0.8.0,<1.0.0, installed: 0.8.1]
    - setuptools [required: >=38.3.0, installed: 40.8.0]
  - msgfy [required: >=0.0.6,<0.1.0, installed: 0.0.7]
    - setuptools [required: >=38.3.0, installed: 40.8.0]
  - pathvalidate [required: >=0.28.0,<1.0.0, installed: 0.28.2]
    - setuptools [required: >=38.3.0, installed: 40.8.0]
  - setuptools [required: >=38.3.0, installed: 40.8.0]
  - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]
  - tabledata [required: >=0.9.1,<1.0.0, installed: 0.9.1]
    - DataProperty [required: >=0.43.1,<1.0.0, installed: 0.43.1]
      - mbstrdecoder [required: >=0.8.0,<1.0.0, installed: 0.8.1]
        - setuptools [required: >=38.3.0, installed: 40.8.0]
      - setuptools [required: >=38.3.0, installed: 40.8.0]
      - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]
      - typepy [required: >=0.6.0,<1.0.0, installed: 0.6.0]
        - mbstrdecoder [required: >=0.8.0,<1.0.0, installed: 0.8.1]
          - setuptools [required: >=38.3.0, installed: 40.8.0]
        - setuptools [required: >=38.3.0, installed: 40.8.0]
        - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]
    - setuptools [required: >=38.3.0, installed: 40.8.0]
    - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]
    - typepy [required: >=0.6.0,<1.0.0, installed: 0.6.0]
      - mbstrdecoder [required: >=0.8.0,<1.0.0, installed: 0.8.1]
        - setuptools [required: >=38.3.0, installed: 40.8.0]
      - setuptools [required: >=38.3.0, installed: 40.8.0]
      - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]
  - typepy [required: >=0.6.0,<1.0.0, installed: 0.6.0]
    - mbstrdecoder [required: >=0.8.0,<1.0.0, installed: 0.8.1]
      - setuptools [required: >=38.3.0, installed: 40.8.0]
    - setuptools [required: >=38.3.0, installed: 40.8.0]
    - six [required: >=1.10.0,<2.0.0, installed: 1.12.0]

The only difference I can see there is pathvalidate

[enjoy-binbin]

I got the same error when i run the example code in sublime2 example_excel

My env is py3.6 and win7 x64，and the other package were based on pip install pytablewriter and pip install pytablewriter[excel]

But when i ran the code in jupyter notebook. It worked.

[Zackhardtoname]

After googling around, I found a fix on SO. https://stackoverflow.com/questions/15900366/all-example-concurrent-futures-code-is-failing-with-brokenprocesspool Quotes:

Under Windows, it is important to protect the main loop of code to avoid recursive spawning of subprocesses when using processpoolexecutor or any other parallel code which spawns new processes.

Basically, all your code which creates new processes must be under if name == 'main': , for the same reason you cannot execute it in interpreter.

[Zackhardtoname]

I'd love to try to fix it and make a pull request, but I am unsure if the maintainers of this repo are already working on it.

[thombashi]

main() guard and freeze_support could not work around the issue when execute by VS code debugger.

@Zackhardtoname A PR would be welcome.

[thombashi]

You can avoid the problem by set max_workers to one as follows:

writer.max_workers = 1

The property available pytablewriter 0.49.0 or later.