uktrade / stream-read-xbrl

Python package to parse Companies House accounts data in a streaming way
https://stream-read-xbrl.docs.trade.gov.uk/
MIT License
17 stars 2 forks source link

Getting an Error #145

Closed IRISDavidD closed 11 months ago

IRISDavidD commented 1 year ago

RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Any idea on a fix for this?

michalc commented 11 months ago

It might be that you're running code in the top level rather than only if __name__ == '__main__. This gives an error like the one reported:

import datetime
from stream_read_xbrl import stream_read_xbrl_sync

with stream_read_xbrl_sync(datetime.date(2013, 8, 31)) as (columns, date_range_and_rows):
    for ((start_date, end_date), rows) in date_range_and_rows:
        print(start_date, end_date)
        for row in rows:
           pass

But this doesn't:

import datetime
from stream_read_xbrl import stream_read_xbrl_sync

if __name__ == '__main__':
    with stream_read_xbrl_sync(datetime.date(2013, 8, 31)) as (columns, date_range_and_rows):
        for ((start_date, end_date), rows) in date_range_and_rows:
            print(start_date, end_date)
            for row in rows:
               pass

This is required due to multiprocessing behaviour that stream-read-xbrl uses under the hood.

If this doesn't solve it - can you post the code you're running, enough so we can reproduce the issue at our end? Ideally a minimum reproducible example, StackOverflow-style

This is admittedly not obvious from the docs at all - will see if can improve them.

michalc commented 11 months ago

Think this is explained by the change in docs. If no, feel free to re-open