mixpanel / mixpanel-utils

Other
85 stars 62 forks source link

FileNotFoundError: [Errno 2] No such file or directory #59

Open lenfree opened 4 months ago

lenfree commented 4 months ago

I am getting a mixed results with export_people() function. Would it be possible to include additional information about the the file error? Otherwise, I would assume it's related to the destination file? Two runs were fine but succeeding ones were failing with below stacktrace.

Python version: 3.12 Environment: Lambda, x86 arch

[ERROR] FileNotFoundError: [Errno 2] No such file or directory
Traceback (most recent call last):
  File "/var/task/index.py", line 104, in lambda_handler
    df_people = fetch_mixpanel_people(
  File "/var/task/index.py", line 22, in fetch_mixpanel_people
    client.export_people(
  File "/var/task/mixpanel_utils/__init__.py", line 1426, in export_people
    profiles = self.query_engage(params, timezone_offset=timezone_offset)
  File "/var/task/mixpanel_utils/__init__.py", line 1299, in query_engage
    return engage_paginator.fetch_all(params)
  File "/var/task/mixpanel_utils/paginator.py", line 69, in fetch_all
    return results + self._concurrent_flatmap(fetcher, list(range(start, end)))
  File "/var/task/mixpanel_utils/paginator.py", line 79, in _concurrent_flatmap
    pool = ThreadPool(processes=self.concurrency)
  File "/var/lang/lib/python3.12/multiprocessing/pool.py", line 930, in __init__
    Pool.__init__(self, processes, initializer, initargs)
  File "/var/lang/lib/python3.12/multiprocessing/pool.py", line 196, in __init__
    self._change_notifier = self._ctx.SimpleQueue()
  File "/var/lang/lib/python3.12/multiprocessing/context.py", line 113, in SimpleQueue
    return SimpleQueue(ctx=self.get_context())
  File "/var/lang/lib/python3.12/multiprocessing/queues.py", line 363, in __init__
    self._rlock = ctx.Lock()
  File "/var/lang/lib/python3.12/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/var/lang/lib/python3.12/multiprocessing/synchronize.py", line 169, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/var/lang/lib/python3.12/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(END RequestId: 142e20e9-6070-44c7-9384-d0a6c29ee3ac
lenfree commented 4 months ago

I think it may be related to https://www.reddit.com/r/learnpython/comments/q3v1d6/is_there_a_library_to_do_multiprocessing_without/.