search

Leibniz-HBI / dabapush

Data Base pusher for social media data (Twitter for the beginning) – pre-alpha version
https://pypi.org/project/dabapush/
MIT License
0 stars 0 forks source link

bug: attempting to read closed file #38

Closed pekasen closed 1 year ago

pekasen commented 1 year ago 
Traceback (most recent call last):
  File "/home/ec2-user/.local/bin/dabapush", line 8, in <module>
    sys.exit(cli())
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/dabapush/run_subcommand.py", line 36, in run
    db.jb_run(targets)
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/dabapush/Dabapush.py", line 194, in jb_run
    self.__dispatch_job__(target)
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/dabapush/Dabapush.py", line 208, in __dispatch_job__
    writer.write(reader.read())
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/dabapush/Writer/Writer.py", line 31, in write
    for item in queue:
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/dabapush/Reader/TwacapicReader.py", line 127, in read
    for res in _res:
  File "/home/ec2-user/.local/pipx/venvs/dabapush/lib64/python3.8/site-packages/dabapush/Reader/TwacapicReader.py", line 124, in <genexpr>
    _res = (loads(line) for line in file)
ValueError: I/O operation on closed file.
Session and Connection Terminated
pekasen commented 1 year ago

https://github.com/Leibniz-HBI/dabapush/blob/553d3f75429627875efdb41f7c0def952485758b/dabapush/Reader/TwacapicReader.py#L124

Since we do this with a generator, we have to ensure that the file is not closed.