[Discussion] folder from new context

[gjeusel]

Avant-Propos

Hello there !

I pretty much love this library idea, and it would fit my use case perfectly !

Now I was just wondering about a few issues I encountered while testing it.

Chain calls of context:

import s3monkey

folder = "/tmp/foldername"
fpath = f"{folder}/somefile.txt"
s3_ctx = s3monkey.S3FS(bucket="s3monkey-testing", mount_point="/tmp")

with s3_ctx as fs:
    pass

with s3_ctx as fs:
    pass

Here is the traceback:

Traceback (most recent call last):
  File "/tmp/sample.py", line 14, in <module>
    with s3_ctx as fs:
  File "/Users/gjeusel/src/s3monkey/s3monkey/core.py", line 122, in __enter__
    self.patcher.setUp()
  File "/Users/gjeusel/src/s3monkey/s3monkey/pyfakefs/fake_filesystem_unittest.py", line 403, in setUp
    temp_dir = tempfile.gettempdir()
  File "/Users/gjeusel/miniconda3/envs/squamish/lib/python3.7/tempfile.py", line 294, in gettempdir
    tempdir = _get_default_tempdir()
  File "/Users/gjeusel/miniconda3/envs/squamish/lib/python3.7/tempfile.py", line 209, in _get_default_tempdir
    with _io.open(fd, 'wb', closefd=False) as fp:
  File "/Users/gjeusel/src/s3monkey/s3monkey/pyfakefs/fake_filesystem.py", line 3995, in open
    file_path, mode, buffering, encoding, errors, newline, closefd, opener)
  File "/Users/gjeusel/src/s3monkey/s3monkey/pyfakefs/fake_filesystem.py", line 4445, in __call__
    return self.call(*args, **kwargs)
  File "/Users/gjeusel/src/s3monkey/s3monkey/pyfakefs/fake_filesystem.py", line 4559, in call
    fakefile_class, key = self.filesystem.open_callback(locals())
  File "/Users/gjeusel/src/s3monkey/s3monkey/core.py", line 154, in open_callback
    if 'w' in locals['mode'] and locals['file_'].startswith(self.mount_point):
AttributeError: 'int' object has no attribute 'startswith'

Path existence while having a folder in it:

While this works just perfectly:

import os
import s3monkey

folder = "/tmp/foldername"
fpath = f"{folder}/somefile.txt"
s3_ctx = s3monkey.S3FS(bucket="s3monkey-testing", mount_point="/tmp")

with s3_ctx as fs:
    os.mkdir(folder)

    with open(fpath, "w") as f:
        f.write("Hello")

    with open(fpath, "r") as f:
        print(f.readline())

    assert os.path.exists(fpath)

If I were to execute the following after having run the previous one - meaning after having created this S3 object and check it does exist in my S3 bucket -, this would fail:

import os
import s3monkey

folder = "/tmp/foldername"
fpath = f"{folder}/somefile.txt"
s3_ctx = s3monkey.S3FS(bucket="s3monkey-testing", mount_point="/tmp")

with s3_ctx as fs:
    assert os.path.exists(fpath)

Environment

• Python 3.7.9
• s3monkey: tip of master branch

Is this repository maintained ?

Also, if you need any more informations please don't hesitate !

Cordially,

[kobayashi]

Thank you for open this issue. I would like to make sure if your mount_point and folder are same?

[gjeusel]

Nop, they aren't ! mount_point="/tmp" folder="/tmp/foldername"

The purpose is to highlight an issue with files contained in a "sub-directory"