unpackAI / unpackai

The Unpack.AI library
https://www.unpackai.com
MIT License
20 stars 4 forks source link

Simplify usage of Google Drive #36

Open jfthuong opened 3 years ago

jfthuong commented 3 years ago

Is your feature request related to a problem? Please describe. Fastai uses fastbook.setup_book as an equivalent to mount Google Drive. This is weird and not very specific. Moreover, it uses the name that is not the default in the code template of Google Colab to mount (/content/drive vs /content/gdrive).

Describe the solution you'd like We could create a GoogleDrive class with different methods:

Describe alternatives you've considered An alternative is to juste create a mount_gdrive function that could return the Path of mounted Google Drive.

Additional context N/A

raynardj commented 3 years ago

I'm more up for

create a mount_gdrive function that could return the Path of mounted Google Drive.

As write up entire file system commandlines in python functions are

raynardj commented 3 years ago

so the function will do the following code?

def mount_gdrive(path='/content/drive'):
    from google.colab import drive
    drive.mount(path)
vtecftwy commented 3 years ago

Good points from @raynardj Also, fastai has added a ls() method to pathlib.Path, and would make the new class ls method kind of redundant.

raynardj commented 3 years ago

Also, fastai has added a ls() method to pathlib.Path, and would make the new class ls method kind of redundant.

Love that feature.

jfthuong commented 3 years ago

so the function will do the following code?

def mount_gdrive(path='/content/drive'):
    from google.colab import drive
    drive.mount(path)

Kind of. We shall also handle the case where we are not in Google Drive, avoid trying to mount twice, and maybe storing where it has been mounted (just few more line of codes :) ).

Also, fastai has added a ls() method to pathlib.Path, and would make the new class ls method kind of redundant.

Actually they added more than just ls:


# fastcore/xtras.py

# Cell
@patch
def readlines(self:Path, hint=-1, encoding='utf8'):
    "Read the content of `self`"
    with self.open(encoding=encoding) as f: return f.readlines(hint)

# Cell
@patch
def read_json(self:Path, encoding=None, errors=None):
    "Same as `read_text` followed by `loads`"
    return loads(self.read_text(encoding=encoding, errors=errors))

# Cell
@patch
def mk_write(self:Path, data, encoding=None, errors=None, mode=511):
    "Make all parent dirs of `self`, and write `data`"
    self.parent.mkdir(exist_ok=True, parents=True, mode=mode)
    self.write_text(data, encoding=encoding, errors=errors)

# Cell
@patch
def relpath(self:Path, start=None):
    "Same as `os.path.relpath`, but returns a `Path`, and resolves symlinks"
    return Path(os.path.relpath(self.resolve(), Path(start).resolve()))

# Cell
@patch
def ls(self:Path, n_max=None, file_type=None, file_exts=None):
    "Contents of path as a list"
    extns=L(file_exts)
    if file_type: extns += L(k for k,v in mimetypes.types_map.items() if v.startswith(file_type+'/'))
    has_extns = len(extns)==0
    res = (o for o in self.iterdir() if has_extns or o.suffix in extns)
    if n_max is not None: res = itertools.islice(res, n_max)
    return L(res)

# Cell
@patch
def __repr__(self:Path):
    b = getattr(Path, 'BASE_PATH', None)
    if b:
        try: self = self.relative_to(b)
        except: pass
    return f"Path({self.as_posix()!r})"

# Cell
@patch
def delete(self:Path):
    "Delete a file, symlink, or directory tree"
    if not self.exists(): return
    if self.is_dir(): shutil.rmtree(self)
    else: self.unlink()