Google Colab Error: "UnsupportedProtocol: protocol 'fat' is not supported"

I am trying to use pyfatfs in Google Colab.

See: https://colab.research.google.com/drive/1vV77OL3aEq3liEAihNOB8AE17GbhZ1dI?usp=sharing

I can get the tests to run (and pass, mostly -- i assume due to uimplemented features). But when I try to execute example:

!pip install pyfatfs !pip install fs

import fs my_fs = fs.open_fs("fat:///dev/core")

I get an error:

UnsupportedProtocol Traceback (most recent call last)

in () 1 import fs ----> 2 my_fs = fs.open_fs("fat:///dev/core") 2 frames /usr/local/lib/python3.6/dist-packages/fs/opener/registry.py in get_opener(self, protocol) 126 else: 127 raise UnsupportedProtocol( --> 128 "protocol '{}' is not supported".format(protocol) 129 ) 130 UnsupportedProtocol: protocol 'fat' is not supported (the baseline google colab file system which I've used for this example doesn't have any FAT file systems, so I'm expecting an error. But this error seems to occur much before that -- not even recognizing the 'fat:///' prefix. I get the same error message in a private environment pointing to a FAT16 file image. Seems like I'm missing something obvious :(

Interesting - at least I (obviously) cannot reproduce this locally:

It seems to me that the way in which PyFilesystem2 searches for extensions does not correctly work with Google Colab:

    @property
    def protocols(self):
        # type: () -> List[Text]
        """`list`: the list of supported protocols."""
        _protocols = list(self._protocols)
        if self.load_extern:
            _protocols.extend(
                entry_point.name
                for entry_point in pkg_resources.iter_entry_points("fs.opener")
            )
            _protocols = list(collections.OrderedDict.fromkeys(_protocols))
        return _protocols

https://github.com/PyFilesystem/pyfilesystem2/blob/8784fd61797fac654511f16fd50c38c8a1675f16/fs/opener/registry.py#L88

And I did indeed find a forum post by someone who experienced similar issues:

https://forum.qiime2.org/t/discovering-plugins-via-importlib-instead-of-pkg-resources/18358

I've printed the output of both importlib and pkg_resources directly in colab, yielding the following results:

>>> import pkg_resources
>>> print(list(pkg_resources.iter_entry_points(group="fs.opener")))
[]

>>> from importlib_metadata import entry_points
>>> print(entry_points().get("fs.opener", []))
(EntryPoint(name='fat', value='pyfatfs.PyFatFSOpener:PyFatFSOpener', group='fs.opener'),)

If we hack the fs.opener.registry._protocols dictionary we are able to get PyFilesystem to properly recognize the FS (but fail ultimately since this is not the proper instance type for the dictionary element):

If you don't rely on PyFilesystem2's opener and instead directly instantiate PyFatFS it works flawlessly:

Here's the link to my Colab:

https://colab.research.google.com/drive/12LJbNDwiYgm7ondxJd8Hfe9H2csB0i-Z?usp=sharing

I'd recommend you to open up an issue with the PyFilesystem2 project instead as it is not caused by pyfatfs and instead can be reproduced with other extensions as well and is caused by the way in which PyFilesystem2 loads extensions that are not built-in and instead installable via pip.

Incidentally there's already a request to move to importlib, but it is planned for 3.0: PyFilesystem/pyfilesystem2#356

nathanhi / pyfatfs

Google Colab Error: "UnsupportedProtocol: protocol 'fat' is not supported" #10

I get an error: