cvqluu / simple_diarizer

Simplified diarization pipeline using some pretrained models - audio file to diarized segments in a few lines of code
GNU General Public License v3.0
141 stars 27 forks source link

"[Errno 30] Read-only file system: 'pretrained_models'" #9

Closed MrEdwards007 closed 1 year ago

MrEdwards007 commented 2 years ago

I am using macOS and I am getting error "[Errno 30] Read-only file system: 'pretrained_models'" From what I can tell, the pretrained models are being fetched if you do not have them.

However, the save location is the root directory which is read-only. This is where I believe is the target directory "./pretrained_model_checkpoints"

Is there another location that can be used that can be used?

PythonKit/Python.swift:706: Fatal error: 'try!' expression unexpectedly raised an error: Python exception: [Errno 30] Read-only file system: 'pretrained_models' Traceback: File "/Users/wedwards/Documents/Development/A_PythonKit_Test/A_PythonKit_Test/Simple Diarizer.py", line 42, in diar = Diarizer( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/simple_diarizer/diarizer.py", line 48, in init self.embed_model = EncoderClassifier.from_hparams( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/speechbrain/pretrained/interfaces.py", line 342, in from_hparams hparams_local_path = fetch( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/speechbrain/pretrained/fetching.py", line 86, in fetch savedir.mkdir(parents=True, exist_ok=True) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 1179, in mkdir self.parent.mkdir(parents=True, exist_ok=True) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 1175, in mkdir self._accessor.mkdir(self, mode)

2022-11-11 13:14:00.531470-0500 A_PythonKit_Test[69382:7584330] PythonKit/Python.swift:706: Fatal error: 'try!' expression unexpectedly raised an error: Python exception: [Errno 30] Read-only file system: 'pretrained_models' Traceback: File "/Users/wedwards/Documents/Development/A_PythonKit_Test/A_PythonKit_Test/Simple Diarizer.py", line 42, in diar = Diarizer( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/simple_diarizer/diarizer.py", line 48, in init self.embed_model = EncoderClassifier.from_hparams( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/speechbrain/pretrained/interfaces.py", line 342, in from_hparams hparams_local_path = fetch( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/speechbrain/pretrained/fetching.py", line 86, in fetch savedir.mkdir(parents=True, exist_ok=True) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 1179, in mkdir self.parent.mkdir(parents=True, exist_ok=True) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 1175, in mkdir self._accessor.mkdir(self, mode)

cvqluu commented 2 years ago

Hi, is it possible for you to fix this by using chmod on this directory?

Specifically, something like

chmod 755 -R ./pretrained_models
MrEdwards007 commented 2 years ago

I would say no, as it is not advised to create a directory in the root of a macOS drive. Folders should be created in the users directory, which are not shared with other users.

I searched through the modules and I identified a workaround. I am sure this is incorrect but this is what I did to get the program to work in diarizer.py I am new to Python and also new to this type of program, so my workaround is likely ill-advised. I created a folder called "pretrained_models" in my user directory and made the following modification.

""" /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/simple_diarizer/diarizer.py if embed_model == "ecapa": self.embed_model = EncoderClassifier.from_hparams( source="speechbrain/spkrec-ecapa-voxceleb",

savedir="pretrained_models/spkrec-ecapa-voxceleb",

savedir="/Users/wedwards/pretrained_models/spkrec-ecapa-voxceleb", <------ THIS IS THE LINE THAT WAS CHANGED run_opts=self.run_opts, ) """

Thank you for your help.

cvqluu commented 2 years ago

Hi, to be clear, I didn't mean do this on the root directory, but on the working directory you were running this from. (Hence the "." in front of ./pretrained_models). Looking at the logs this would be /Users/wedwards/Documents/Development/A_PythonKit_Test/A_PythonKit_Test/pretrained_models, assuming you are executing your script from this directory also.

The default behaviour of the saving should be to make the ./pretrained_models directory in the working directory you are running your script from (you can see this behaviour by playing around in the colab notebook). It might be that executing your code from a different directory will work, since you don't seem to have write permissions.

MrEdwards007 commented 2 years ago

Unfortunately, I am not comprehending. The writing directory for me does not exist. I have provided the directory creation attempts below

Is the the line in diarizer.py supposed to read as savedir="./pretrained_models/spkrec-ecapa-voxceleb" ? That is not what is present. When I was troubleshooting the issue, to get to a point where I could determine what was occurring, I put a print statement in the pathlib.py to watch the directory creations

    def mkdir(self, mode=0o777, parents=False, exist_ok=False):
        """
        Create a new directory at this given path.
        """
        print(f"MKDIR ATTEMPT -- self, {self}") #<---- ADDED THIS LINE

        try:
            self._accessor.mkdir(self, mode)
        except FileNotFoundError:
            if not parents or self.parent == self:
                raise
            self.parent.mkdir(parents=True, exist_ok=True)
            self.mkdir(mode, parents=False, exist_ok=exist_ok)
        except OSError:
            # Cannot rely on checking for EEXIST, since the operating system
            # could give priority to other errors like EACCES or EROFS
            if not exist_ok or not self.is_dir():
                raise

The outcome was MKDIR ATTEMPT -- self, /Users/wedwards/.matplotlib MKDIR ATTEMPT -- self, /Users/wedwards/.matplotlib MKDIR ATTEMPT -- self, pretrained_models/spkrec-ecapa-voxceleb MKDIR ATTEMPT -- self, pretrained_models

The folder creation attempts were "/Users/wedwards/.matplotlib" <- Folder does exist as a hidden folder "pretrained_models/spkrec-ecapa-voxceleb" <- cant exist because there isn't a reference to an existing directory "pretrained_models" <- cant exist because there isn't a reference to an existing directory

Not knowing what the correct way to resolve the problem, my workaround was to modify diarizer.py, which is located in for me in the following directory.

/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/simple_diarizer/diarizer.py

        if embed_model == "ecapa":
            self.embed_model = EncoderClassifier.from_hparams(
                source="speechbrain/spkrec-ecapa-voxceleb",
#                savedir="pretrained_models/spkrec-ecapa-voxceleb", # <-- ORIGINAL
                  savedir="/Users/wedwards/pretrained_models/spkrec-ecapa-voxceleb", # <-- CHANGED
                run_opts=self.run_opts,
            )

Hopefully the printout of the directory creation attempts will help resolve the issue.

cvqluu commented 1 year ago

Hi, it seems like this is a general write permissions issue with your setup.

I'm glad you were able to fix it, but since it works on colab, and on my machine, I'm going to assume it might just be a quirk with your setup in some way.

Please raise this again if you find other issues relating to write permissions.