plasticityai / magnitude

A fast, efficient universal vector embedding utility package.
MIT License
1.62k stars 118 forks source link

[FEATURE REQUEST] Show download progress when downloading magnitude files from the server #44

Open ParikhKadam opened 5 years ago

ParikhKadam commented 5 years ago

Magnitude(MagnitudeUtils.download_model('glove/medium/glove.6B.{}d'.format(self.glove_dim), download_dir=os.path.join(base_dir, 'magnitude')), case_insensitive=True)

The above piece of code, when downloading data from server, isn't showing progress to users. It feels like the command prompt is not doing any task when the script runs. This can be easily done through tqdm. This feature will be of great help.

Thank you..

ParikhKadam commented 5 years ago

You can just change this code:

urlretrieve(
                    remote_file_path,
                    os.path.join(download_dir, local_file_name_tmp)
)

as:

    class DownloadProgressBar(tqdm):
        def update_to(self, b=1, bsize=1, tsize=None):
            """
            b: int, optional
                Number of blocks just transferred [default: 1].
            bsize: int, optional
                Size of each block (in tqdm units) [default: 1].
            tsize: int, optional
                Total size (in tqdm units). If [default: None] remains unchanged.
            """
            if tsize is not None:
                self.total = tsize
            self.update(b * bsize - self.n)

            with DownloadProgressBar(unit='B', unit_scale=True, miniters=1, desc=filename) as t:
                local_filename, _ = urlretrieve(remote_file_path,
                                               filename=local_file_name_tmp,
                                               reporthook=t.update_to)

Here is your code - https://github.com/plasticityai/magnitude/blob/aec98628b5547773ca8c4114ec6d1ad51e21b230/pymagnitude/__init__.py#L2152

I ain't a contributor here.. so I need to fork this repo, make changes and then create a new pull request which seems a lot of work. You can just change a few lines in order to make this work @AjayP13