stanfordnlp / stanza

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages
https://stanfordnlp.github.io/stanza/
Other
7.27k stars 891 forks source link

FutureWarning: You are using `torch.load` with `weights_only=False` #1429

Open mskaif opened 3 hours ago

mskaif commented 3 hours ago

Describe the bug FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. state = torch.load(filename, lambda storage, loc: storage)

This warning is triggered by all torch.load used in stanza. The issue does not cause any problem with data processing at the moment but the long warnings are distracting.

To Reproduce Steps to reproduce the behavior:

  1. upgrade torch to 2.4.1

Expected behavior no error

Environment (please complete the following information):

mskaif commented 3 hours ago

The error can be suppressed by using the following before calling stanza functions but is not a solution

import warnings warnings.simplefilter(action='ignore', category=FutureWarning)

source: https://github.com/ultralytics/ultralytics/issues/14994#issuecomment-2364356239

AngledLuffa commented 2 hours ago

Aware of it. There's a limitation where we are saving plenty of things other than weights in the current file. Config strings and numbers, mostly. Would those still work?

On Tue, Oct 22, 2024, 10:19 PM mskaif @.***> wrote:

The error can be suppressed by using the following before calling stanza functions but is not a solution

import warnings warnings.simplefilter(action='ignore', category=FutureWarning)

source: ultralytics/ultralytics#14994 (comment) https://github.com/ultralytics/ultralytics/issues/14994#issuecomment-2364356239

— Reply to this email directly, view it on GitHub https://github.com/stanfordnlp/stanza/issues/1429#issuecomment-2430930006, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2AYWPIB3WEPPBSGN337ALZ44WXBAVCNFSM6AAAAABQN6FUXGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZQHEZTAMBQGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>