A security tool designed to perform thorough scans on a target using OpenVAS, Zap, and Nexpose. It seamlessly consolidates and integrates the scan results, providing a comprehensive overview of the security vulnerabilities identified.
The vulnerability is present in PunktSentenceTokenizer, sent_tokenize and word_tokenize. Any users of this class, or these two functions, are vulnerable to a Regular Expression Denial of Service (ReDoS) attack.
In short, a specifically crafted long input to any of these vulnerable functions will cause them to take a significant amount of execution time. The effect of this vulnerability is noticeable with the following example:
from nltk.tokenize import word_tokenize
n = 8
for length in [10**i for i in range(2, n)]:
# Prepare a malicious input
text = "a" * length
start_t = time.time()
# Call `word_tokenize` and naively measure the execution time
word_tokenize(text)
print(f"A length of {length:<{n}} takes {time.time() - start_t:.4f}s")
Which gave the following output during testing:
A length of 100 takes 0.0060s
A length of 1000 takes 0.0060s
A length of 10000 takes 0.6320s
A length of 100000 takes 56.3322s
...
I canceled the execution of the program after running it for several hours.
If your program relies on any of the vulnerable functions for tokenizing unpredictable user input, then we would strongly recommend upgrading to a version of NLTK without the vulnerability, or applying the workaround described below.
Patches
The problem has been patched in NLTK 3.6.6. After the fix, running the above program gives the following result:
A length of 100 takes 0.0070s
A length of 1000 takes 0.0010s
A length of 10000 takes 0.0060s
A length of 100000 takes 0.0400s
A length of 1000000 takes 0.3520s
A length of 10000000 takes 3.4641s
This output shows a linear relationship in execution time versus input length, which is desirable for regular expressions.
We recommend updating to NLTK 3.6.6+ if possible.
Workarounds
The execution time of the vulnerable functions is exponential to the length of a malicious input. With other words, the execution time can be bounded by limiting the maximum length of an input to any of the vulnerable functions. Our recommendation is to implement such a limit.
The nltk package is vulnerable to ReDoS (regular expression denial of service). An attacker that is able to provide as an input to the _read_comparison_block() function in the file nltk/corpus/reader/comparative_sents.py may cause an application to consume an excessive amount of CPU.
NLTK through 3.8.1 allows remote code execution if untrusted packages have pickled Python code, and the integrated data package download functionality is used. This affects, for example, averaged_perceptron_tagger and punkt.
This PR contains the following updates:
==3.4.5
->==3.9
GitHub Vulnerability Alerts
CVE-2021-43854
Impact
The vulnerability is present in
PunktSentenceTokenizer
,sent_tokenize
andword_tokenize
. Any users of this class, or these two functions, are vulnerable to a Regular Expression Denial of Service (ReDoS) attack. In short, a specifically crafted long input to any of these vulnerable functions will cause them to take a significant amount of execution time. The effect of this vulnerability is noticeable with the following example:Which gave the following output during testing:
I canceled the execution of the program after running it for several hours.
If your program relies on any of the vulnerable functions for tokenizing unpredictable user input, then we would strongly recommend upgrading to a version of NLTK without the vulnerability, or applying the workaround described below.
Patches
The problem has been patched in NLTK 3.6.6. After the fix, running the above program gives the following result:
This output shows a linear relationship in execution time versus input length, which is desirable for regular expressions. We recommend updating to NLTK 3.6.6+ if possible.
Workarounds
The execution time of the vulnerable functions is exponential to the length of a malicious input. With other words, the execution time can be bounded by limiting the maximum length of an input to any of the vulnerable functions. Our recommendation is to implement such a limit.
References
For more information
If you have any questions or comments about this advisory:
CVE-2021-3842
NLTK is vulnerable to REDoS in some RegexpTaggers used in the functions
get_pos_tagger
andmalt_regex_tagger
.CVE-2021-3828
The nltk package is vulnerable to ReDoS (regular expression denial of service). An attacker that is able to provide as an input to the
_read_comparison_block()
function in the filenltk/corpus/reader/comparative_sents.py
may cause an application to consume an excessive amount of CPU.CVE-2024-39705
NLTK through 3.8.1 allows remote code execution if untrusted packages have pickled Python code, and the integrated data package download functionality is used. This affects, for example, averaged_perceptron_tagger and punkt.
Release Notes
nltk/nltk (nltk)
### [`v3.9`](https://togithub.com/nltk/nltk/compare/3.8.1...3.9) [Compare Source](https://togithub.com/nltk/nltk/compare/3.8.1...3.9) ### [`v3.8.1`](https://togithub.com/nltk/nltk/compare/3.8...3.8.1) [Compare Source](https://togithub.com/nltk/nltk/compare/3.8...3.8.1) ### [`v3.8`](https://togithub.com/nltk/nltk/compare/3.7...3.8) [Compare Source](https://togithub.com/nltk/nltk/compare/3.7...3.8) ### [`v3.7`](https://togithub.com/nltk/nltk/compare/3.6.7...3.7) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6.7...3.7) ### [`v3.6.7`](https://togithub.com/nltk/nltk/compare/3.6.6...3.6.7) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6.6...3.6.7) ### [`v3.6.6`](https://togithub.com/nltk/nltk/compare/3.6.5...3.6.6) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6.5...3.6.6) ### [`v3.6.5`](https://togithub.com/nltk/nltk/compare/3.6.4...3.6.5) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6.4...3.6.5) ### [`v3.6.4`](https://togithub.com/nltk/nltk/compare/3.6.3...3.6.4) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6.3...3.6.4) ### [`v3.6.3`](https://togithub.com/nltk/nltk/compare/3.6.2...3.6.3) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6.2...3.6.3) ### [`v3.6.2`](https://togithub.com/nltk/nltk/compare/3.6.1...3.6.2) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6.1...3.6.2) ### [`v3.6.1`](https://togithub.com/nltk/nltk/compare/3.6...3.6.1) [Compare Source](https://togithub.com/nltk/nltk/compare/3.6...3.6.1) ### [`v3.6`](https://togithub.com/nltk/nltk/compare/3.5...3.6) [Compare Source](https://togithub.com/nltk/nltk/compare/3.5...3.6) ### [`v3.5`](https://togithub.com/nltk/nltk/compare/3.4.5...3.5) [Compare Source](https://togithub.com/nltk/nltk/compare/3.4.5...3.5)Configuration
đ Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
đŠ Automerge: Disabled by config. Please merge this manually once you are satisfied.
â» Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
đ Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.