tchaikov
commented
10 months ago tested using:
$ JAVA_HOME=/usr/lib/jvm/jre-11 pytest -c $PWD/pytest.ini --cassandra-dir $HOME/dev/scylladb/build/cmake offline_tools_test.py::TestOfflineTools::test_sstablelevelreset --count 20 --keep-test-dir
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.11.4, pytest-7.3.1, pluggy-1.0.0
Using --randomly-seed=2281201837
rootdir: /home/kefu/dev/scylla-dtest
configfile: pytest.ini
plugins: timeout-1.4.2, subtests-0.5.0, repeat-0.9.1, randomly-3.5.0, metadata-2.0.4, forked-1.6.0, elk-reporter-0.2.1, xdist-2.2.0, flake8-1.1.1, html-3.2.0, typeguard-3.0.2, anyio-3.7.1, asyncio-0.20.3
timeout: 2700.0s
timeout method: signal
timeout func_only: False
asyncio: mode=Mode.STRICT
collected 20 items
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[8-20] PASSED [ 5%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[3-20] PASSED [ 10%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[6-20] PASSED [ 15%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[12-20] PASSED [ 20%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[10-20] PASSED [ 25%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[18-20] PASSED [ 30%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[16-20] PASSED [ 35%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[17-20] PASSED [ 40%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[2-20] PASSED [ 45%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[15-20] PASSED [ 50%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[13-20] PASSED [ 55%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[11-20] PASSED [ 60%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[14-20] PASSED [ 65%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[1-20] PASSED [ 70%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[5-20] qPASSED [ 75%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[9-20] PASSED [ 80%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[19-20] PASSED [ 85%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[4-20] PASSED [ 90%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[20-20] PASSED [ 95%]
offline_tools_test.py::TestOfflineTools::test_sstablelevelreset[7-20] PASSED [100%]
============================================================================================================== warnings summary ==============================================================================================================
../../.local/lib/python3.11/site-packages/botocore/utils.py:25
/home/kefu/.local/lib/python3.11/site-packages/botocore/utils.py:25: DeprecationWarning: 'cgi' is deprecated and slated for removal in Python 3.13
import cgi
../../.local/lib/python3.11/site-packages/botocore/httpsession.py:28
/home/kefu/.local/lib/python3.11/site-packages/botocore/httpsession.py:28: DeprecationWarning: 'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x. Read more in this issue: https://github.com/urllib3/urllib3/issues/2680
from urllib3.contrib.pyopenssl import orig_util_SSLContext as SSLContext
../../.local/lib/python3.11/site-packages/cassandra/io/asyncorereactor.py:34
/home/kefu/.local/lib/python3.11/site-packages/cassandra/io/asyncorereactor.py:34: DeprecationWarning: The asyncore module is deprecated and will be removed in Python 3.12. The recommended replacement is asyncio
import asyncore
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================================= 20 passed, 3 warnings in 970.85s (0:16:10) =================================================================================================
in 5cb0985a, we tried harder to remove more unsealed sstables. but that's not enough. there is chances that scylla leaves us with a single '-TOC.txt.tmp' component, without corresponding '-Data.db'. in that case,
sstablelevelresetstill fails when trying to open all tables in the sstable data directory.so, in this change, instead of enumerating the sstables by globbing 'Data.db', we just glob 'TOC.txt.tmp'. and remove all components sharing the same prefix. this will clean all unsealed sstables files.
Fixes https://github.com/scylladb/scylladb/issues/15210 Signed-off-by: Kefu Chai kefu.chai@scylladb.com