Open JIN77553311 opened 1 week ago
weihao1115
commented
6 days ago Hi @JIN77553311 ,
Thank you for your interest in our work.
In our experiments, despite the presence of small randomness, we have never encountered such a performance difference. Please provide your PyTorch version and your corresponding environment details so we can find a 3090 GPU to see what happened.
Best regards,
JIN77553311
commented
5 days ago Thank you for your response.
The packages installed in our virtual environment and their respective versions, are listed below:
Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
affine 2.4.0 pypi_0 pypi
aiohappyeyeballs 2.4.3 pypi_0 pypi
aiohttp 3.10.10 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
albucore 0.0.17 pypi_0 pypi
albumentations 1.4.18 pypi_0 pypi
annotated-types 0.7.0 pypi_0 pypi
antlr4-python3-runtime 4.9.3 pypi_0 pypi
asttokens 2.4.1 pyhd8ed1ab_0 conda-forge
async-timeout 4.0.3 pypi_0 pypi
attrs 24.2.0 pypi_0 pypi
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
beautifulsoup4 4.12.3 pypi_0 pypi
ca-certificates 2024.8.30 hbcca054_0 conda-forge
cat-sam 0.1 dev_0
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
dill 0.3.8 pypi_0 pypi
efficientnet-pytorch 0.7.1 pypi_0 pypi
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
eval-type-backport 0.2.0 pypi_0 pypi
executing 2.1.0 pyhd8ed1ab_0 conda-forge
filelock 3.16.1 pypi_0 pypi
fonttools 4.54.1 pypi_0 pypi
frozenlist 1.4.1 pypi_0 pypi
fsspec 2024.6.1 pypi_0 pypi
fvcore 0.1.5.post20221221 pypi_0 pypi
gdown 5.2.0 pypi_0 pypi
gputil 1.4.0 pypi_0 pypi
huggingface-hub 0.25.2 pypi_0 pypi
idna 3.10 pypi_0 pypi
imageio 2.36.0 pypi_0 pypi
importlib-metadata 8.5.0 pypi_0 pypi
importlib-resources 6.4.5 pypi_0 pypi
iopath 0.1.10 pypi_0 pypi
ipykernel 6.29.5 pyh3099207_0 conda-forge
ipython 8.12.0 pyh41d4057_0 conda-forge
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jupyter_client 7.3.4 pyhd8ed1ab_0 conda-forge
jupyter_core 5.7.2 pyh31011fe_1 conda-forge
kiwisolver 1.4.7 pypi_0 pypi
lazy-loader 0.4 pypi_0 pypi
ld_impl_linux-64 2.40 h12ee557_0
libffi 3.4.4 h6a678d5_1
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libsodium 1.0.18 h36c2ea0_1 conda-forge
libstdcxx-ng 11.2.0 h1234567_1
loguru 0.7.2 pypi_0 pypi
matplotlib 3.9.2 pypi_0 pypi
matplotlib-inline 0.1.7 pyhd8ed1ab_0 conda-forge
monai 1.4.0 pypi_0 pypi
multidict 6.1.0 pypi_0 pypi
multiprocess 0.70.16 pypi_0 pypi
munch 4.0.0 pypi_0 pypi
ncurses 6.4 h6a678d5_0
nest-asyncio 1.6.0 pyhd8ed1ab_0 conda-forge
networkx 3.2.1 pypi_0 pypi
numpy 1.24.4 pypi_0 pypi
nvidia-cublas-cu11 11.10.3.66 pypi_0 pypi
nvidia-cuda-nvrtc-cu11 11.7.99 pypi_0 pypi
nvidia-cuda-runtime-cu11 11.7.99 pypi_0 pypi
nvidia-cudnn-cu11 8.5.0.96 pypi_0 pypi
omegaconf 2.3.0 pypi_0 pypi
opencv-python 4.9.0.80 pypi_0 pypi
opencv-python-headless 4.10.0.84 pypi_0 pypi
openssl 3.0.15 h5eee18b_0
packaging 24.1 pyhd8ed1ab_0 conda-forge
pandas 2.2.3 pypi_0 pypi
parso 0.8.4 pyhd8ed1ab_0 conda-forge
pexpect 4.9.0 pyhd8ed1ab_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 11.0.0 pypi_0 pypi
pip 24.2 py39h06a4308_0
platformdirs 4.3.6 pyhd8ed1ab_0 conda-forge
portalocker 2.10.1 pypi_0 pypi
pretrainedmodels 0.7.4 pypi_0 pypi
prompt-toolkit 3.0.48 pyha770c72_0 conda-forge
prompt_toolkit 3.0.48 hd8ed1ab_0 conda-forge
propcache 0.2.0 pypi_0 pypi
psutil 5.9.0 py39h5eee18b_0
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.3 pyhd8ed1ab_0 conda-forge
pyarrow 17.0.0 pypi_0 pypi
pydantic 2.9.2 pypi_0 pypi
pydantic-core 2.23.4 pypi_0 pypi
pygments 2.18.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.2.0 pypi_0 pypi
pysocks 1.7.1 pypi_0 pypi
python 3.9.20 he870216_1
python-box 7.2.0 pypi_0 pypi
python-dateutil 2.9.0.post0 pypi_0 pypi
python_abi 3.9 2_cp39 conda-forge
pytz 2024.2 pypi_0 pypi
pyyaml 6.0.2 pypi_0 pypi
pyzmq 25.1.2 py39h6a678d5_0
rasterio 1.4.1 pypi_0 pypi
readline 8.2 h5eee18b_0
regex 2024.9.11 pypi_0 pypi
requests 2.32.3 pypi_0 pypi
safetensors 0.4.5 pypi_0 pypi
scikit-image 0.24.0 pypi_0 pypi
scipy 1.13.1 pypi_0 pypi
segmentation-models-pytorch 0.3.4 pypi_0 pypi
setuptools 75.1.0 py39h06a4308_0
six 1.16.0 pyh6c4a22f_0 conda-forge
soupsieve 2.6 pypi_0 pypi
sqlite 3.45.3 h5eee18b_0
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
tabulate 0.9.0 pypi_0 pypi
termcolor 2.5.0 pypi_0 pypi
tifffile 2024.8.30 pypi_0 pypi
timm 0.9.7 pypi_0 pypi
tk 8.6.14 h39e8969_0
tokenizers 0.20.1 pypi_0 pypi
torch 1.13.1 pypi_0 pypi
torchprofile 0.0.4 pypi_0 pypi
torchvision 0.14.1 pypi_0 pypi
tornado 6.1 py39hb9d737c_3 conda-forge
tqdm 4.66.5 pypi_0 pypi
traitlets 5.14.3 pyhd8ed1ab_0 conda-forge
transformers 4.46.0.dev0 pypi_0 pypi
typing_extensions 4.12.2 pyha770c72_0 conda-forge
tzdata 2024.2 pypi_0 pypi
unrar 0.4 pypi_0 pypi
urllib3 2.2.3 pypi_0 pypi
wcwidth 0.2.13 pyhd8ed1ab_0 conda-forge
wheel 0.44.0 py39h06a4308_0
xxhash 3.5.0 pypi_0 pypi
xz 5.4.6 h5eee18b_1
yacs 0.1.8 pypi_0 pypi
yarl 1.15.4 pypi_0 pypi
zeromq 4.3.5 h6a678d5_0
zipp 3.20.2 pypi_0 pypi
zlib 1.2.13 h5eee18b_1
Thank you for your help.
Dear Authors,
Thank you for your excellent research and for sharing the code.
I am currently working on reproducing the performance reported in your paper for the SBU, WHU, and KVASIR datasets. I have conducted experiments using an A5000 GPU (and also performed them on an RTX 3090), with more than 10 independent runs for each dataset. However, the average performance I obtained differs significantly from the values reported in the paper (CAT-SAM-A, from Table 4), as shown below:
Below are the experimental settings I used:
Despite efforts to control for randomness, the reproduced results consistently fall outside the reported mean and standard deviation range presented in Table 12 of your paper.
I am writing to ask if there might be any experimental settings or factors I may have overlooked. Any guidance you could provide would be greatly appreciated in helping me align my results with the reported performance.
Thank you again for your valuable research and for taking the time to address this inquiry.