search

ZhouHuang23 / FSPNet

45 stars 12 forks source link

Reproducibility problems #15

Open alessandriniluca opened 4 months ago

alessandriniluca commented 4 months ago

First of all, thank you for the work you did in the paper!

I'm sorry to bother you, but I am facing some problems that I'll explain here below

Problems during Machine setup

Unfortunately, I'm encountering some troubles in running the code (and I think some versions are missing for correct reproducibility). As suggested in your README.md, I am running the experiments on ubuntu. More specifically, lsb_release -a returns:

Distributor ID: Ubuntu
Description:    Ubuntu 22.04 LTS
Release:    22.04
Codename:   jammy

Possible error in requirements.txt or wrong cuda

The error I'm obtaining arises when i perform pip install -r requirements.txt, and it says:

Collecting torch==1.6.0 (from -r requirement.txt (line 1))
  Using cached torch-1.6.0-cp38-cp38-manylinux1_x86_64.whl (748.8 MB)
Collecting scipy==1.2.2 (from -r requirement.txt (line 2))
  Using cached scipy-1.2.2.tar.gz (23.1 MB)
  Preparing metadata (setup.py) ... done
ERROR: Ignored the following yanked versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3, 0.11.0, 0.15.0
ERROR: Ignored the following versions that require a different python version: 1.11.0 Requires-Python <3.13,>=3.9; 1.11.0rc1 Requires-Python <3.13,>=3.9; 1.11.0rc2 Requires-Python <3.13,>=3.9; 1.11.1 Requires-Python <3.13,>=3.9; 1.11.2 Requires-Python <3.13,>=3.9; 1.11.3 Requires-Python <3.13,>=3.9; 1.11.4 Requires-Python >=3.9; 1.12.0 Requires-Python >=3.9; 1.12.0rc1 Requires-Python >=3.9; 1.12.0rc2 Requires-Python >=3.9
ERROR: Could not find a version that satisfies the requirement torchvision==0.4.0 (from versions: 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.1, 0.11.2, 0.11.3, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0)
ERROR: No matching distribution found for torchvision==0.4.0

Steps to reproduce:

On an ubuntu distribution with cuda 12.3, after having installed miniconda:

Possible missing versions

The following package / libraries versions are not specified:

Problems regarding results reproducibility

This problem could be due to the different versions of things I used, and I think this should be solved when the above one is solved.

Since on the server I have not enough space, I run this test on my laptop and desktop both running Arch Linux. In order to solve the above issue, I tried to change some version of python and packages.

Through conda I installed:

Through poetry I found compatible versions for python libraries, which may differ with respect to the ones you used. The pyproject.toml I declared is like the following:

[tool.poetry]
name = "fspnet"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.9"
torch = "1.7.1"
torchvision = "0.8.2"
scipy = "^1.12.0"
opencv-python = "4.4.0.46"
timm = "0.6.5"
imageio = "^2.34.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

I tried to run predictions on COD10K test set, but seem being quite different with respect to the result you obtained. I suspect that this may be a problem of packages versions, and should be solved when the above problem is solved. Issues persist both when using a GTX 1050 (laptop configuration) and RTX 4080 (Desktop configuration)

Thank you in advance for your availability

Baber-Jan commented 2 months ago

I have the same issue. However, I found that the given weights dont include the weights for group_converters in the model. The weights have weights for 'module.converter' but there is no module in the given model architecture.

Can you please upload the weights for the given architecture?