SuperScientificSoftwareLaboratory / TileSpGEMM

Source code of the PPoPP '22 paper: "TileSpGEMM: A Tiled Algorithm for Parallel Sparse General Matrix-Matrix Multiplication on GPUs" by Yuyao Niu, Zhengyang Lu, Haonan Ji, Shuhui Song, Zhou Jin, and Weifeng Liu.
38 stars 7 forks source link

Fix Undefined Behavior #2

Open elvircrn opened 2 years ago

elvircrn commented 2 years ago

Hi all.

I am in the process of reproducing the paper that you guys published.

After following the steps from the README, and testing out the code on the following matrix: https://sparse.tamu.edu/SNAP/CollegeMsg, I received a SEGFAULT.

In particular, here are the steps that I took:

https://github.com/SuperScientificSoftwareLaboratory/TileSpGEMM
cd TileSpGEMM
mkdir data
cd src
make
./test -d 0 -aat 0 [REDACTED]/CollegeMsg/CollegeMsg.mtx

I've done this on the following GPUs with differing CPUs:

RTX 3090 RTX 2080 GTX 1650 Ti

and have therefore managed to produce crashes on multiple machines.

After attempting to integrate this library and taking a closer look at the SEGFAULT, I found three problems in the code with the help of Valgrind:

The first one is a invalid read in line 180 in csr2tile.h. Since ri is compared against rowlen-1, and the for loop goes to BLOCK_SIZE, in the last block, an invalid memory read is done causing a random crash somwhere downstream. This fix changes the logic of the code a bit and I would definitely like to receive feedback here.

The second and third bug seem to be a typo (file: csr2tile.h) and innocent enough at first glance. The memsets touched by this PR write over tile_csr_Col (it should actually zero out tile_csr_Value) and it writes past what was allocated at tile_csr_Col since sizeof(unsigned char) < sizeof(float), causing random crashes depending on how the code is run.

After applying this patch, the code stops crashing.

Please let me know if more information is needed.

Thanks, Elvir

elvircrn commented 2 years ago

I've also added one more change to this PR since I've also encountered a crash when CHECK_RESULT=1 flag is added to the Makefile. When the flag is set, the A` matrix is used after being freed. This unblocked me from actually running the tests.

xcx6 commented 6 months ago

Could you tell me if you have encountered a situation where matrix information is displayed incorrectly right from the start of reading, and furthermore, when reading the same matrix, the displayed rows and columns are not fixed? Below is my configuration.Note: I ran this on a cloud server, and GPU was not enabled for this run. However, even with GPU enabled, the results are the same.3Q~ EU0)UPF7IH39K)H4L)A%~)Y U@TWOWX2BXE)~{HKAAUS5(M

elvircrn commented 6 months ago

Hi xcx6,

It seems like you didn't fully reproduce this error.

Please try using this matrix: https://sparse.tamu.edu/SNAP/CollegeMsg, as reported by my issue.

Thanks, Elvir

xcx6 commented 6 months ago

Thank you for your response. In fact, I have already tried the three datasets mentioned in your issue as well as the author's dataset, and the problems are all similar.If the dataset does not undergo any additional processing, then the result is like this.

8O}Y%XJ7}_7ZX$X9DY_`2}N

xcx6 commented 6 months ago

If choose to transpose the matrix, it will result in a segmentation fault error. )GS9Y@B RK0Y3FPD837E @U

xcx6 commented 6 months ago

If choose to transpose the matrix, it will result in a segmentation fault error. )GS9Y@B RK0Y3FPD837E @U