microsoft / torchscale

Foundation Architecture for (M)LLMs
https://aka.ms/GeneralAI
MIT License
2.98k stars 201 forks source link

Different batch sizes lead to different evalution results for LongVIT #107

Open HHHedo opened 2 months ago

HHHedo commented 2 months ago

Hi, Thanks for your great work. When I use the LongVIT for survival prediction, I find that different batch sizes will lead to different C-index results. After I print the risk for each person, I also find that some risks are different. The followings are some examples

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

bs = 8| bs =1 |bs =2 -- | -- | -- TCGA-06-0168 -1.2465172 | TCGA-06-0168 -1.2457247 | TCGA-06-0168 -1.2465172 TCGA-06-0645 -3.1927319 | TCGA-06-0645 -3.193125 | TCGA-06-0645 -3.1927319 TCGA-06-0646 -2.8353539 | TCGA-06-0646 -2.8353539 | TCGA-06-0646 -2.8353539 TCGA-06-0750 -0.9242315 | TCGA-06-0750 -0.924083 | TCGA-06-0750 -0.9242315 TCGA-06-5410 -1.7878933 | TCGA-06-5410 -1.7888575 | TCGA-06-5410 -1.7877667 TCGA-06-5411 -0.7950437 | TCGA-06-5411 -0.7950437 | TCGA-06-5411 -0.7950437 TCGA-06-5413 -0.8715491 | TCGA-06-5413 -0.8715491 | TCGA-06-5413 -0.87227124 TCGA-06-5856 -1.512358 | TCGA-06-5856 -1.512358 | TCGA-06-5856 -1.512358 TCGA-12-0618 -3.3228474 | TCGA-12-0618 -3.3228474 | TCGA-12-0618 -3.3228474 TCGA-12-0821 -3.0764904 | TCGA-12-0821 -3.0764904 | TCGA-12-0821 -3.0764904 TCGA-12-3653 -2.5728168 | TCGA-12-3653 -2.5728168 | TCGA-12-3653 -2.5728168 TCGA-19-2629 -2.5467625 | TCGA-19-2629 -2.5467625 | TCGA-19-2629 -2.5467625 TCGA-19-5960 -2.1180227 | TCGA-19-5960 -2.1180227 | TCGA-19-5960 -2.1180227 TCGA-26-1442 -1.2899182 | TCGA-26-1442 -1.2899779 | TCGA-26-1442 -1.2899182 TCGA-26-5133 -2.0898883 | TCGA-26-5133 -2.0900764 | TCGA-26-5133 -2.0894563 TCGA-27-1830 -1.2196785 | TCGA-27-1830 -1.2196785 | TCGA-27-1830 -1.2196785 TCGA-27-1831 -2.6002393 | TCGA-27-1831 -2.599653 | TCGA-27-1831 -2.599653

Looking forward to your help!