IPDPS for HPC

[ielhajj]

Why was IPDPS not considered for the HPC venues? It is typically the second target after SC and has higher h5-index than HPDC and ICS.

SC: h5-index = 48 IPDPS: h5-index = 41 HPDC: h5-index = 29 ICS: h5-index = 29

[fastalgo]

I agree, many UC Berkeley professors submit their papers to IPDPS.

[ielhajj]

Also, SIGHPC is involved in IPDPS: http://www.sighpc.org/events/workshops It is also involved in SC, PPoPP, and HPDC however, it is not involved in ICS. ICS is a SIGARCH event: http://www.ics-conference.org

[ielhajj]

Any verdict on IPDPS?

In the FAQ (http://csrankings.org/faq.html), it is listed that:

Why so few conferences per area? Only the very top conferences in each area are listed. All conferences listed must be roughly equivalent in terms of number of submissions, selectivity and impact to avoid creating incentives to target less selective conferences.

IPDPS (h5-median=52) is more equivalent to SC (h5-median=67) on these metrics than are HPDC (h5-median=44) and ICS (h5-median=41).

The FAQ also lists:

How were research areas determined? Nearly all categories are based on research-focused ACM SIGs. Areas not represented by ACM SIGs are intended to span most established research-centric areas of computer science.

IPDPS is SIGHPC while ICS is SIGARCH, so IPDPS is again more appropriate when SIGs are considered.

[tiwarid]

I agree with above comments. IPDPS should be considered before HPDC and ICS. I have asked many folks who publish in HPC area regularly (both in and outside academia), they all consider IPDPS as a top-venue for HPC research, right after SC. So, it's somewhat surprising that IPDPS doesn't feature in CSRankings!

[tiwarid]

In fact, this year's Google scholar metric is showing IPDPS ahead of even SC (which was not the case last year as shown in the first post in this thread). ICS and HPDC are quite behind and are actually slightly weaker than last year's numbers. I feel that justification for adding IPDPS is as strong as having SC in the list. Both SC and IPDPS get roughly 300-500 submissions. ICS and HPDC get 100-160 submissions (I added this piece since we should note h5-index is indeed affected by the number of accepted papers which are also a function of submitted paper -- which in turn are also a function of and indicative of which venues the community values more, and then a cycle ;-). There is little to choose between HPDC and ICS -- perhaps HPDC since it is more HPC focused.

IPDPS h5-index = 46 Supercomputing h5-index = 42 HPDC h5-index = 28 ICS h5-index = 25

[ielhajj]

I think the h5-index difference between IPDPS and SC is impacted by the fact that IPDPS'18 proceedings have come out but SC'18 proceedings haven't come out yet. But regardless, they are both way ahead of HPDC and ICS as you mentioned.

If a third conference were to be chosen, I think PPoPP would be most appropriate. It has h5-index=32 which is higher than HPDC and ICS. Moreover, it is actually co-sponsored by SIGHPC (sighpc.org homepage advocates for SC, IPDPS, and PPoPP at the time of writing this post) whereas ICS and HPDC are SIGARCH conferences (HPDC is occasionally co-sponsored by SIGHPC).

[fycus-tree]

@ielhajj h-index is a volume statistic. If PPoPP accepts more papers than HPDC or ICS, and it has roughly the same h-index, it's a much weaker conference.

EDIT: I had the numbers wrong

[ielhajj]

@fycus-tree I agree that h-index is impacted by volume and that should be considered, but it's not true that PPoPP has higher volume than HPDC and ICS. See stats below. They are roughly the same.

Year	Submitted	Accepted	Rate
PPoPP '10	173	29	17%
PPoPP '11	165	26	16%
PPoPP '12	173	26	15%
PPoPP '13	146	26	18%
PPoPP '14	184	28	15%
PPoPP '17	132	29	22%

Year	Submitted	Accepted	Rate
HPDC '11	170	22	13%
HPDC '12	143	23	16%
HPDC '13	131	20	15%
HPDC '14	130	21	16%
HPDC '15	116	19	16%
HPDC '16	129	20	16%
HPDC '17	100	19	19%
HPDC '18	111	22	20%

Year	Submitted	Accepted	Rate
ICS '10	180	32	18%
ICS '11	161	35	22%
ICS '12	161	36	22%
ICS '13	202	43	21%
ICS '14	160	34	21%
ICS '15	160	40	25%

[fycus-tree]

@ielhajj my bad. Thanks for that information. I lazily trusted Google's preview, which gave me this, which said 352 out of 1,637, but that was the total over the last 20 years, not a given year. Thanks for that table

[fycus-tree]

On a separate note, the analysis in #859 shows that PPoPP (and ICS) are not very high on the "# of R1 universities that publish there". The HPC cluster, in order of that metric, is

1. IPDPS
2. SC
3. J. Parallel Distrib. Comput.
4. ICPP
5. ICCS
6. SIAM J. Scientific Computing
7. ICS
8. PPOPP
9. J. Comput. Physics,

I think maybe Emery's perspective as a US faculty member may be affecting this grouping. The current CS Rankings for ICS vs HPDC & SC support this slightly.

[tiwarid]

PPoPP vs HPDC vs ICS: I agree PPoPP might be slightly better choice based on h5-index metric. Though PPoPP does not seem to a popular top venue for HPC researchers. I can imagine a few legitimate reasons. PPoPP PC often does not necessarily feature a rich and large set of HPC experts. The kind of papers PPOPP tends of accept (or receive) are not very HPC-centric. PPoPP papers typically do not cover wide variety of HPC topics -- a few HPC papers are mostly focused on compilers, programming languages, and runtime systems. Perhaps, because of these issues it has never really become a top popular choice for HPC folks. The name of the conference may be playing a bit of its part. For example, many HPC "algorithms and systems" folks believe that PPoPP is more suitable for programming runtime and compiler works.

[tiwarid]

On a separate note, the analysis in #859 shows that PPoPP (and ICS) are not very high on the "# of R1 universities that publish there". The HPC cluster, in order of that metric, is

1. IPDPS
2. SC
3. J. Parallel Distrib. Comput.
4. ICPP
5. ICCS
6. SIAM J. Scientific Computing
7. ICS
8. PPOPP
9. J. Comput. Physics,

I think maybe Emery's perspective as a US faculty member may be affecting this grouping. The current CS Rankings for ICS vs HPDC & SC support this slightly.

Great point. Perhaps, this data also supports inclusion of IPDPS in the list?

[fycus-tree]

@tiwarid Unfortunately, IPDPS is a larger venue than SC and HPDC (by a factor of two or more). So they're not comparable in Emery's criteria.

Now, on the other hand, all of these venues are tiny compared to an AI/CV venue like NIPS or CVPR, but Emery's mean-by-category takes care of that (in a way).

[fycus-tree]

PPoPP vs HPDC vs ICS: I agree PPoPP might be slightly better choice based on h5-index metric. Though PPoPP does not seem to a popular top venue for HPC researchers... For example, many HPC "algorithms and systems" folks believe that PPoPP is more suitable for programming runtime and compiler works.

The analysis in #859 actually shows this to be true as well. If you look at the "16 clusters" result, PPoPP doesn't cluster with IPDPS/SC/ICS. It instead clusters with EuroSys/Eurocrypt/CDC/CommACM,etc.

[tiwarid]

Unfortunately, IPDPS is a larger venue than SC and HPDC (by a factor of two or more). So they're not comparable in Emery's criteria.

Typical number of accepted papers are: HPDC ~30, ICS ~30, SC ~90, and IPDPS ~110. Typical number of submissions are HPDC ~150, ICS ~170, SC ~400, and IPDPS ~500. All of them have similar acceptance rate (HPDC is more selective than others). IPDPS (and now SC) uses a more rigorous 2-round review/revision model -- similar to SIGMETRICS (not Eurosys). ICS and HPDC have traditional reviewing style. Yes, there are variations and they affect other metrics such as h5-index. I am also aware that Emery takes all three metrics into account (acceptance rate, number of submissions, and h5-index) and tries to come with a sample of three which form a "equivalence class".

I think we (at least I) have been going about the whole issue in the wrong way by asking and showing evidence that IPDPS should replace HPDC/ICS. If we follow this approach, we get ourselves trapped. One may end up concluding IPDPS is not comparable with HPDC/ICS/SC as you correctly pointed out. Hence, the original list will not be revised ever.

Instead, we should ask if we were going to pick three conferences from scratch to represent the HPC area, which would be those? Which conferences proceedings people would people to pick read the latest and greatest HPC works? If my group made a breakthrough in HPC, which conferences would I try to get my paper accepted (assuming I can change the deadline of conferences and their notification date as per my wish)?

HPC folks I have worked with, they typically prefer this order (or something very similar): SC, IPDPS, HPDC, Cluster, JPDC, and rest (ICS, PPoPP, CCGrid, Concurrency Practices, ASPLOS etc.) That doesn't mean ICS is of any lesser value or competence. It just may not be my first choice for publishing breakthrough research in HPC; SIGARCH actually sponsors ICS and ICS gets great architecture submissions. We also have to note that SC and IPDPS receive very large number of papers that the third conference is going to be invariably much smaller. So, no matter which three we choose, we will always have this problem -- one looking like a misfit.

[sbak5]

Unfortunately, IPDPS is a larger venue than SC and HPDC (by a factor of two or more). So they're not comparable in Emery's criteria.

Typical number of accepted papers are: HPDC ~30, ICS ~30, SC ~90, and IPDPS ~110. Typical number of submissions are HPDC ~150, ICS ~170, SC ~400, and IPDPS ~500. All of them have similar acceptance rate (HPDC is more selective than others). IPDPS (and now SC) uses a more rigorous 2-round review/revision model -- similar to SIGMETRICS (not Eurosys). ICS and HPDC have traditional reviewing style. Yes, there are variations and they affect other metrics such as h5-index. I am also aware that Emery takes all three metrics into account (acceptance rate, number of submissions, and h5-index) and tries to come with a sample of three which form a "equivalence class".

I think we (at least I) have been going about the whole issue in the wrong way by asking and showing evidence that IPDPS should replace HPDC/ICS. If we follow this approach, we get ourselves trapped. One may end up concluding IPDPS is not comparable with HPDC/ICS/SC as you correctly pointed out. Hence, the original list will not be revised ever.

Instead, we should ask if we were going to pick three conferences from scratch to represent the HPC area, which would be those? Which conferences proceedings people would people to pick read the latest and greatest HPC works? If my group made a breakthrough in HPC, which conferences would I try to get my paper accepted (assuming I can change the deadline of conferences and their notification date as per my wish)?

HPC folks I have worked with, they typically prefer this order (or something very similar): SC, IPDPS, HPDC, Cluster, JPDC, and rest (ICS, PPoPP, CCGrid, Concurrency Practices, ASPLOS etc.) That doesn't mean ICS is of any lesser value or competence. It just may not be my first choice for publishing breakthrough research in HPC; SIGARCH actually sponsors ICS and ICS gets great architecture submissions. We also have to note that SC and IPDPS receive very large number of papers that the third conference is going to be invariably much smaller. So, no matter which three we choose, we will always have this problem -- one looking like a misfit.

I don't think Cluster is preferred than others. Its h-index is much lower than others and in terms of paper quality, it is not considered better than any conferences you mentioned in 'rest'.

Even though, IPDPS has high h-index, the number of publications should be also considered. Usually, SC is more competitive than IPDPS and HPDC is hardest to get accepted.

Even though SIGARCH sponsors ICS, ICS is usually considered as one of premier conferences in ICS. I haven't seen anyone who submit a paper to JPDC other than other conferences which is against the general publication preference in computer science. (First publish in a conference and extended work in Journal).

Highly HPC centric conferences SC, IPDPS (Big size) HPDC, ICS (Small size)

In addition, ICS, HPDC include less papers from scientific applications and their optimizations for specific supercomputers and more papers in more general parallel computing perspective while SC usually has many papers for specific supercomputers, which may not be applicable to industry applications.

Conferences covering HPC and others(Architecture, Compilers, Programming Languages, Cloud Computing). PPOPP (Language,Compiler, Runtime) CCGrid (Cluster Computing, Cloud/Grid Computing) PACT(Architecture, Compiler, and Runtime)

CCGrid seems to include many HPC expertise (See steering committee) and kind of venue for cluster and cloud computing. Its H-index is higher than others. PPOPP and PACT seems more oriented to Compiler and Languages. So, only for HPC, SC/IPDPS, ICS, HPDC are premier venues and sometimes others are considered I mentioned above.

[github-actions[bot]]

Stale issue message