CWP version for CSBS journal: comments from reviewers

[jouvin]

This is the comments received by email on Sept. 5 turned into a task list for easier tracking. **Be sure to update appropriately the Google Docs-based draft answer to CSBS reviewers before marking a task done.***

Introduction

• [x] line 141: I suggest to exchange the word "contributors" to "stakeholders" + I would suggest to start a new bullet point before "by the effective"
• [ ] line 201: please provide a reference for the "20 million lines of code"
• [ ] line 296: please provide a reference for "AArch64 may achieve lower power costs..."
• [ ] line 299: please provide a reference or a more specific name/example for "More extreme is an architecture that would see specialised..."
• [ ] line 377: please provide examples incl. references for "which developed novel and major new technologies..."

Section 3.2:

• [ ] line 684: The first action item for detector simulation seems out of place. While it is desirable to extend the validity of the physics modeling towards the FCC, this is not really a computing issue, nor does it impact the speed of simulations that will be necessary for the HL-LHC, except if by making it more accurate the result is slower code. If I go to the section on current practices, this is a bit better targeted towards improving accuracy AND efficiency. My suggestion is to at least mention software performance as a goal in this bullet.

• [ ] lines 715-727: it is striking that this section (and section 3.1) discusses human resources, while the following sections do not. It might be more impactful to isolate all human resource discussions to section 4.

Section 3.3:

• [ ] line 1103: remove the double "the"
• [ ] lines 1212-1219+1221: in lines 1212-1219 you speak of algorithms etc., but in 1221 you talk about educating physicists in modern coding practices - NO, these are two different things: algorithms and coding practise is like theoretical and experimental particle physics; both are connected but require different educational profiles. Concerning algorithms I advise that HEP stronger collaborates with computer science, which are experts in algorithms.
• [ ] The last piece of the R&D program, l. 1320, is not very specific. It would be useful to sharpen up the deliverable here, rather than using an "e.g. charged particle tracking" and "a number of such efforts"

Section 3.4:

• [x] “Scope and Challenges”: I’m sure that HEP Data Management also adheres to the FAIR principles. This is a much used buzzword, but not using it here in this context might raise the question why. Hence, you should make a conscious decision whether or not to mention these FAIR principles.

• [x] line 1334: “quasi-real time”; perhaps better: “near real-time”

• [x] The first piece of the R&D "enable ... to be plugged in dynamically" could do with some more specificity. The bullet is very general. Is it in conflict with the 3rd to last about interacting and exchanging data?

• [x] line 1371: provide examples (incl. references) for "...emergence of new analysis tools coming from industry and open source projects..."

• [x] line 1386 ff.: This is also called “provenance”. You might want to use this term here.

• [x] lines 1476ff.: please provide references for each given example technology

• [x] line 1569: I would have expected more milestones, e.g. in the direction of uptake of the software (or at least an evaluation) mentioned above or integrating/interfacing them in existing environments

Section 3.5 (Machine Learning):

• [ ] line 1636: “… most CPU intensive elements …”; GPUs are apparently well suited for at leas some of the ML algorithms. You might want to mention them here.
• [ ] line 1710: “HSF IML”; for better readability, you might want to write “HEP Software Foundation IML” here.

Section 3.7:

• [ ] line 2150: please provide a reference for "two orders of magnitude..."
• [ ] lines 2153-2161: it would be good to have some "test projects" for SDNs in HEP, but nothing could be found later in the R&D programme from line 2242 ff.
• [ ] lines 2238-2241: please update the numbers and the reference, which is from 2017
• [ ] lines 2257 + 2301: will these working groups be part of HSF or somewhere else - in any case state something

Section 3.8:

• [ ] line 2405: It is striking that this section adds a 2018 programme. Others only discuss 2020 and 2022. Is this an intentional difference?

Section 3.10:

• [ ] lines 2746-2780: the whole Section should be improved. The challenge of R&D is not clear. Seems to be a minor activity compared to the other subsections in section 3.
• [ ] line 2762: the text here is vague e.g. "the wg will also work towards a more convenient access... through a client-server interface", "...a service to deliver streamed event data would be designed". More details would be nice
• [ ] line 2774: please provide a reference/link for the mentioned workshop

Section 3.11:

• [ ] In contrast to previous sections, this R&D section does not have dates. Just short-term and long-term
• [ ] line 2861: is it git or github - is there an effect of the recent purchase from Microsoft, please elaborate.

Section 3.12:

• [ ] line 3291 please provide examples and/or a reference for "next-generation identity federations". There are some out there, e.g. bwIDM.

Section 3.13 (Security)

• [ ] Line 3171: add data privacy. Data privacy: legal questions, as in unauthorised access to personal data. Data protection: avoiding unauthorised access.
• [ ] line 3286: “… attributes published by each federation …”: Federations do not provide attributes. And attributes are not necessarily linked to authentication.
• [ ] line 3298: “Although federated identity provides…”: X.509 is also a ferreted ID solution.
• [ ] lines 3322+3333: please also give the long name for WISE and FIM4R
• [ ] line 3333: CERN is not the only partner in AARC, also major Tier-1s also participate, e.g. KIT and NIKHEF. Please re-formulate/add this.

Section 4:

• [ ] line 3432: what does "promoting specific champions in the field" mean here in detail, please provide examples. How can they be promoted?

Section 4.1:

• [ ] lines 3447: please provide examples (+ a reference) from science where "users express their requirements and computer specialists implement solutions", otherwise this falls from heaven and this argumentation can not be used.

Section 4.2 and 4.3:

• [ ] these are rather weak sections compared to the importance of the topic. After reading the sections 3.x I would have expected to also see a work programm and milestones, but nothing is here - a pity... You could generate ideas e.g. establish working groups on this topic, organise workshops, etc. but nothing is mentioned...not good.

These detailed comments where preceded by the following general remarks:

This document provides to the High Energy Physics community a clear status of the current software and computing practices which have been set along the last decade in order to achieve the exploitation of the data produced by the four detectors installed on the Large Hadron Collider at CERN. It also, and this is its main objective, proposes a program of work (PoW) in various identified areas that must be done to be able to successfully process the data that will be produced by the upgraded LHC (HL-LHC) in less than 10 years from now.

Reviewing the document was made more difficult due to the poor formatting of the manuscript. Therefore, some of the line numbers mentioned below might be wrong and we encourage the authors to seek a solution for the formatting problems before resubmission.

In addition, it seems that many/most of the references (only analysing the reference names) are from within the HEP community. A broader view should be taken, i.e. references from outside the HEP community should be considered. 
 The proposed program of work appears to be exhaustive from a technology point of view; at least with the current HEP community knowledge of the evolution of hardware and software that may occur during this period. 


Some general points were mentioned in the referee reports. These might be addressed in the Introduction and/or Conclusion
 (or Section 2?):

Attention must be better paid to
 ⁃ Carefully take into account the consequences on the computing facilities (Tiers 0 to 1) of such or such solution that may be raised from the POW and that may lead to extra cost. For instance, HPC or GP/GPU facilities if they provide more efficient (in CPU time) ways to do specific processing are much more expensive traditional HTC farms.
 ⁃ extra cost that may be generated by Hybrid Computing especially if made of commercial resources
. ⁃ not to produce solutions that may lead to the set up of dedicated hardware that then may not be mutualized with other experiments yet supported by the same funding agencies as HL-LHC
.

• Tier 1s must be part of these works at the earliest possible stage.



Even though it is well understood that this document focuses on HEP computing challenges, the proposed program does not make any reference to the European or worldwide computing for research context. It is mandatory that all these works take into account the evolution of the e-infrastructures they rely on . This may require some closed interactions with bodies like EGI, OSG, EUDAT, PRACE, NRENs… This « collaborative » work must be more clearly identified within the relevant POW if not within a dedicated one.

Finally, the work package relative to Security and Authentication and Authorization Infrastructure purposes a sum-up of the most important actions expected to be achieved. As these issues are beyond the scope of the HEP community and depend on the various national security policy it is of first importance that this work package is handled at the very stage of the proposed roadmap

[davidlange6]

i started a google doc to organize thoughts - it would be too complex on a GitHub issue I think

https://docs.google.com/document/d/1y1xrJxPkJcGxpuF_7NrCfCqLiy4b4Ahg2zZoQ9SWNCk/edit

[jouvin]

@davidlange6 sure the issue was here to track what has been addressed. In fact I hesistate between a Google doc and the file I added in the PR #104 (answers_to_csbs_reviewers.txt) to share thoughts... Thanks for the Google Docs.

[davidlange6]

off topic - how are people cloning this documents repository? The entire thing just to get one of the N documents?

[jouvin]

This is the easiest with Git... Do you really care? It's true that it is non optimal because of the PDF in this repo. We should move to LFS for storing them more efficiently...

[graeme-a-stewart]

Yes, it's only 74MB and should be mainly 'sources' for LaTeX (some PDFs, of course). I think that having one repository for every document would just be unwieldy in a different way...

[davidlange6]

Eventually we will all care when there are enough papers there. Should think how to organize for the future.

On 18 Oct 2018, at 17:00, Michel Jouvin notifications@github.com<mailto:notifications@github.com> wrote:

This is the easiest with Git... Do you really care? It's true that it is non optimal because of the PDF in this repo. We should move to LFS for storing them more efficiently...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/HSF/documents/issues/105#issuecomment-431042727, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEzyw6H--lXRfU32gE3mz3TM8MNUBeeMks5umJc9gaJpZM4XlQLH.

[jouvin]

Let's stop this discussion here so that the issue remains about its original topic... I opened #106 to follow up the discussion started by @davidlange6 .

[jouvin]

Closed as the work has been done and followed in a different place (https://docs.google.com/document/d/1y1xrJxPkJcGxpuF_7NrCfCqLiy4b4Ahg2zZoQ9SWNCk/edit). Not worth updating the issue...