Follow-up tasks for the LST algorithm

[VourMa]

This issue is a simplified version of SegmentLinking/cmssw#75, meant to list and keep track of the various upcoming developments for the LST algorithm that were either brought up during the PR #45117 review or planned by the development team. The order of the tasks below roughly corresponds to their priority/timescale, as estimated by the developers.

• [ ] Integrate accumulated algorithm developments held back during integration PR review (SegmentLinking/cmssw#117).

  More details

  - [ ] Update LST README, printouts and minor fixes: - Covered partially by SegmentLinking/cmssw#55. A follow up to accommodate the developments made in SegmentLinking/cmssw#72 will be needed.

  Links to specific comments

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1681737544 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1681741075 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1833259595 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1833261173 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1844301482 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1844313384

  - [ ] Deal with the configurability of the pT threshold: - Covered by SegmentLinking/cmssw#39. - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749245650

• [ ] LST in HLT

  More details

  - [ ] Rebase and merge SegmentLinking/cmssw#107.

• [ ] Changes in LST conditions data

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749249842: I would prefer if the possibility of overriding the conditions data setting `LST_BASE` or `TRACKLOOPERDIR` would be disabled when building this code inside CMSSW. - [ ] Remove lst_INF - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1625114981 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749241840 - [ ] https://github.com/cms-sw/cmssw/pull/45117#issuecomment-2479912047: `LSTESData::geometryDataDir` needs a check that `getenv("CMSSW_SEARCH_PATH")` is not null.

• [ ] LST workflow

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1664589679: Respect new relval_2026.py structure. - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1682593922: Decide if we want different workflows for different backends. - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1682597612: Implement a "GPU vs CPU" workflow. - [ ] https://github.com/cms-sw/cmssw/pull/45117#issuecomment-2479912047: Fill seed stop info when LST runs (expected by DQM/validation).

• [ ] Make general-purpose functions widely available in CMSSW

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1674782508 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749143439 - The plan would be to move all general-purpose functions (work started in SegmentLinking/cmssw#71): - Follow up PR with general functions. - Follow up PR moving LST to use those and delete custom implementations.

• [ ] Rewrite the loops in the kernels using uniform_elements, independent_groups, independent_group_elements, etc.

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749244105: Experiment with `make_workdiv`. - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1682849644: No need to set the grid size to 1, the alpaka execution will run the blocks one after the other automatically. You can set the grid size to 1 - the difference will be in how the kernel are executed over the elements. Either way, I think whatever approach you choose should be documented. - [ ] https://github.com/cms-sw/cmssw/pull/45117#pullrequestreview-2286131239

  Links to specific comments

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747147790 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747148816 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747157117 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747159610 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747160439 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747161690 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747492866 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747493175 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747493376 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747497128 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747501157 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747514758 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747514956 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747515175 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747519657 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747520652 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748006364 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748008415 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748008598 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748008657 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748008802 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009215 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009259 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009459 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009869 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748010016 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748010582 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748010837 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748015075 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748015269 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748058899 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749143810 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749144167 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749163618 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749163821 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749164934 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749165613 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749236516 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749236618 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749236663 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749237704 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749237801 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238020 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238089 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238179 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238234 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238443 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238487

• [ ] Rewrite kernels with proper, concrete dimensions instead of templated types

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#pullrequestreview-2286131239:

  Links to specific comments

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747146697 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747161172 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747493770 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747495326 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747501517 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747519297 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747520469 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748005531 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748005961 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748006364 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748008415 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748008961 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009215 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009259 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009395 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009581 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748009869 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748010016 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748010483 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748010582 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748010772 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748013396 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748058899 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749143631 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749143810 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749143984 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749144167 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749163156 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749163618 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749163821 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749164831 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749165439 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749236366 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749237268 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749237704 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749237801 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749237960 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238020 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238089 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238179 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238234 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238303 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238386 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238443 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749238487 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1833303542

• [ ] Improvements to the data format interface between LST and CMSSW

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#pullrequestreview-2289657004: Define the LST inputs and output as PortableCollections, and pass them (or their Views) to the LST algorithm.

  Links to specific comments

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1682835338: Fill a host SoA directly in `LSTPhase2OTHitsInputProducer`, copy it to device, and avoid the 6 intermediate copies of the `std::vector`s. - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1718975196: Performance-wise best would be to move them in `LSTProducer::produce()`, that would require the `LST::hits()` etc to either return a mutable reference, or a value that was moved-from in the `hits()` method itself. - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1718978133: Eventually (or ideally) we'd want to issue the `alpaka::memcpy()` calls to copy the data from device to host in `acquire()`, and have the destination host memory accessed only in `produce()`.

• [ ] Construct LST condition data from the existing EventSetup info

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#pullrequestreview-2286131239: - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749373735 - [ ] https://github.com/cms-sw/cmssw/pull/45117#issuecomment-2480475750

• [ ] Improvements in storing and using "magic numbers

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#pullrequestreview-2286131239:

  Links to specific comments

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747431534 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1747438638 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748017178 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748057472 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748057673 - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1748057925

• [ ] Miscellaneous improvements

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749139063: Try std::binary_search(data, data + ndata, search_val) from C++20. - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1749241199: Review uint4 usage.

• [ ] Improvements to ESProducer

  More details

  - [ ] https://github.com/cms-sw/cmssw/pull/45117#discussion_r1699051274: Make it more clear which members are on the device memory and which are on the host memory. Suggest to consider moving all data that are used only in host code into a separate ES data product.

FYI @slava77 @ariostas

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @VourMa.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[makortel]

assign reconstruction, heterogeneous, alca, hlt, upgrade

[makortel]

@cms-sw/tracking-pog-l2

[cmsbuild]

New categories assigned: reconstruction,heterogeneous,alca,hlt,upgrade

@atpathak,@consuegs,@fwyzard,@jfernan2,@makortel,@mandrenguyen,@Martin-Grunewald,@mmusich,@Moanwar,@perrotta,@srimanob,@subirsarkar you have been requested to review this Pull request/Issue and eventually sign? Thanks