CMSSW DXR replacement - Githubissues

smuzaffar commented 4 months ago

@cms-sw/all-l2 , CMSSW DXR is not working properly[a] after moving to LLVM 17. Project has been archived as read only and there was no fix/update to it for the last 4 years. We have kept it working for latest llvm versions and python3 support in our fork of it but with LLVM17 more than 50% of our files do not index properly. Enabling deprecated warning broke dxr index completely. As temporary workaround I have add back no-deprecated warning flags for dxr jobs. This allow to generate dxr index but still for 50% of cmssw sources e.g index for https://cmssdt.cern.ch/dxr/CMSSW/source/FWCore/Framework works but there are no search index for https://cmssdt.cern.ch/dxr/CMSSW/source/RecoParticleFlow/PFSimProducer.

Instead of trying to keep this dead project alive we are looking into its replacements [b]. If you know any better open source DXR replacement or if you have used any of the tools in [b] then please do share your experience.

[a] https://cms-talk.web.cern.ch/t/dxr-and-lxr-errors/34274

[b]

Hound: https://github.com/hound-search/hound
Text Sherlock: https://github.com/cbess/text-sherlock
OpenGrok: https://oracle.github.io/opengrok/
- https://github.com/oracle/opengrok/wiki/Installations

cmsbuild commented 4 months ago

cms-bot internal usage

cmsbuild commented 4 months ago

A new Issue was created by @smuzaffar.

@antoniovilela, @sextonkennedy, @rappoccio, @makortel, @Dr15Jones, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

makortel commented 4 months ago

assign core

makortel commented 4 months ago

FYI @wddgit

cmsbuild commented 4 months ago

New categories assigned: core

@Dr15Jones,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

dan131riley commented 4 months ago

I suppose https://searchfox.org ought to be in consideration too?

wddgit commented 4 months ago

Everything you said about DXR makes sense. Does all this also apply to LXR? Does LXR have the same issues and is LXR also an unsupported dead project?

dan131riley commented 4 months ago

Everything you said about DXR makes sense. Does all this also apply to LXR? Does LXR have the same issues and is LXR also an unsupported dead project?

In my minimal research, LXR is still an ongoing project, but doesn't have any institutional support (i.e., it's volunteers), and it doesn't use any of the modern analysis tools.

aandvalenzuela commented 3 months ago

Hello, I am starting to look into https://searchfox.org/ as a possible replacement

iarspider commented 3 months ago

I am testing hound.

iarspider commented 3 months ago

Hound:

pros: easy to run (docker run ...), fast, can handle (with a bit of effort) multiple branches/tags in one repository
cons: just a text search engine (i.e. no identifier search)

smuzaffar commented 3 months ago

cons: just a text search engine (i.e. no identifier search)

we already have LXR as free-text search which can also do identifier search, so hound is not what we need

iarspider commented 3 months ago

Text Sherlock:

Pros: indexer is a separate script, so maybe one can do indexing on build machines
Cons: last commit was 3 years ago, indexing uses whoosh, which was not updated since 2016

tostafin commented 2 months ago

Hi, I created a guide for indexing files in the CMSSW repo in VSCode using clangd. You can check it out here: https://github.com/CTPPS/cmssw_vscode.

cms-sw / cmssw

CMSSW DXR replacement #44446