search: explore structural search alternatives/future

[keegancsmith]

In a recent discussion about the future of structural search / our use of SEMGREP the following opportunities were identified. This issue is for further discussion / to not forget about this.

• Running the SEMGREP CLI via a Batch Change spec / executors
  • A batch step could be committing the SEMGREP CLI output file to a repo and pushing it to their code host
• Testing Structural Search with Search Jobs (slow is fine, as long as it's complete)
• Building a SEMGREP UX in Sourcegraph

@keegancsmith: I think being SEMGREP specific is not a good use of our platform. I think making something which makes it easy for users to take advantage of our platform is the way to go. IE we have great infra for computing and storing results from all code. We could build something which demonstrates that it would be easy to use SEMGREP (in a container with our remote executors?).

https://sourcegraph.slack.com/archives/C1JH2BEHZ/p1699293562878619?thread_ts=1698736550.260449&cid=C1JH2BEHZ

cc @marcleblanc2 @sourcegraph/search-platform

/cc @sourcegraph/search-platform

[camdencheek]

Just dropping a link to a prototype and accompanying slack thread I've been hacking on to introduce structural capabilities into regex. Julie mentioned it seems relevant 🙂