Support to shard the LSIF output per document and emit begin/end events

[dbaeumer]

Currently the LSIF output is a pure graph with no indication of whether more information will be available for a certain document or symbol. This makes it very hard to decided when information outputted can be optimized for a different storage format (for example to store it into a DB). Currently the whole output must be store (for example on disk) and the post-processed. To ease this and to be able to decided early on if information differs from a previous dump we propose to make the following changes to the LSIF output format:

Emit events

The following events will be emitted:

• begin of a project indexing
• end of a project indexing
• begin of a document indexing
• end of a document indexing

After the end has been emitted no more information regarding the project or document can be emitted. It is allowed to have n projects/documents open for process (necessary to support cyclic references). However the exporting tool should do its best to keep the number of open projects/documents to a minimum.

Shard results per document

Results that are split across documents (for example reference results) are currently emitted using lazy results (e.g. items are added using a item edge from a reference result to a range). This has two disadvantages:

• large output since a lot of item edges are emitted
• hard to determine if the portion of a result belonging to a document has changed compared to the last LSIF export. However this will be eased with the begin/end events already

Since LSIF exporting tools usually process projects on a file by file bases we propose to shard the result per document as well.

[dbaeumer]

From @LukaszMendakiewicz

Please correct me if I'm wrong, I understand that the discussion settled on the following (assuming we are talking about exporter using resultSets).

Element	Frequency
project vertex	one per graph (i.e. separate LSIF file for each project)
document vertex	one per document referenced in project (either directly mentioned in project file or found transitively like C++ headers)
range vertex	one per interesting range (obvious)
resultSet vertex	one per "symbol" (i.e. can be linked to from ranges from multiple documents)
referenceResult	one per "symbol" per document, has all results inlined in its nested array
declarationResult	ditto
definitionResult	ditto
hoverResult	one per "symbol" (effectively one per resultSet)

[LukaszMendakiewicz]

We chatted with Bertan about this part:

referenceResult - one per "symbol" per document

and concluded that splitting referenceResult per document will either need to be required in the spec for him to realize the benefits, or we should not bother with it at all.

I presume the splitting might also be communicated as a "capability" (or maybe rather a "promise") in the dump metaData, if we want to go with capabilities direction already.

[LukaszMendakiewicz]

I'm wondering if it would be beneficial to use monikers also for reference resolution within a project (similar scenario for across projects is discussed in https://github.com/microsoft/lsif-node/issues/41) -- mostly for performance/scalabilty reasons as a function of the output graph size.

This assumes we require per document splitting of referenceResult vertices.

This description would also be equivalent in C++ and C#, AFAIU.

Consider a project consisting of multiple documents (D of them), and using different instantiations of a template/generic (e.g. widget<int>, widget<char>; I of them total). This might be slightly more interesting in C++ where non-type template arguments are permitted, e.g. array<1>, array<2>, ..., array<2000> are all distinct types (but related for the purposes discussed below).

Each instantiation will want to have a different resultSet vertex, at the least to have a place to hang its own unique hoverResult vertex (e.g. one where T is int vs one where T is char), so the graph will end up with I of those.

For FAR purposes though, we want to treat all related instantiations as close enough to collapse in the result set under the single prototype (e.g. widget<T>).

I can imagine four different strategies here:

1) There is one global referenceResult vertex (actually ruled out by requiring per documment splitting, but useful for establishing a baseline). All resultSet vertices for separate instantiations link with textDocument/references to it. 2) There is a referenceResult vertex per document. All resultSet vertices for separate instantiation link to every "related" referenceResult vertex across all documents. 3) There is a referenceResult vertex per document. All resultSet vertices for separate instantiations have monikers and each links only to a subset of referenceResult vertices (e.g. the ones corresponding to the documents where given instantiation is encountered), with stitching of those subgraphs during FAR resolution done through moniker matching. 4) There is a referenceResult vertex per document, and one global referenceResult vertex for grouping the per-document ones together (its referenceResults array contains the IDs of the per-document ones). All resultSet vertices for separate instantiations link to the global one.

The number of entries in the graph for each would then be:

strategy	1	2	3	4
number of referenceResult vertices	1	D	D	1 + D
number of textDocument/references edges	I	I * D	M	I (+ D implicit ones^)

where M maybe could be approximated as I / D on average. ^ - the implicit ones are the ones inlined into the global referenceResult vertex's referenceResults array

3 or 4 looks the most interesting here, with following additional observations:

• 3 makes intra-project resolution more consistent with inter-project resolution
• 4 is more consistent with the current version of the LSIF specification

Please share your thoughts on this.

[dbaeumer]

Regarding partitioning per document: my current thinking is to make this mandatory in the spec. I would also remove the support for lazy results (the one that add elements to a set using a item edge) since it will not be necessary anymore. The granularity for a lazy partition will be a document (not a single range).

Regarding the FAR reference example. I would like to keep support for project scope reference results without monikers. I still think that for simple language servers asking for monikers to get find all references inside a project to work might be a little bit too much. So I am more leaning towards 4.

An alternative idea (in terms modelling) but still in the spirit of 4 is:

We could generalize the refersTo edge from a range to a result set that into something like a next edge which can also link from a result set to a result set. For @LukaszMendakiewicz example this would mean we have the following setup:

• a result set for widget<int> and a result set for widget<char>
• a result set for widget<T>
• a next edge between the result set for widget<int> and widget<T>. Same for result set for widget<char>

The result set for widget<T> would link to the reference and the declaration results which are share between all instantiation of widget<T>

In general we would walk the next edge until we find a corresponding request specific edge or no more next edge is available.

How would we aggregate the reference result partitions to the result set for widget<T>:

1. we can either have separate textDocument/references edges to all reference result partitions
2. we have an aggregator vertex with an ID array pointing to the reference result vertex ids.
3. we introduce the concept of an 1:n edge (e.g from: Id, to: Id[])

I am actually now in favor of 3. since it maps more nicely to the graph. The would mean:

• the result set for widget<T> has an edge textDocument/references to a single referenceResult vertex
• the referenceResult vertex as D 1:n edges to the ranges being part of the reference result.

In total this gives the following number of emitted elements:

• number of referenceResult vertices: 1
• number of textDocument/references edge: 1
• number of resultSet vertices: I + 1
• number of next edges: I
• number of 1:N item edges: D

Total: D + 2*I + 3

[dbaeumer]

Actually I prototypes the 1:n edge concept and it looks promising. Here is a output for a type that is declared at two different locations (however inside the same file):

// Definition result
{"id":10,"type":"vertex","label":"definitionResult"}
// Definition one
{"id":12,"type":"vertex","label":"range","start":{"line":0,"character":17},"end":{"line":0,"character":20},"tag":{"type":"definition","text":"Foo","kind":11,"fullRange":{"start":{"line":0,"character":0},"end":{"line":1,"character":1}}}}
// Definition two
{"id":19,"type":"vertex","label":"range","start":{"line":3,"character":17},"end":{"line":3,"character":20},"tag":{"type":"definition","text":"Foo","kind":7,"fullRange":{"start":{"line":3,"character":0},"end":{"line":4,"character":1}}}}
// Insert them into the definition result
{"id":13,"type":"edge","label":"items","outV":10,"inVs":[12,19]}

Let me know what you think about this approach.

[LukaszMendakiewicz]

For clarity, let's call Dirk's strategy "5", and its three variants "5.1", "5.2", and "5.3".

I agree that "5" is a refinement of "4".

One part that I do not understand in 5.3 is how you want to express the 1:n edge between 1 referenceResult vertex and n range vertices as you were arguing for removing the ability to have lazy references expressed through edges. I understand that with that removal the references would be required to be inlined as an array in referenceResult vertex, so would they be arrays of arrays? How then the per-document partitioning would be realized?

I also have further reservations about 5.3 due to the "multiedge" concept you are proposing which I do not think is a widely recognized concept in the graph theory. I believe this will be problematic theoretically -- "here be dragons" -- in my experience any time we step beyond solid mathematical foundations we ultimately run into unexpected "weirdness" and limitations. More practically, it may also be difficult for various languages trying to host LSIF exporters to re-use their available graph-support libraries, as the "multiedge" is a novel concept that would probably not be available in many.

This is how I imagine 4 and 5 based on the descriptions we have shared. Please correct if I am missing anything in my understanding. Dashed lines represent "implicit" edges, i.e. the arrays of IDs in vertices. Cloud covers the part that I am somehow not sure about what you meant as described above.

(Editable version: LSIF-per-doc-references.pptx)

[dbaeumer]

What I meant with removing the support for lazy references is to add them in any point in time during generating the LSIF output. The once occurring in a document can only be emitted during the begin/end event of the document.

Regarding 1:n edges: what always bothered me with the current ReferenceResult with arrays of RangeIds is that in graphs relationship between vertices are expressed using edges. We basically inlined these edges in ReferenceResult using ID arrays which IMO is not very graph friendly. I did this for performance reason to lower the number of emitted edges. In a pure graph model there would be an edge between a 'ReferenceResult' and the ranges it contains. So my thinking was that introducing a 1:n edge is more graph friendly then inlining the IDs. And they can be easily translated into n 1:1 edges. So IMO we are fine here in terms of the graph theory. Their idea is solely to make the emitted elements more compact (e.g. the size of the dump smaller).

[dbaeumer]

One additional thing to note is that edges can have properties as well. So I would add a documentId property to the 1:n edge that indicates the document to which the edge belongs. So picture (5.3.) will be per document as well.

[LukaszMendakiewicz]

Thanks for clarifying these parts. So if we go with 5.3 then the inlined arrays on referenceResult, declarationResult, and definitionResult will be removed and multiedges required instead?

[BertanAygun]

Couple comments:

• 4 seems to go against idea of having an end document marker since "referenceResult" can only be output at the very end due to implicit edges. ie we can never finish a document until all other documents are processed and individual referenceResults are output for any reference in that document.

• 5.* options don't have the problem as above it looks like. Thinking from size perspective I think both 5.2 and 5.3 would be equivalent since we group edges in to an inline array.

• Just to make sure in case of 5.2/5.3, the resultSet would have the moniker edge still giving a clue on how those referenceresults are joined right?

[LukaszMendakiewicz]

You're right Bertan, 4 would go against that idea if "implicit" edges were used, but if explicit edges were used it should be sound. At that point it would be equivalent to 5.1 modulo vertex and edge labels.

[LukaszMendakiewicz]

This is how I imagine 5.3 clarified after today's discussions:

The two groups of "reference" edges can also optionally be collapsed into two multiedges in the JSON output.

[LukaszMendakiewicz]

I’ve prototyped what we discussed yesterday and ended up with the following graph for an illustrative example project.

Note:

• Big boxes are the per-document partitions, emitted in the numbered order (1, 2, 3) (Graphviz layouting cannot be forced to respect the order AFAIK).
• Vertex 0 (project) should really be outside of any box (I think that’s Graphviz bug that they lay it out inside).

Also attaching the source Graphviz file [graph.txt] if you want to inspect more fine-grained the order of emission (should be easy to read by hand if you notice that clusterX encompasses the begin/end document segment).

Does it look like what you were expecting? Would it be problematic for incremental consumption if the other time around the exporter changed its mind and dumped e.g. "doc 2" before "doc 1"? Note that then the subgraph (up to range vertices) rooted in 3 (resultSet) would be emitted within "doc 2" boundary.

[BertanAygun]

I assume there will be doc id, in edge properties as well which are missing from that last graph right? The 5.3 graph above is more towards what I expected. The ordering or where resultSet is rooted shouldn't matter since to see if 2 documents are same I wouldn't consider edges going to other documents. In fact such graphs would be one of the test cases we use.

[LukaszMendakiewicz]

Yes, some edges would have documentId property -- I have omitted these to reduce noise in the picture, but I guess it would actually clarify better if I had them. Let me generate a new one.

I said "some edges" because I think the documentId property would be only on edges that: 1) have inV of kind range, and 2) have label other than contains (for contains, documentId would be always equal to outV so redundant)

Do we want to optimize this further and with 3) require the property only on edges that are "escaping" the current document (i.e. the edges between clusters in the last picture)?

[BertanAygun]

1 and 2 sounds good, I think #3 is too much of an optimization that causes unnecessary complexity on the output side. In the reader side, I have to parse documentId property anyway to check if it exists or not so I don't think it provides any significant optimization.

[LukaszMendakiewicz]

This is how it looks with documentId properties on edges (in square brackets):

However implementing it gives a feeling that this data point does not really belong on the edge -- there were some hoops I had to jump through to model this in data structures on my side.

I am wondering if it would not be better if documentId was a property on a range vertex instead -- would it be much more complicated or costly on your side Bertan to follow the edge before reading that value? This would also reduce redundancy (in the example above 10 instances of documentId would reduce to 6).

If you want to test it out, I can give you exporter versions going both ways so you can compare them on your side.

[BertanAygun]

In a larger example though I assume the multi-edge will refer to multiple ranges, so having document id in each range might end up causing a larger LSIF output.

From reading cost perspective, what this would mean is that I would have to keep range ids from open documents in memory so that I can quickly find document id given the result -> range edge which I wouldn't have to do otherwise.

The semantics can start to look odd as well, for example can we have a multi edge that points to ranges from multiple documents now, ideally such a multi edge would refer to ranges from same document so readers can only look up the first range but such a requirement would have to rely on documentation instead of spec.

[dbaeumer]

@LukaszMendakiewicz thanks for the great diagrams.

I implemented a prototype for TS as well and I stumbled over similar issues. Here is what I did:

• contains and item are 1:N edges. I even use it now for ranges contain in documents.
• the documentId is more a cluster ID. Since we cluster by document it is the same id. But may be we should make this concept more first class. But I am going back and forth on this.
• I don't thing that we should add a documentId to range. If we want to spare the id on the edge someone can always walk the contains edge to find the document.
• Hover is another interesting thing if a hover is computed on a multi declaration spread over n files. The hover result itself is a single literal and can not be easily split over the n files (clusters). Here I was thinking if the textDocument/hover edge in this case must list the clusters it is depending on so that we can invalidate the right clusters.

I also went over all data we have and hover was so only one that I noticed needs this additional support.

[dbaeumer]

Here is an output for a single file index.ts with the following content:

export interface Foo {
}

export interface Foo {
}

export namespace Foo {
}

{"id":1,"type":"vertex","label":"metaData","version":"0.4.0","projectRoot":"file:///c:/Users/dirkb/Projects/mseng/VSCode/lsif-node/samples/typescript"}
{"id":2,"type":"vertex","label":"project","kind":"typescript"}
{"id":3,"type":"vertex","label":"$event","kind":"begin","scope":"project","data":2}
{"id":4,"type":"vertex","label":"document","uri":"file:///c:/Users/dirkb/Projects/mseng/VSCode/lsif-node/samples/typescript/index.ts","languageId":"typescript","contents":"ZXhwb3J0IGludGVyZmFjZSBGb28gew0KfQ0KDQpleHBvcnQgaW50ZXJmYWNlIEZvbyB7DQp9DQoNCmV4cG9ydCBuYW1lc3BhY2UgRm9vIHsNCn0="}
{"id":5,"type":"vertex","label":"$event","kind":"begin","scope":"document","data":4}
{"id":6,"type":"vertex","label":"resultSet"}
{"id":7,"type":"vertex","label":"referenceResult"}
{"id":8,"type":"edge","label":"textDocument/references","outV":6,"inV":7}
{"id":10,"type":"vertex","label":"definitionResult"}
{"id":11,"type":"edge","label":"textDocument/definition","outV":6,"inV":10}
{"id":12,"type":"vertex","label":"range","start":{"line":0,"character":17},"end":{"line":0,"character":20},"tag":{"type":"definition","text":"Foo","kind":11,"fullRange":{"start":{"line":0,"character":0},"end":{"line":1,"character":1}}}}
{"id":13,"type":"edge","label":"next","outV":12,"inV":6}
{"id":14,"type":"vertex","label":"hoverResult","result":{"contents":[{"language":"typescript","value":"interface Foo\nnamespace Foo"}]}}
{"id":15,"type":"edge","label":"textDocument/hover","outV":6,"inV":14}
{"id":16,"type":"vertex","label":"range","start":{"line":3,"character":17},"end":{"line":3,"character":20},"tag":{"type":"definition","text":"Foo","kind":11,"fullRange":{"start":{"line":3,"character":0},"end":{"line":4,"character":1}}}}
{"id":17,"type":"edge","label":"next","outV":16,"inV":6}
{"id":18,"type":"vertex","label":"range","start":{"line":6,"character":17},"end":{"line":6,"character":20},"tag":{"type":"definition","text":"Foo","kind":7,"fullRange":{"start":{"line":6,"character":0},"end":{"line":7,"character":1}}}}
{"id":19,"type":"edge","label":"next","outV":18,"inV":6}
{"id":20,"type":"edge","label":"item","outV":10,"inVs":[12,16,18],"document":4}
{"id":21,"type":"edge","label":"item","outV":7,"inVs":[12,16,18],"document":4,"property":"definitions"}
{"id":22,"type":"vertex","label":"moniker","kind":"export","scheme":"tsc","identifier":"lib/index:Foo"}
{"id":23,"type":"edge","label":"moniker","outV":6,"inV":22}
{"id":24,"type":"edge","label":"contains","outV":4,"inVs":[12,16,18]}
{"id":25,"type":"vertex","label":"foldingRangeResult","result":[{"startLine":0,"startCharacter":20,"endLine":1,"endCharacter":1},{"startLine":3,"startCharacter":20,"endLine":4,"endCharacter":1},{"startLine":6,"startCharacter":20,"endLine":7,"endCharacter":1}]}
{"id":26,"type":"edge","label":"textDocument/foldingRange","outV":4,"inV":25}
{"id":27,"type":"vertex","label":"documentSymbolResult","result":[{"id":12},{"id":16},{"id":18}]}
{"id":28,"type":"edge","label":"textDocument/documentSymbol","outV":4,"inV":27}
{"id":29,"type":"vertex","label":"$event","kind":"end","scope":"document","data":4}
{"id":30,"type":"edge","label":"contains","outV":2,"inVs":[4]}
{"id":31,"type":"vertex","label":"$event","kind":"end","scope":"project","data":2}

I called the property on the edge document since no edge properties have Id in its named although the refer to vertices ids.

It also has the begin and end events. In addition I renamed refersTo to next. The moniker moved from range to resultSet but only if all monikers for all declaration ranges are the same.

[dbaeumer]

Here is an example for two ts files with an exported function with is referenced on both sides:

provider.ts

export function foo(): void {
}

foo();

index.ts

import { foo } from './provide';

foo();

{"id":1,"type":"vertex","label":"metaData","version":"0.4.0","projectRoot":"file:///c:/Users/dirkb/Projects/mseng/VSCode/lsif-node/samples/typescript"}           
{"id":2,"type":"vertex","label":"project","kind":"typescript"}                                                                                                    
{"id":3,"type":"vertex","label":"$event","kind":"begin","scope":"project","data":2}                                                                               
{"id":4,"type":"vertex","label":"document","uri":"file:///c:/Users/dirkb/Projects/mseng/VSCode/lsif-node/samples/typescript/provide.ts","languageId":"typescript",
"contents":"ZXhwb3J0IGZ1bmN0aW9uIGZvbygpOiB2b2lkIHsNCn0NCg0KZm9vKCk7"}                                                                                            
{"id":5,"type":"vertex","label":"$event","kind":"begin","scope":"document","data":4}                                                                              
{"id":6,"type":"vertex","label":"resultSet"}                                                                                                                      
{"id":7,"type":"vertex","label":"referenceResult"}                                                                                                                
{"id":8,"type":"edge","label":"textDocument/references","outV":6,"inV":7}                                                                                         
{"id":10,"type":"vertex","label":"definitionResult"}                                                                                                              
{"id":11,"type":"edge","label":"textDocument/definition","outV":6,"inV":10}                                                                                       
{"id":12,"type":"vertex","label":"range","start":{"line":0,"character":16},"end":{"line":0,"character":19},"tag":{"type":"definition","text":"foo","kind":12,"full
Range":{"start":{"line":0,"character":0},"end":{"line":1,"character":1}}}}                                                                                        
{"id":13,"type":"edge","label":"next","outV":12,"inV":6}                                                                                                          
{"id":14,"type":"vertex","label":"hoverResult","result":{"contents":[{"language":"typescript","value":"function foo(): void"}]}}                                  
{"id":15,"type":"edge","label":"textDocument/hover","outV":6,"inV":14}                                                                                            
{"id":16,"type":"vertex","label":"range","start":{"line":3,"character":0},"end":{"line":3,"character":3},"tag":{"type":"reference","text":"foo"}}                 
{"id":17,"type":"edge","label":"next","outV":16,"inV":6}                                                                                                          
{"id":18,"type":"edge","label":"item","outV":10,"inVs":[12],"document":4}                                                                                         
{"id":19,"type":"edge","label":"item","outV":7,"inVs":[12],"document":4,"property":"definitions"}                                                                 
{"id":20,"type":"edge","label":"item","outV":7,"inVs":[16],"document":4,"property":"references"}                                                                  
{"id":21,"type":"vertex","label":"moniker","kind":"export","scheme":"tsc","identifier":"lib/provide:foo"}                                                         
{"id":22,"type":"edge","label":"moniker","outV":6,"inV":21}                                                                                                       
{"id":23,"type":"edge","label":"contains","outV":4,"inVs":[12,16]}                                                                                                
{"id":24,"type":"vertex","label":"foldingRangeResult","result":[{"startLine":0,"startCharacter":27,"endLine":1,"endCharacter":1}]}                                
{"id":25,"type":"edge","label":"textDocument/foldingRange","outV":4,"inV":24}                                                                                     
{"id":26,"type":"vertex","label":"documentSymbolResult","result":[{"id":12}]}                                                                                     
{"id":27,"type":"edge","label":"textDocument/documentSymbol","outV":4,"inV":26}                                                                                   
{"id":28,"type":"vertex","label":"$event","kind":"end","scope":"document","data":4}                                                                               
{"id":29,"type":"vertex","label":"document","uri":"file:///c:/Users/dirkb/Projects/mseng/VSCode/lsif-node/samples/typescript/index.ts","languageId":"typescript","
contents":"aW1wb3J0IHsgZm9vIH0gZnJvbSAnLi9wcm92aWRlJzsNCg0KZm9vKCk7"}                                                                                             
{"id":30,"type":"vertex","label":"$event","kind":"begin","scope":"document","data":29}                                                                            
{"id":31,"type":"vertex","label":"resultSet"}                                                                                                                     
{"id":32,"type":"vertex","label":"definitionResult"}                                                                                                              
{"id":33,"type":"edge","label":"textDocument/definition","outV":31,"inV":32}                                                                                      
{"id":34,"type":"vertex","label":"range","start":{"line":0,"character":9},"end":{"line":0,"character":12},"tag":{"type":"definition","text":"foo","kind":7,"fullRa
nge":{"start":{"line":0,"character":9},"end":{"line":0,"character":12}}}}                                                                                         
{"id":35,"type":"vertex","label":"hoverResult","result":{"contents":[{"language":"typescript","value":"(alias) function foo(): void\nimport foo"}]}}              
{"id":36,"type":"edge","label":"textDocument/hover","outV":31,"inV":35}                                                                                           
{"id":37,"type":"vertex","label":"range","start":{"line":2,"character":0},"end":{"line":2,"character":3},"tag":{"type":"reference","text":"foo"}}                 
{"id":38,"type":"edge","label":"next","outV":37,"inV":31}                                                                                                         
{"id":39,"type":"edge","label":"item","outV":7,"inVs":[37],"document":29,"property":"references"}                                                                 
{"id":40,"type":"edge","label":"contains","outV":29,"inVs":[34,37]}                                                                                               
{"id":41,"type":"vertex","label":"$event","kind":"end","scope":"document","data":29}                                                                              
{"id":42,"type":"edge","label":"contains","outV":2,"inVs":[4,29]}                                                                                                 
{"id":43,"type":"vertex","label":"$event","kind":"end","scope":"project","data":2}                                                                                

The new model with next edges will make the import model nicer as well. In TS imports are type aliases which in general have new monikers. I do not emit this right now to nices share the reference and declaration result. But with the new model I can easily link the resultSet for the type alias to the real thing.

Interesting nodes are the reference partitions emitted as with id 19, 20 and 39

[dbaeumer]

I also continued thinking about the hover. Another approach would be to treat hovers as their own cluster. This will allow to update them independently. In the DB we would reach to a hover using a moniker instead of the edge if a hover is bound to a symbol having a moniker.

[BertanAygun]

For hover, I didn't imagine we would split it per document and instead point to same vertex but I realize that also means a document would be considered changed if hover result changed externally.

I like the moniker based approach and was going to recommend it too. In absance of that another option for readers could be to use the range moniker for the definition range that points to the hover result. I don't know if the assumption of hover info being linked to moniker of definition range is correct for all languages though.

[LukaszMendakiewicz]

There will be a lot of resultSets that will not have definition ranges, so you cannot rely on hanging hoverResult there.

If the hover is looked up based on moniker, would such look up be scoped to only the local LSIF file?

There also might be problems where hover does not map 1:1 to a moniker. For example:

a.cpp

void f(int i = 1);

b.cpp

void f(int i = 2);

Here f is the same symbol in both files, but has different default argument value in each. FAR though finds results in both files as equal, so both should have the same moniker. Hover though might want to be sensitive to show the default argument value specific to a given context, so would be different between the two files.

Before one could have modeled this as:

[BertanAygun]

Since we had already considered different monikers for reference/implementation results, could we apply the same to hover? I am guessing the cases like above are rarer than normal cases where hover is frequently shared.

Btw if I understood Dirk's proposal correctly, we can still have edges as above without monikers but that would mean the document is considered updated if hover content changes.

[dbaeumer]

I agree with @BertanAygun and it is actually what I have in mind: we use the moniker to store the hover in the DB not in the LSIF format. So the graph from here https://github.com/microsoft/lsif-node/issues/39#issuecomment-500099440 is what we should emit. Plus the following:

• for the resultSet for void f(int i = 1); we use a moniker f,int=1
• for the resultSet for void f(int i = 2); we use a moniker f,int=2
• for the resultSet pointing to the referenceResult. We use a moniker f,int

[LukaszMendakiewicz]

Taking a stab at formalizing the rule we are discussing here:

If a resultSet is linked to (possibly transitively through other resultSets) from ranges belonging to more than one document and the resultSet has a link to a meaningful result vertex (hoverResult, declarationResult, definitionResult, referenceResult, etc.) then the resultSet is required to have a link to a moniker vertex.

Accurate?

Then practically, when resolving a cross-(document/project/graph) relationship, the moniker closest to the meaningful result vertex for a given operation will be used.

Fair?

[dbaeumer]

+1

[dbaeumer]

Please keep in mind that dumps without monikers will still be valid. However those will only be useful for local use.