New files added to the project not indexed

[realSaltyFish]

I'm using OmniSharp on Neovim for Unity development. I have noticed that each time I create a new script file in Unity, that file won't be indexed by OmniSharp and I have to run :OmniSharpRestartServer for it to be indexed. Given that restarting the server takes a very long time (more than one minute), it's a very annoying problem.

I am using Arch Linux, Neovim 0.8.0 and coc.nvim. I have set let g:OmniSharp_server_use_mono = 1.

Basic language server features work, but when I refer to other classes in the project, OmniSharp does not work at all:

I also cannot refer to the new class from other files. OmniSharp complains that the class definition couldn't be found:

Interestingly the logs indicate that the creation of new file was indeed detected:

No errors or warnings were reported in the log.

[nickspoons]

I think this is really a server issue, I don't think there's anything that OmniSharp-vim can do here, although I may be wrong, I haven't done any .net Framework work on Linux for a while.

Can you share the log please? Even better, provide a small repro project to test with.

Which version of mono do you have installed?

[nickspoons]

Do Unity projects list every .cs file in the .csproj file?

[realSaltyFish]

Can you share the log please? Even better, provide a small repro project to test with.

There is really nothing in there, just the normal startup process. (Is there a "verbose mode" for OmniSharp server?) I can create a new test project and upload the logs later if that will be helpful.

To reproduce:

• Create a new Unity project and configure Unity to use Neovim as code editor. I had to create a "proxy" shell script named "code" to trick Unity into thinking I'm using VS Code, otherwise it doesn't provide the option to generate the .csproj files. The shell script calls nvr so that all files are opened within the same Neovim session.
• Generate the .csproj files.
• Create a script A.cs.
• Double click A.cs file to open it. OmniSharp loads normally.
• Go back to Unity editor and create another script file B.cs.
• Double click B.cs to open it, and the problem appears.

Note that if I quit Neovim and restart it in any way (either by opening a script from Unity editor or from the command-line), or restart the OmniSharp server, B.cs is indexed correctly.

Which version of mono do you have installed?

Mono JIT compiler version 6.12.0

Do Unity projects list every .cs file in the .csproj file?

Not sure what you mean. Typically Unity generates a bunch of .csproj files for its own libraries. These files' names start with Unity. or UnityEngine..

[nickspoons]

Create a new Unity project

Ok but installing Unity and configuring it as described is a huge job. I just want to see what the sln/project files look like.

Do Unity projects list every .cs file in the .csproj file?

Not sure what you mean

I mean, inside the .csproj file itself, is every single .cs file listed in an "Includes" property? .NET Framework used to work like that, I don't know if it still does

Do you also need to restart in the same way if you use vscode as the editor?

[nickspoons]

Is there a "verbose mode" for OmniSharp server?

" Use lower case 'debug' to also see raw JSON requests and responses
let g:OmniSharp_loglevel = 'DEBUG'

I'm not sure if it's necessary here though. I'm interested in seeing the msbuild and mono resolution etc.

[realSaltyFish]

Here are all the CSharp-related stuff in a new project. Let me know if it's not enough. Test.zip

In particular, the file Assembly-CSharp.csproj contains something like this:

Is that what you are looking for?

[realSaltyFish]

Do you also need to restart in the same way if you use vscode as the editor?

Never used vscode before, but I just tested with this plugin (I unchecked "use modern net" and put in "mono path" to make sure it's using mono):

It works and I don't need to restart anything. New script files are correctly indexed.

[nickspoons]

Ok if it's working correctly in vscode then there must be a difference between how OmniSharp-vim and vscode communicate with the server (it's the same server). I'll have a go with the repro project later on, thanks.

[nickspoons]

By the way, you do see the problem behaviour when running the test project, right?

[realSaltyFish]

Yes I do. Perhaps due to the reason you just mentioned, it also doesn't work if you create new files using :e or just touch them from the command-line.

I'm very new to CSharp so I don't really know what's the "correct" way to create new .cs files, or anything about how the CSharp build system works. But please feel free to request more info on my end.

[nickspoons]

Well the old .NET Framework projects were pretty clumsy, now all you do is create the file and you're away. Unfortunately Unity is still using .NET Framework so files are individually listed in the .csproj as you've seen. Vscode apparently has a way of updating the project without restarting it so I'll try to see what they do. I'll just be comparing the logs from vscode and OmniSharp-vim to see what the difference is.

[nickspoons]

So, how can you see that new files are "correctly indexed" when you're using vscode? I've been trying and I'm not seeing new files being recognised by vscode - new methods and classes in new files are not available as completions or usages from existing files. AFAIR when working with .NET Framework projects, I always had to add the new file to the project and restart the language server before getting that kind of support. I think this is a limitation of .NET Framework tooling.

[realSaltyFish]

I did some more tests, and it turns out that vscode also doesn't recognize the new file if you create it directly in vscode. However if you create the file in Unity, vscode immediately provides related completion items. Any hints on what might be happening here?

[nickspoons]

No I don't know, and I don't have Unity installed so I can't check. Unity will be updating the .csproj file to include the new file. Maybe vscode has a watcher on the csproj, and then perhaps re-loading just that project (instead of the whole solution)? You could look at the OmniSharp-roslyn logs in vscode to try to see what it's doing.

[realSaltyFish]

I used fswatch to monitor filesystem changes when I create a script in Unity Editor versus in vscode, and based on what we discussed I do think this is the case. Can OmniSharp-vim do similar things (reload a single file)?

[realSaltyFish]

This is what I think matters in the OmniSharp-roslyn logs. Note that it's reloading the .csproj files modified by Unity. Such log lines never appeared when I use vim.

[nickspoons]

There isn't currently any functionality for reloading/updating a single project in OmniSharp-vim. I'll look into it, it shouldn't be too complicated. Watching a csproj file is outside OmniSharp-vim scope, but restarting individual projects on demand should be fast and a good addition.

[nickspoons]

Would you be able to set the vscode OmniSharp-roslyn loglevel to debug, and see what endpoints are being called? A request JSON would be very helpful

[realSaltyFish]

This log file includes everything that happened after I create the new file. I see some JSON requests in there- hope it's helpful.

omnisharp.log

[realSaltyFish]

Is it true that OmniSharp-roslyn actually does the monitoring and informs the vscode plugin about all file changes it detects? If so I think it's not too difficult to do exactly what the vscode plugin does, i.e. reloading related project files automatically.

[nickspoons]

No, if that were happening this would already be working. The file watching is done by the omnisharp-vscode plugin I believe. I suspect that vscode itself already has some file watching tools built in.

[nickspoons]

I've been playing a bit with this and have made some progress. All changes still local, I'll make a PR when I have some time.

I've added a command to reload the current project by sending a /filesChanged request for the .csproj file to the server. This works quite well, it's fast and causes omnisharp-roslyn to "see" the new files that have been included in a legacy project.

There are some problems though. The way we get the current project name is to make a /project request for a file. However, newly added files are not currently associated with a project so they fail ... catch 22. An alternative could have been to search up the directly tree for the .csproj, but your test example has all of the .csproj files in the same directory, so that won't help. So I'll add a way to explicitly reload a project but it's not quite as tidy: :OmniSharpReloadProject path/to/my.csproj which will do the trick.

I'm not sure that there will be a tidier solution than this. Barring filewatchers on the .csproj, which I think is difficult (impossible?) without using external tools, and then we have dependency/platform issues. Also, this issue only applies to .NET Framework projects which are becoming obsolete (mostly only used by Unity these days, and they are preparing to move away to the new project types). In dotnet core projects files are automatically added to the new project, and from a running OmniSharp-vim instance the new file just needs to be opened to be indexed, there's no need to reload the project.

[realSaltyFish]

Appreciate your work! In my use case the :OmniSharpReloadProject command is already enough. I think I'll run fswatch outside vim and use nvr to remotely execute the reload command in vim once a change to a .csproj file is detected. Not to mention the only project I'll ever need to reload seems to be the Assembly-CSharp.csproj.

I agree that there is not much significance in implementing it in a "better" way as perhaps this issue only affects Unity users. But I am willing to discuss more if it could be done better. I don't quite understand what you mean by "get the current project name". In my use case I load a solution file which brings in a bunch of .csproj files. How do I determine which is the "current" one?

[nickspoons]

How do I determine which is the "current" one?

Exactly 😄. In your repro there are loads of projects but they're mostly Unity things. The existing .cs file is only associated with one project, which is the one you want the new files added to. From the existing .cs file we can ask OmniSharp-roslyn which project it is associated with and get the correct answer. However, as OmniSharp-roslyn doesn't yet know about the new file, it can't tell us which project it is associated with.

I could imagine a slightly awkward workflow where you are a new file in Unity, then in OmniSharp-vim go to an existing.cs file, and from there reload the project, and then open the new file, which now should be included in the project.

[nickspoons]

I haven't had a lot of time lately but I'll try to finish this command off tonight and we can see what works.

[realSaltyFish]

I could imagine a slightly awkward workflow where you are a new file in Unity, then in OmniSharp-vim go to an existing.cs file, and from there reload the project, and then open the new file, which now should be included in the project.

I began to understand now. To me it seems that traversing the file tree to find all .csproj files is the best way to do "semi-automatic" project reloading. It should be fairly easy using grep, and can be hooked to BufReadPre. So when a new file is created and opened in vim, OmniSharp can locate the project file it belongs to and reload that project. It's unlikely that I create a new file and never edit it before using it in other files anyway.

I can try to implement this part when the reload command is done. There are a lot of things that can go wrong but drafting a proof of concept should be quick.

Or I can also live with explicitly reloading Assembly-CSharp.csproj whenever I create a new file. That's somehow dirty but it solves my issue.

[nickspoons]

We do have a list of "all" project files in OmniSharp-vim already, I'll get you a command later on. But yeah grep is a totally sensible way to script something up

[nickspoons]

We do have a list of "all" project files in OmniSharp-vim already, I'll get you a command later on

@RealSaltyFish here is a one-liner to get all of the project files available for the solution:

echo map(copy(OmniSharp#GetHost(bufnr()).job.projects), {_,p -> p.path})

[nickspoons]

@RealSaltyFish I've created PR #819, do you want to try it out and see if it will make the adding-file process faster?

[realSaltyFish]

here is a one-liner to get all of the project files available for the solution:

This is good, but I also need to look inside all these project files to see which one contains the new .cs file. grep can do that for me. What do you think?

[realSaltyFish]

I've created PR #819

Thank you! I pulled it and it works. This command is super helpful. Reloading a project costs ~3s - much better than reloading the entire solution.

I'm working on writing and integrating my own file watcher script but that's out of scope for this project ; )

Update: I was able to get vscode-like experience by running in the background a simple bash script that calculates the hash of the project file once every 5 seconds and sends the reload command via nvr when a change is detected.

Some additional comments:

• Is the project file path relative to the cwd in vim?
• It might be a good idea to report an error if the specified file does not exist.
• Is it possible to print a message to indicate that the reload is completed successfully?

[nickspoons]

Ok that's great that it works. Your background script sounds good but don't you think it'd be simpler to use something like inotifywait or entr?

• Is the project file path relative to the cwd in vim

The path can be relative to the cwd or absolute. When you tab-complete the command, the options are all relative.

• It might be a good idea to report an error if the specified file does not exist.

Good idea

• Is it possible to print a message to indicate that the reload is completed successfully?

I have a TODO comment (not committed) to do exactly this, echoing a message when complete, and triggering an autocmd (or calling a callback).

[realSaltyFish]

don't you think it'd be simpler to use something like inotifywait or entr?

Yeah I tried those but Unity modifies the file 10+ times when a new script is created and I don't want to reload the project 10+ times : (

When you tab-complete the command, the options are all relative.

Oh so there is tab completion, super cool!

I have a TODO comment (not committed) to do exactly this

Awesome

Also: Do you think using grep instead of a /project query will be helpful? I feel like it does no harm anyway, and can help deal with newly-created files. But it requires the new file to be added to the project file to work. Unity does that but I already have a better solution for Unity... Not sure if there are other potential use cases and how the workflow might be like.

[nickspoons]

Yeah I tried those but Unity modifies the file 10+ times when a new script is created and I don't want to reload the project 10+ times : (

Makes sense!

Do you think using grep instead of a /project query will be helpful?

Do you mean for the OmniSharp-vim solution? It would make things easier in some ways but it's platform dependent so is best avoided.

[nickspoons]

Hmm. Thinking about this, vim itself could pretty easily open those csproj files and search them, and that would be platform independent.

However, it might make more sense to add that functionality to vim-sharpenup, which is an OmniSharp-vim support/extension plugin and already contains some similar functionality (adding and removing files to .csproj)

[realSaltyFish]

Hmm. Thinking about this, vim itself could pretty easily open those csproj files and search them, and that would be platform independent.

Fair - I haven't used windows for years so didn't even think about cross-platform issues.

However, it might make more sense to add that functionality to vim-sharpenup

Sounds interesting, gonna check it out!

[nickspoons]

@RealSaltyFish I've added a couple of commits to the PR. The first does a filereadable() check on the project file as you suggested.

The second updates the job loaded properties, outputs a message like "Reloaded Assembly-CSharp.csproj in 3.2s" and triggers the OmniSharpReady autocmd. I use the vim-sharpenup statusline so this means my statusline also shows that the server changes from a "Ready" state to a "Loading" state and back to "Ready" when the reload is complete.

[nickspoons]

OK I've merged #819. I haven't looked into the .csproj grepping yet but as mentioned earlier, my current inclination is to not include that here, but possibly in vim-sharpenup.