Detecting changes in other files

[wojciech-kulik]

This is most likely connected with #43.

Besides the problem with new files, there is also one more annoying. Whenever I update some other file by for example adding a new parameter to a function, these changes are not visible for LSP until I rebuild the project and restart LSP.

Actually, LSP restart is not needed if I alternatively just update buffers with errors. However, diagnostics won't disappear automatically until some changes are made.

I wonder how Xcode handles it. Does it recompile just a single file? Because Xcode is able to almost instantly notice new params.

Running a build in a background for every change in a buffer, doesn't seem reasonable, so probably it is more difficult case than #43.

[SolaWing]

xcode does do background index. even more, if you open same file with xcode and keep it in background, you will notice your symbols is updated automatically too. xcode may only run only single file command, not the entire build and precondition handle so it's fast to only update index. @fireplusteam seems may familiar more. I may try to rerun the single file build command to see if it works..

[wojciech-kulik]

Would be great to try. If it works, it would significantly improve the work :)

[fireplusteam]

@SolaWing , on a file change xCode starts a build of a file with regards to dependencies of this file. If it requires some modules rebuilt first, it would rebuild it first, but yes in the end the whole project has to be updated to make full symbols work. It's faster because Xcode doesn't release XCBuildService each time after rebuild, so it can cache any data to speed up the response.

@wojciech-kulik actually using a proxy you can tell XCBBuildService to do the same. You just need to send a request which is in fact is JSON converted to binary format. Saying you can set some environmental variables and start the request with xcodebuild but change the JSON on a fly and imitate the indexing feature. Anyway for cross-modules you need to rebuild to update frameworks. So because of that, I don't use that feature but directly start to rebuild a project in two cases:

1. The project file or some of the subproject files are changed
2. A user made a change in a module and switched to another one in your editor

If you made a change in a single module, indexing is working fine within that module but not for others, that's why manipulating with flags for a newly added file is not useful because we have a lot of use cases that can be resolved with the full rebuild as only XCBBuildService knows how to resolve them.

There are several ways to decrease response time you can: a) async parser of logs b) keep xcbuildservice process and don't restart it each time when you trigger a build with xcodebuild. It does use some caching internally to speed up the time response but it's released each time after a manual build. Xcode communicates with it like with a service and keeps it until Xcode is closed. (But that solution requires a proxy and private implementation of it)

[SolaWing]

We can discuss how to background Indexing here. background indexing also fix the detecting changes issue.

I have a few vague ideas for implementation:

1. rerun command from logs directly, which is fast but may have some dependency compatibility issues
2. run xcodebuild in background (may only within a minimum targets, not all code). xcodebuild can help deal with dependency issues
3. hack the index call inside xcode. But I don't like this hack method that is strongly coupled with xcode internal implementation

[yaroslavyaroslav]

I believe this is related to this. Haven't dug it yet even with SPM, so not so sure if it works with Xcode build at all. But anyway, Sourcekit-lsp in Swift 6.0 brings background indexing support.

https://github.com/apple/sourcekit-lsp/pull/1416

[fireplusteam]

@yaroslavyaroslav , in our case, it would not be working. @SolaWing , I use 2'd approaches and it's the best. I trigger to rebuild the whole project as it's the faster way to update the indexes. First I tried to optimize it and only triggered only for a few targets but in that case xcodebuild can not maintain a cache properly so it ended up in longer time to update the indexes with a project with a lot of targets. As a side effect of this approach you always have an up to date project when you want to run it on a simulator. The only things are:

1. to implement async parser of build logs on xcode build server side so it's updating while building
2. if we find a way not to re-create XCBuildService each time when xcodebuild is triggered it can save time but in that case, we need to proxy this private service. (as xCode holds this service all the time and it caches some data structure for faster builds).

So from xcode build server we need just an async parser of building logs

[wojciech-kulik]

@SolaWing Regarding the second approach. I'm a little bit worried that this build running in the background will be interfering with user actions. What if a user runs another build at the same time? I guess the build may fail with random errors then.

[fireplusteam]

XCBuildService is working like a service, it like accepts the request and triggers some build actions. But it's not easy to implement this and it's something that can be achieved as an additional option for a user. for example in my extension a user may use or not use it (currently it's only used to provide as much compile flags as possible no matter how many errors you have in the project), so it's just up to him to take this additional risk :).

[fireplusteam]

@wojciech-kulik , also in my extension I just cancel an indexing build when a user wants to run it's own, so it's always only one build in a single time, otherwise xcodebuild tells you that a database is locked with another building process. You're not allowed to perform two builds at the same time.

[fireplusteam]

also we have an option like dry run in XCBBuildService, it's private and it only generates dependencies and compile flags without actually compiling the sources, it's private and useless if you have multimodular project because you still need to build all the targets to make indexing working properly

[wojciech-kulik]

@wojciech-kulik , also in my extension I just cancel an indexing build when a user wants to run it's own, so it's always only one build in a single time, otherwise xcodebuild tells you that a database is locked with another building process. You're not allowed to perform two builds at the same time.

Exactly, I had this issue on my mind.

2. run xcodebuild in background (may only within a minimum targets, not all code). xcodebuild can help deal with dependency issues

So in order to implement this, we would need to be able to cancel this background build from the outside. For example, as I provide features to build the app, I need to be able to cancel this background build before running my own.

[fireplusteam]

I believe running background build is not something that we want to be a part of xcode build server as there can be additional difficulties with passing all that required args to make a build and then cancel it, for me only the async parser is nice to have :)