search

CoatiSoftware / Sourcetrail

Sourcetrail - free and open-source interactive source explorer
https://www.sourcetrail.com/
GNU General Public License v3.0
14.44k stars 1.34k forks source link

SourceTrail unable to create project as unprivileged OR privileged user on Windows #1002

Open assarbad opened 4 years ago

assarbad commented 4 years ago

Version affected: 2020.1.117 (the installed version, not portable one) Windows version: 10.0.18363 Build 18363 (x64, obviously)

The error I receive is:

Error on accessing shared memory. Indexing not possible. Please restart computer or run as admin: boost::interprocess::intermodule_singleton initialization failed

The log file looks like this:

17:31:55 | 2284 | SharedMemory.cpp:197 SharedMemory::SharedMemory() | ERROR: boost exception thrown at shared memory creation - srctrlmem_grbg_cllctr_64: Reached the end of the file.
17:31:55 | 2284 | SharedMemoryGarbageCollector.cpp:38 SharedMemoryGarbageCollector::createInstance() | ERROR: boost exception thrown at shared memory garbage collector: Reached the end of the file.
17:32:25 | 2284 | ApplicationSettingsPrefiller.cpp:84 ApplicationSettingsPrefiller::prefillMavenExecutablePath() | INFO: Prefilling Maven path
17:32:25 | 2268 | INFO: send MessageStatus Starting Sourcetrail 64 bit, version 2020.1.117
17:32:25 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Starting Sourcetrail 64 bit, version 2020.1.117
17:32:25 | 2268 | INFO: send MessageStatus Load settings: C:/Users/o.schneider/AppData/Local/Coati Software/Sourcetrail 64-bit/ApplicationSettings.xml
17:32:25 | 2284 | utilityQt.cpp:66 utility::loadFontsFromDirectory() | INFO: Loaded FontFamily: Fira Sans
17:32:25 | 2284 | utilityQt.cpp:66 utility::loadFontsFromDirectory() | INFO: Loaded FontFamily: Fira Sans
17:32:25 | 2284 | utilityQt.cpp:66 utility::loadFontsFromDirectory() | INFO: Loaded FontFamily: Source Code Pro
17:32:25 | 2284 | utilityQt.cpp:66 utility::loadFontsFromDirectory() | INFO: Loaded FontFamily: Source Code Pro
17:32:25 | 2284 | utilityQt.cpp:66 utility::loadFontsFromDirectory() | INFO: Loaded FontFamily: Source Code Pro
17:32:25 | 2284 | utilityQt.cpp:66 utility::loadFontsFromDirectory() | INFO: Loaded FontFamily: Roboto
17:32:25 | 2284 | utilityQt.cpp:66 utility::loadFontsFromDirectory() | INFO: Loaded FontFamily: Roboto
17:32:26 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Load settings: C:/Users/o.schneider/AppData/Local/Coati Software/Sourcetrail 64-bit/ApplicationSettings.xml
17:32:26 | 2268 | INFO: send MessageStatus Enabled console and file logging.
17:32:26 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Enabled console and file logging.
17:32:26 | 2268 | INFO: send MessagePingReceived 
17:32:26 | 2268 | INFO: send MessageStatus Ran Maven executable path detection, no path found.
17:32:26 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Ran Maven executable path detection, no path found.
17:32:26 | 2268 | INFO: send MessageLoadProject , settingsChanged: false, refreshMode: 0
17:32:26 | 2268 | INFO: send MessageWindowFocus 
17:32:26 | 2268 | INFO: send MessageWindowFocus 
17:32:26 | 2268 | INFO: send MessageWindowFocus 
17:32:26 | 2268 | INFO: send MessageWindowFocus 
17:32:26 | 2268 | INFO: send MessageWindowFocus 
17:32:26 | 2268 | INFO: send MessageWindowFocus 
17:32:26 | 2268 | INFO: send MessageWindowFocus 
17:32:26 | 2268 | INFO: send MessagePingReceived 
17:32:56 | 2268 | INFO: send MessagePingReceived 
17:32:57 | 2268 | INFO: send MessageWindowFocus 
17:33:16 | 2268 | INFO: send MessageWindowFocus 
17:33:26 | 2268 | INFO: send MessagePingReceived 
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\14.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\14.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VCExpress\14.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VCExpress\14.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\12.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\12.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VCExpress\12.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VCExpress\12.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\11.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\11.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VCExpress\11.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VCExpress\11.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\10.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VCExpress\10.0
17:33:47 | 2284 | CxxVs10To14HeaderPathDetector.cpp:101 CxxVs10To14HeaderPathDetector::getVsInstallPathUsingRegistry() | INFO: Found working regestry key for VS install path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VCExpress\10.0
17:33:49 | 2268 | INFO: send MessageWindowFocus 
17:33:56 | 2268 | INFO: send MessagePingReceived 
17:34:04 | 2268 | INFO: send MessageWindowFocus 
17:34:26 | 2268 | INFO: send MessagePingReceived 
17:34:49 | 2268 | INFO: send MessageStatus Created project: C:/work/project-name/123.srctrlprj
17:34:49 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Created project: C:/work/project-name/123.srctrlprj
17:34:49 | 2268 | INFO: send MessageLoadProject C:/work/project-name/123.srctrlprj, settingsChanged: false, refreshMode: 0
17:34:54 | 2268 | INFO: send MessageWindowFocus 
17:34:56 | 2268 | INFO: send MessagePingReceived 
17:34:57 | 2268 | INFO: send MessageWindowFocus 
17:35:26 | 2268 | INFO: send MessageStatus Loading Project: C:/work/project-name/123.srctrlprj - loading
17:35:26 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Loading Project: C:/work/project-name/123.srctrlprj
17:35:26 | 2268 | INFO: send MessageStatus Project could not load any symbols because the index database is empty. Please re-index the project.
17:35:26 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Project could not load any symbols because the index database is empty. Please re-index the project.
17:35:26 | 2268 | INFO: send MessageRefresh 
17:35:26 | 2268 | INFO: send MessagePingReceived 
17:35:26 | 2268 | INFO: send MessageIndexingShowDialog 
17:35:26 | 2268 | INFO: send MessageRefresh 
17:35:47 | 2268 | INFO: send MessageShowStatus 
17:35:50 | 2268 | INFO: send MessageWindowFocus 
17:35:56 | 9780 | SharedMemory.cpp:197 SharedMemory::SharedMemory() | ERROR: boost exception thrown at shared memory creation - srctrlmem_test_95b6318a-8724: boost::interprocess::intermodule_singleton initialization failed
17:35:56 | 9780 | SharedMemory.cpp:115 SharedMemory::checkSharedMemory() | ERROR: boost exception thrown at shared memory check: boost::interprocess::intermodule_singleton initialization failed
17:35:56 | 2268 | INFO: send MessageStatus Error on accessing shared memory. Indexing not possible. Please restart computer or run as admin: boost::interprocess::intermodule_singleton initialization failed - error
17:35:56 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Error on accessing shared memory. Indexing not possible. Please restart computer or run as admin: boost::interprocess::intermodule_singleton initialization failed
17:35:56 | 2268 | INFO: send MessageWindowFocus 
17:36:13 | 2284 | IDECommunicationController.cpp:167 IDECommunicationController::handlePing() | INFO: vs instance detected via plugin port
17:36:13 | 2268 | INFO: send MessagePingReceived 
17:36:22 | 2268 | INFO: send MessageWindowFocus 
17:36:24 | 2268 | INFO: send MessageWindowFocus 
17:36:28 | 2268 | INFO: send MessageWindowFocus 
17:36:28 | 2268 | INFO: send MessageWindowFocus 
17:36:29 | 2268 | INFO: send MessageWindowFocus 
17:36:31 | 2268 | INFO: send MessageWindowFocus 
17:36:34 | 2268 | INFO: send MessageWindowFocus 
17:36:34 | 9780 | SharedMemory.cpp:197 SharedMemory::SharedMemory() | ERROR: boost exception thrown at shared memory creation - srctrlmem_test_95b6318a-8724: boost::interprocess::intermodule_singleton initialization failed
17:36:34 | 9780 | SharedMemory.cpp:115 SharedMemory::checkSharedMemory() | ERROR: boost exception thrown at shared memory check: boost::interprocess::intermodule_singleton initialization failed
17:36:34 | 2268 | INFO: send MessageStatus Error on accessing shared memory. Indexing not possible. Please restart computer or run as admin: boost::interprocess::intermodule_singleton initialization failed - error
17:36:34 | 2268 | StatusBarController.cpp:87 StatusBarController::setStatus() | INFO: STATUS Error on accessing shared memory. Indexing not possible. Please restart computer or run as admin: boost::interprocess::intermodule_singleton initialization failed
17:36:34 | 2268 | INFO: send MessageWindowFocus 
17:36:36 | 2268 | INFO: send MessageWindowFocus 
17:36:36 | 2284 | IDECommunicationController.cpp:167 IDECommunicationController::handlePing() | INFO: vs instance detected via plugin port
17:36:36 | 2284 | IDECommunicationController.cpp:167 IDECommunicationController::handlePing() | INFO: vs instance detected via plugin port
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2284 | IDECommunicationController.cpp:167 IDECommunicationController::handlePing() | INFO: vs instance detected via plugin port
17:36:36 | 2284 | IDECommunicationController.cpp:167 IDECommunicationController::handlePing() | INFO: vs instance detected via plugin port
17:36:36 | 2284 | IDECommunicationController.cpp:167 IDECommunicationController::handlePing() | INFO: vs instance detected via plugin port
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:36 | 2268 | INFO: send MessagePingReceived 
17:36:38 | 2268 | INFO: send MessageWindowFocus 
17:36:38 | 2284 | IDECommunicationController.cpp:167 IDECommunicationController::handlePing() | INFO: vs instance detected via plugin port
17:36:38 | 2268 | INFO: send MessagePingReceived 
17:36:38 | 2268 | INFO: send MessagePingReceived 
17:37:49 | 2268 | INFO: send MessageWindowFocus

I only switched out one path segment above for project-name. However, the original path looks similar (also a dash and ony latin characters).

What other information do you need?

We have at least one anti-malware solution installed, but the component that injects a DLL into every created Win32 process has been disabled.

However, "shared memory" smells a bit like it may be some kind of HIPS getting in the way. But I have no real idea (as of yet) on how to debug this.

mlangkabel commented 4 years ago

I just tested this with the same Sourcetrail version on a machine that has the same Windows version installed and was unable to reproduce this issue. However, that machine does not have and additional anti-malware software installed.

Even though this issue prevents you from indexing a Sourcetrail project, it already happens at the very first line of the log file you provided.

What do you mean by "(the installed version, not portable one)"? Do you mean that this issue is not happening with the portable version? Or that you didn't test the portable version?

assarbad commented 4 years ago

What do you mean by "(the installed version, not portable one)"? Do you mean that this issue is not happening with the portable version? Or that you didn't test the portable version?

I did not test it. But I will promptly try that. Although this doesn't really seem like the sort of issue that would be caused by system-wide installation vs. portable.

I'll get back to you shortly.

Btw: anything I can do to diagnose this further?

assarbad commented 4 years ago

The error remains the same even with the portable version.

assarbad commented 4 years ago

Okay, found the apparent cause (but no workaround).

C:\ProgramData\boost_interprocess (which seems to be a rather unfortunate - because generic - name) in case other programs on my system also use that boost module.

I mean, I get it. This is IPC, but is it IPC that should work across software packages?

What I also found kind of odd was this:

C:\ProgramData>icacls boost_interprocess
boost_interprocess  No permissions are set. All users have full control.
Successfully processed 1 files; Failed processing 0 files
C:\ProgramData>icacls .
. NT AUTHORITY\SYSTEM:(OI)(CI)(F)
  BUILTIN\Administratoren:(OI)(CI)(F)
  CREATOR OWNER:(OI)(CI)(IO)(F)
  BUILTIN\Benutzer:(OI)(CI)(RX)
  BUILTIN\Benutzer:(CI)(WD,AD,WEA,WA)

Successfully processed 1 files; Failed processing 0 files

... could be intentional. But right now I don't have the time to investigate.

Note the discrepancy between the container (C:\ProgramData) and the subdirectory (C:\ProgramData\boost_interprocess).

assarbad commented 4 years ago

Relevant documentation is here. So presumably it's meant to be created in that place.

I have no idea why it fails with NAME COLLISION, though.

According to this answer you can, however, define BOOST_INTERPROCESS_SHARED_DIR_PATH to be something other than the default.

mlangkabel commented 4 years ago

I think that this is the default place where boost puts it's shared memory files. At least I don't see any place in our code base where this directory is set.

For example, on my system I see a file with the path C:\ProgramData\boost_interprocess\1588921174\srctrlmem_grbg_cllctr_64

And icacls tells me that in the C:\ProgramData\boost_interprocess\1588921174 directory all users have full control.

mlangkabel commented 4 years ago

If using the default implementation, (BOOST_INTERPROCESS_BOOTSTAMP_IS_LASTBOOTUPTIME undefined) and the Startup Event is not found, this might be due to some buggy software that floods or erases the event log.

Maybe this is the issue. When did you last restart your machine?

assarbad commented 4 years ago

If using the default implementation, (BOOST_INTERPROCESS_BOOTSTAMP_IS_LASTBOOTUPTIME undefined) and the Startup Event is not found, this might be due to some buggy software that floods or erases the event log.

Maybe this is the issue. When did you last restart your machine?

I did it - among other things - in an attempt to get this to work. I think I need to see if I can figure out the meaning of that particular thing. However, the question here is which part is "buggy". Because I'm not sure why any software would rely on the boot time from - it seems - the event log when there are perfectly valid other ways on a Windows system to achieve the same.

Yes, the event log is actually being sanitized as part of my startup routine. So this may well be self-inflicted. I'll see what I can find in regard to that BOOST_INTERPROCESS_BOOTSTAMP_IS_LASTBOOTUPTIME thingamy.

assarbad commented 4 years ago

The good news first, your pointers provided for the ability to establish a workaround. Rebooting without clearing the event log did work around this issue. But it's sort of awkward for a software to force me not to clear the event log on my own accord (which of course cannot be blamed on SourceTrail, I understand that!).

Longer answer follows.

Yep, you appear to be right:

#if 0 == BOOST_INTERPROCESS_BOOTSTAMP_VALUE_SUM
#  define BOOST_INTERPROCESS_BOOTSTAMP_IS_EVENTLOG_BASED
#endif

... in boost/interprocess/detail/win32_api.hpp is the issue.

Whoever came up with this may have missed the fact that on Windows there is no option to do anything like with systemd (e.g. journalctl --vacuum-time=10d), that is to trim the journal (event log in Windows) to a certain number of days. It's clear the log or let it grow (or recycle eventually). So the choice of having the event log based method as the default makes quite a number of assumptions on how a user/admin has to use/configure their system.

Looking at that Boost code is quite shocking to me, because Boost is generally held in high regard everywhere and the praise they receive from experts confirms that. But this decision is certainly a rubbish decision and indicates little regard for how Windows works. It means you rely on the user never clearing the event log (or as you put it, not to be too long since last reboot, which could have caused the event log to overflow and records to get recycled). And we're not in the era of pre-SP2 XP either, when reboots were common and frequent. So even that is a plausible scenario, not to mention that the event log sizes are configurable.

Anyway, long story short. I think using another way to come up with a sensible time stamp would make more sense.

What strikes me as odd in the Boost code is the fact that they're making use of the native API (Nt*-prefixed functions) in that code, but don't use a more sensible method of fetching the boot timestamp for a default.

Not having a fallback, on the other hand, makes sense. As the purpose appears to be to literally cause clashes when the same timestamp comes up.

Still, it would be much more sensible to query the creation time of \REGISTRY or \SystemRoot in the Object Manager namespace. My tool NT Objects demonstrates this. And yes, these can be queried as totally unprivileged user. I just verified it after a reboot and I have the following time stamps (times in UTC):

(as a side-note, these values are actually roughly nine seconds before the timestamp of the event 6005 "The Event log service was started.")

I didn't check what PSTools (or psinfo in particular) use, but using event ID 6005 really doesn't strike me as a very prudent course of action.

And the error being thrown is also rather generic for an error that could reasonably occur weeks or months into a given boot session. And what's more, running a program with such a widely used library may succeed now, but after the event log cycles or I clear it, the program just starts to fail with a rather unspecific error message.

I'll see how I can get in touch with the Boost project regarding this.