Elusive log statements

[aghreed]

Hello!

First, thank you for the awesome project — been using this at my organization to understand better of our design system adoption and usage.

I have been running into an issue however, which is that often I'm not seeing log statements making it into the terminal. I've yet to determine the specific conditions, but I've narrowed it down to when the statsWorker invokes collectStats.

I typically get one console log output from inside the collectStats function per worker — but after that it goes dark. And this leaves me in a situation of not knowing if an unexpected issue (unhandled error) has happened or if the script is just running. At first, I would let this script run, but after hours and hours of no more output I give up and kill the process.

I guess my question succinctly is: have you encountered this issue at all? If so, is there a way around it?

I have forked the repository and have been applying tweaks. But haven't found success yet. At this point, I'm thinking it likely has to do with some of the dependencies, namely worker_threads. But perhaps I'm just doing something silly.

Any advice would be greatly appreciated! Thanks

[smoogly]

Hey Alex, tracker doesn't do anything fancy: logging is simple console.log statements. If the tracker is working, but the logs are missing, I'd probably look at execution environment first.

How are you running tracker? What is the Node version? Can you see the process consuming CPU in the monitoring tools?

Re working for hours, this is quite normal for a first run (with no cache) when long history is requested over a big number of projects. E.g., with since: new Date(<a few years ago>). What about running against a single repo with a shallow history — since set a week ago or so? That should finish within minutes.

[aghreed]

Hey Arseny — thanks for the quick reply!

I'm running tracker locally on a M2 MacBook Pro. Node v18.18.0. Yarn v1.22.17. And I do see the process in Activity Monitor (gobbling up lots of memory too).

My issue sounds very similar to the one mentioned in this issue comment.

While the initial config file I had could be called ambitious, I have been minimizing the target area while I investigate why the script seems to hang. It's now to only two repos, and back to February of this year. That said, I've experienced this same mysterious missing logs no matter how narrow I seem to make the target.

[aghreed]

What I am struggling to still wrap my head around is why it will work sometimes for a while.

For instance yesterday I was able to get ~80% of the timeline analyzed that I wanted. But it seems to be hanging on the last few commits.

And now subsequent runs (with that cache in place) appear to just be hanging (at least according to the console output) at that entry point of the collectStats function.

[aghreed]

Okay wanted to add a bit more color to this while I was at it.

Here's a screen shot of the forked source of collectStats I'm running locally:

And then the output in my console:

[Thread 8 - Project - 0s] Using cached stats for commit 874042f3f6a41682d8d9f09d7a6fd723b9f54bfe
[Thread 9 - Project - 0s] Using cached stats for commit 2be44d325a279eeedc28e9d462721b1f590a6d24
[Thread 10 - Project - 0s] Using cached stats for commit a17670375706d8a41064e6435e786887741fb91d
[Thread 11 - Project - 0s] Using cached stats for commit c341f44496ba6955ee0f585bdb8d08938c2f9216
[Thread 7 - Project - 0s] Using cached stats for commit 0997102b14e1af872a3e5f14622a3394b1875605
[Thread 6 - Project-Next - 12s] Cloning from {repoURL}
[Thread 4 - Project-Next - 12s] Cloning from {repoURL}
[Thread 2 - Project-Next - 12s] Cloning from {repoURL}
[Thread 3 - Project-Next - 12s] Cloning from {repoURL}
[Thread 5 - Project-Next - 12s] Cloning from {repoURL}
[Thread 1 - Project-Next - 12s] Cloning from {repoURL}
[Thread 1 - Project-Next - 2s] Checking out commit 67ce3472f3be40c1708864fe31ebe11df9319cab
[Thread 4 - Project-Next - 2s] Checking out commit 4d6ed4ea1469802125bba921e60c86caf92f9eab
[Thread 3 - Project-Next - 2s] Checking out commit 675539a5a224abc9862073c4198901d5dd4af084
[Thread 6 - Project-Next - 2s] Checking out commit 491fbccad4b0e1092a485666d9e5bb95a5480c78
[Thread 5 - Project-Next - 2s] Checking out commit 6fcf65e0e4a044d2c6a79bb6e05a7a478ec4d11f
[Thread 2 - Project-Next - 2s] Checking out commit 0fcae441e4b81f04c506fcd27d8ca2ca1bbd5e5e
[Thread 4 - Project-Next - 4s] Starting stats collection: Tue Jun 18 2024 17:12:43 GMT-0400 (Eastern Daylight Time)
[Thread 3 - Project-Next - 4s] Starting stats collection: Tue Jun 18 2024 17:12:43 GMT-0400 (Eastern Daylight Time)
[Thread 5 - Project-Next - 4s] Starting stats collection: Tue Jun 18 2024 17:12:43 GMT-0400 (Eastern Daylight Time)
[Thread 6 - Project-Next - 4s] Starting stats collection: Tue Jun 18 2024 17:12:43 GMT-0400 (Eastern Daylight Time)
[Thread 1 - Project-Next - 4s] Starting stats collection: Tue Jun 18 2024 17:12:43 GMT-0400 (Eastern Daylight Time)
[Thread 2 - Project-Next - 4s] Starting stats collection: Tue Jun 18 2024 17:12:43 GMT-0400 (Eastern Daylight Time)

Despite there being a log statement directly on the next line (Something something something) it never seems to show up in my terminal 🤔