Open SimonGoring opened 3 years ago
Hi @SimonGoring, do you have any details on the last log statements? I think adding additional logging would certainly be useful for debugging and determining where it's actually hanging.
Notably condenser doesn't do anything fancy to my knowledge with say threads or other forms of deadlocks, so my guess is it might be a query timeout or a connection timeout to the database. It probably wouldn't be too hard to actually hook into where queries are issued to print the last issued query in a debug mode; would be happy to take a look at a PR for that.
I've created a public gist with my config file and a link to a dump of the database I'm applying condenser against. The issue I'm running into is that condenser appears to hang (over 24hrs with no new text to screen on verbose mode), but I'm not sure how to debug the issue, or know whether or not anything is actually happening.
I'm running condenser as part of a broader workflow through a bash script:
That's more an FYI about how we're trying to use it though. The key element is that we're just calling condenser with
python3 direct_subset.py -v
and the config file is linked above in the gist.The goal of this issue is to note that there seems to be a point at which condenser is hanging, and to figure out a way to debug it so I can fix it.