search

dmusican / enchilada

0 stars 0 forks source link

Cluster Query Issues #20

Closed dgrosscc closed 2 months ago

dgrosscc commented 2 months ago

I bet these things are related to issues you fixed in other places. We are trying all the options, because they should be valuable for the task at hand, so we're finding lots of bugs!

When running a cluster query using the attached file as the cluster center text file (it asks for a .txt in the window, but requires a .csv file!), ClusterCenterToQuery.csv

The following issues come up (according to my student):

  1. The particles are not being placed into the collection and there's an error log being produced that looks like it may have to do with time stamps?
  2. The program will allow deletion of the empty collections and it's also popping up in the error log.

See screenshots below: CollectionsNotDeletableErrorLog CollectionsNotDeletableErrorLog_2 CollectionsNotDeletable_output CollectionsNotDeletable_output2

dgrosscc commented 2 months ago

Follow up-- when the student checked back about 10 minutes later, the particles were listed in the correct folder. Something is strange, but it's not the same strange we thought. She could also delete the collections that were empty at that point.... Weird?

dmusican commented 2 months ago

I'll take a look when I can! Looks like similar problems to the others.

dmusican commented 2 months ago

I've been working on this. It appears to be yet more problems with other unrelated portions of the code holding on to connections to the database that they're supposed to give up, and coding in many places that was designed for speed rather than maintainability. It's very hard to know in some places if database connections are successfully being closed after they are opened, .... and clearly some are not, which results in the locking errors. I'm rewriting every instance of this that I can find so that they use good modular practices and clearly close their connections, but in a number of especially thorny cases, removing these tendrils without breaking what they're connected to is something of a surgical procedure.

It's in progress.

dgrosscc commented 2 months ago

Thanks for the update! The things you have fixed so far are hugely helpful, and we will be very patient about this set. I really appreciate that you are giving this any attention at all! Do know that it's making a difference. Deborah

On Mon, Jul 22, 2024 at 6:59 AM Dave Musicant @.***> wrote:

I've been working on this. It appears to be yet more problems with other unrelated portions of the code holding on to connections to the database that they're supposed to give up, and coding in many places that was designed for speed rather than maintainability. It's very hard to know in some places if database connections are successfully being closed after they are opened, .... and clearly some are not, which results in the locking errors. I'm rewriting every instance of this that I can find so that they use good modular practices and clearly close their connections, but in a number of especially thorny cases, removing these tendrils without breaking what they're connected to is something of a surgical procedure.

It's in progress.

— Reply to this email directly, view it on GitHub https://github.com/dmusican/enchilada/issues/20#issuecomment-2242781641, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGEJQIMHNCVKGYDWVGEESLZNTXZRAVCNFSM6AAAAABLC6DRHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBSG44DCNRUGE . You are receiving this because you authored the thread.Message ID: @.***>

dmusican commented 2 months ago

Still working on it. The code that moves particles en masse from one collection to another is a Gordian knot. It's a textbook example of how not to do things. We also have different versions of the same code multiple times over written by different people who didn't realize, I think, that the other versions were there, and they're used sort of interchangeably in different places but with hacks around them. Various pieces of code jump in and out of them at will. I'm getting my head wrapped around it, but my goal at this point is definitely not to make the quick fix but to drill out the disease.

dgrosscc commented 2 months ago

Thanks for taking this on! And for the update. Deborah

On Tue, Jul 23, 2024 at 1:16 PM Dave Musicant @.***> wrote:

Still working on it. The code that moves particles en masse from one collection to another is a Gordian knot. It's a textbook example of how not to do things. We also have different versions of the same code multiple times over written by different people who didn't realize, I think, that the other versions were there, and they're used sort of interchangeably in different places but with hacks around them. Various pieces of code jump in and out of them at will. I'm getting my head wrapped around it, but my goal at this point is definitely not to make the quick fix but to drill out the disease.

— Reply to this email directly, view it on GitHub https://github.com/dmusican/enchilada/issues/20#issuecomment-2245938159, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGEJQMYHFFVMUTO7X32BBLZN2MXZAVCNFSM6AAAAABLC6DRHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVHEZTQMJVHE . You are receiving this because you authored the thread.Message ID: @.***>

dmusican commented 2 months ago

Fixed, I think. Very messy code, and I worry about what else I broke. The tests pass and it seems to be working, as far as I can tell.

There are other portions of the code that still exhibit these same database access problems. I'm just going to put off working on them until they're a problem, because each fix is getting more and more time consuming. If something is wrong, you'll let me know. :)

This will go into the next release; I'm going to try to see if one of the other issues is a quick fix before doing so.

dgrosscc commented 2 months ago

Awesome, thanks for the update! Deborah

On Thu, Aug 1, 2024 at 10:58 AM Dave Musicant @.***> wrote:

Fixed, I think. Very messy code, and I worry about what else I broke. The tests pass and it seems to be working, as far as I can tell.

There are other portions of the code that still exhibit these same database access problems. I'm just going to put off working on them until they're a problem, because each fix is getting more and more time consuming. If something is wrong, you'll let me know. :)

This will go into the next release; I'm going to try to see if one of the other issues is a quick fix before doing so.

— Reply to this email directly, view it on GitHub https://github.com/dmusican/enchilada/issues/20#issuecomment-2263418708, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGEJQKFJCIOKHYCAD43GLLZPJLKDAVCNFSM6AAAAABLC6DRHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRTGQYTQNZQHA . You are receiving this because you authored the thread.Message ID: @.***>