Large # of rotatable bonds, and download page failure.

[danwentworthart]

Tool: Carbohydrate Builder

Bug Description: From Rob: "Hi Guys, when I try to build this (or anything similar) it tells me that there are too many rotatable bods. But when I select an allowable number of rotamers, the download page doesn't update and I am unable to build it. This happens whether I select it from a library or build from text."

DNeup5Aca2-6DGalpb1-4DGlcpNAcb1-2[DNeup5Aca2-6DGalpb1-4DGlcpNAcb1-4]DManpa1-3[DNeup5Aca2-6DGalpb1-4DGlcpNAcb1-2[DNeup5Aca2-6DGalpb1-4DGlcpNAcb1-6]DManpa1-6]DManpb1-4DGlcpNAcb1-4DGlcpNAcb1-OH

[danwentworthart]

Tried replicating on test.glycam.org and failed to replicate. Seems to work. Will keep digging. Maybe he found it on glycam.org.

[danwentworthart]

Did replicate on actual. It is possible this will resolve when we update glycam.org to match test.glycam.org.

[Lachele]

I could not replicate on Actual. So, it's intermittent? I am building 64 conformers now. Not all are done.

If we have all just tried to build 64 conformers on different sites, things might be slow for a minute.

On Tue, Mar 14, 2023 at 9:25 AM Dan Wenworth @.***> wrote:

Did replicate on actual. It is possible this will resolve when we update glycam.org to match test.glycam.org.

— Reply to this email directly, view it on GitHub https://github.com/GLYCAM-Web/website/issues/128#issuecomment-1468099562, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCV7D2ZPGMJCXJAO5LQC4TW4BWUFANCNFSM6AAAAAAVZSCEZU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- :-) Lachele @.*** lachelefoley.com I'm happy people are learning that sleep is important. Now, please stop trying to kill the nocturnal folks.

[danwentworthart]

Suspect this relates to the size of the queue prior to this request?

[Lachele]

No. That's not the issue. I found the problem. I think the bug is in GEMS. There is trouble on LiveTest, too. The queue is empty.

On Tue, Mar 14, 2023 at 10:09 AM Dan Wenworth @.***> wrote:

Suspect this relates to the size of the queue prior to this request?

— Reply to this email directly, view it on GitHub https://github.com/GLYCAM-Web/website/issues/128#issuecomment-1468174316, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCV7D6UEUG62DZHHX7NQCTW4B33DANCNFSM6AAAAAAVZSCEZU . You are receiving this because you commented.Message ID: @.***>

-- :-) Lachele @.*** lachelefoley.com I'm happy people are learning that sleep is important. Now, please stop trying to kill the nocturnal folks.

[Lachele]

It wasn't in GEMS. It was in GMML. I believe this is fixed. Re-open if I am wrong.

[gitoliver]

Note there are probably separate bugs discussed here that should have their own issue, some are known already, but it needs more investigation to see what the issues are: I just built the submitted sequence on actual (zany-zorn). On the options page I'm able to deselect down to an allowable number of rotamers: Note that the website calculates current rotamers separately from gmml/gems which calculates total possible/likely. There is a known bug in how gmml calculates total possible/likely, but that wasn't/isn't the issue here. So all is ok with the above, except that LiteMol isn't displaying the default, and doesn't seem to check for it more than once. I can understand why the default might not be ready in time for initial page load as it's a big structure. When I get to the download page: The default is displayed correctly in LiteMol, but while polling seems to be active and update for the first few minutes, after 10 or so it seems to stop updating. I'm now "stuck" with a bunch of "All Complete" and minimizing. Separate note, it's doing the status flip thing still, I'm just reporting the behavior again in case we think it's fixed:

https://user-images.githubusercontent.com/17079588/228458380-1e455d3c-f249-45dd-8fc1-c0ffff110651.mp4

Also offtopic while all these are being queued, submitted minimizing etc I'm able to build other structures on a separate tab, so great load balancing is going on relative to before.

Here is the page after 10-15 mins, everything seems to be hanging (one eye is green as I clicked on it): Reloading the page of course gets me the download page from the other thing I built in a separate tab in the meantime (we need the download page to be project specific, for this and it would be nice to be able to send links to people).

When I rebuild the structure and request the exact same 16 rotamers, they don't immediately all say "All Compete". I get a mix of All Complete and minimizing. It seems like more completed than before, but something funky is going on. Here is the zoomed out final download page after the second time, again it seems to be stuck: and after a few mins, I tried a page reload while it initally seemed to be hanging like this: it eventually started working again, and the polling started working and everthing flipped to complete! So it seems like it might be a combination of some jobs taking ages, perhaps all running on the same CPU or RAM has become limited, or it's just that our polling is the issue.

[Lachele]

I'll take a look again. I might need to finish what I was doing in GEMS in the sequence Entity.

This could be anything. The "load balancing" might simply be that the structures are never being submitted. That's what was happening before. It could be a systems issue, etc. But, it is probably related to the following.

Note that if more than 64 structs are available, then the default 64 always get built and minimized. This is what my changes in GEMS will fix. I made a summary here: https://github.com/GLYCAM-Web/gems/issues/74#issue-1645006542

[Lachele]

In the future, copy in the sequence :-)

[danwentworthart]

Thank you Oliver, for taking such detailed notes. At a glance, I can see multiple issues. After lunch, Charles and I will break this up into separate tickets and do proper assessments. Educated guesses about downloads page:

1. Session testing is not a thing we do yet, but clearly is needed. We are actively building a strategy to assess, test, and improve session management logic.
2. LiteMol improvements will be on the radar soon. This ties into the new capability for us to modularize JavaScript, so will come up during a refactor of cb_options.js and cb_downloads.js
3. Polling can always improve. I suspect that responses are coming back in a stagger pattern which I may be able to remove in the refactoring of cb_downloads.js. We do need to question timing here: it is scheduled to become a gems service. Any non-js website work here is going to be replaced. Also: Possible that current work in gems could address some of this.

[Lachele]

I need to finish my work in GEMS. It appeared that the GMML fix was all that was really needed.

The jobs that aren't building are never even getting their simulation and slurm submission files made. They are never being submitted for minimization.

Let's talk during or after the meeting this morning.

BTW - yes, Oliver's details are great. But, a sequence would help. :-)

[Lachele]

And... yeah... the sequence was up there at the top. My bad. I did read all the new stuff, though. :-)

Anyhow, this appears (again) to be a thing in GEMS. It seems to be a problem with the accounting when GEMS is deciding which structures exist and which need building. For example (in my DevEnv where I can control things more), I requested 4 structures. The website shows me four structures, but in the project directory, I have two new builds and three existing builds - so, five structures. The work I was doing before in GEMS will be needed. I'll get back to that immediately.

[gitoliver]

Some of the structures I selected are left in a minimizing state: https://test.glycam.org/cb/download/9edcb3b8-fe1f-4e7e-ac6c-cc0cf300981b/ If someone can show me how to check the guts of actual I will look at this.