search

m2ms / fragalysis-frontend

The React, Redux frontend built by webpack
Other
1 stars 1 forks source link

"Preparing download" step intolerably slow #748

Open phraenquex opened 2 years ago

phraenquex commented 2 years ago

Once the progress bar starts flashing regularly (about 30s later?), the lag goes away. So maybe something with the front-end/back-end handshake? But the whole process is not "seconds", as @duncanpeacock claimed. So this is definitely a Big Problem.

@boriskovar-m2ms says it's the backend.

boriskovar-m2ms commented 2 years ago

The backend is responsible for the length of the operation not quite sure what can be done about that except for some extensive caching mechanism. I will look into what can be done to make front end less laggy in the process

duncanpeacock commented 2 years ago

It depends on what you want to download. Firstly, note that the timings I did on my test stack for preparing the Mpro zip file (and did report in the meeting - although clearly the message did not get through) ranged from 5 seconds to 40 seconds - although the major changes I had to do to get the final requirements changes in may have slowed this down slightly.

The longer timings started to occur when you add the dif files in to the mix. For Mpro, these tripled the size of the zip file from about 500MB to about 1.5GB.

As I understand, the original idea was to allow users to speed up the process by only downloading parts of the target. There comes a point (I think probably when adding the "dif" files) when it's actually faster for the user to download the complete zip file as it is, rather than recreating it.

I suggest we do some timings on staging to see how it goes on the staging stack - but a couple of options:

phraenquex commented 2 years ago

Might have been an artefact (user illusion) from #753?

Start with profiling the backend process in production, using Mpro and after #753 is in production.

duncanpeacock commented 2 years ago

Mpro: Timings for Prepare Download

753 not in prod yet, so ignoring SDF files for now.

Contents: Pdb 17.07 seconds 1.2GB Everything except SDF: 31.63 seconds 3.9GB.

TBC.

ag-m2ms commented 2 years ago

Hello, the issue comes from taking a screenshot of the website which is taken right after clicking Prepare download. Are the screenshots necessary? The library which takes screenshots has to iterate over all of the DOM elements and operate on each one, which is in general expensive. The more DOM elements there are, the longer it takes. A good example is loading all of the molecules for a target. Taking screenshots can't be offloaded to a worker thread because web workers don't have access to DOM, so it has to happen on the main thread, which also have to deal with user interaction. The other option I see would be to virtualize lists, which would require a bigger refactoring, though it wouldn't solve the problem at its core. Even with a few molecules loaded in the list the UI would hang for a couple of seconds. The only other option would be to use MediaDevices.getDisplayMedia which requires user's permissions. As far as I know it can only take a screenshot of the whole display or of the whole window. The webpage would have to be cropped out of it, which I am not sure would be easy, if at all possible. The nice thing about it, though, is that it is instant.

phraenquex commented 2 years ago

@ag-m2ms : Are you clear what delay this ticket refers to? It's NOT the little hiccup (a few seconds) shortly after hitting the "Prepare" button, when the wait bar stops; it's the much longer one (20-60 seconds), when wait bar does move, but the frontend is clearly waiting for something in the backend. (At least, it seems clear to me as user.)

@duncanpeacock said yesterday he suspects that there is something not quite right with the frontend/backend handshake.

@boriskovar-m2ms @duncanpeacock did you get around to profiling this yesterday?

boriskovar-m2ms commented 2 years ago

Sorry I thought that you're talking about that hiccup and the beginning. The waiting is caused that we make a POST call and we just wait for this POST call to return. There is no difference between this API call and dozens of other API calls in the frontend. We just wait until the backend prepares the zip file and gets back to us with the URL. There is nothing that frontend can do to speed this up.

duncanpeacock commented 2 years ago

Boris and I spoke yesterday about the backend/frontend handshake. I've made some changes to plug the hole I'd discovered in the code and improve the error communication from the API (this will also yelp other users of the API). I'm just testing this on my stack at the moment. Once I've done a sanity check, Boris can also download the backend repo and do some testing.

On Wed, 12 Jan 2022 at 14:37, boriskovar-m2ms @.***> wrote:

Sorry I thought that you're talking about that hiccup and the beginning. The waiting is caused that we make a POST call and we just wait for this POST call to return. There is no difference between this API call and dozens of other API calls in the frontend. We just wait until the backend prepares the zip file and gets back to us with the URL. There is nothing that frontend can do to speed this up.

— Reply to this email directly, view it on GitHub https://github.com/m2ms/fragalysis-frontend/issues/748#issuecomment-1011054561, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIV35BVDVLHCF564JA5FKU3UVV73JANCNFSM5KR5WVTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

phraenquex commented 2 years ago

See #578 - that might make this WONTFIX.

Also, the error catching ticket might have been the actual problem.