Performance testing of site

[ellemindwave]

As per your email @iteles .

Need to know:

• What is involved in this?
• How long should be allocated to this

[reddog]

@ellemindwave At what point do we need more information from @iteles on this?

[iteles]

I don't think we need this to come from me, @Danwhy and @Cleop can discuss this with you both tomorrow 👍

[Cleop]

To be estimated/outlined tomorrow

• Speed / concurrent users
• IE testing from 8+ #518 #545
• penetration testing

[reddog]

@cleop Thanks for your notes.

@ellemindwave Do we have a good understanding of why this issue is needed and why it should be done at this point?

@cleop If load testing is required, is there a clear understanding of service level the site should support? Additionally the site will be moving hosting, see #284. Does that mean that load testing at this stage would still be useful?

[Cleop]

Summary from Friday meeting:

• Penetration testing is not needed at this stage as users cannot insert text onto the screen (this will change when the search bar is implemented #508 if a user's search term stays on the page, however #508 is now being removed from the sprint)
• IE testing is needed due to GDS requirements.
• Speed / concurrent user testing is needed due to the errors that have been occurring with too many users during some demos and due to the potential influx of users at live. Whilst moving to AWS may improve this somewhat, it's important demos in the meantime go to plan and the team are aware that potential PR spikes in the future could mean the load is significantly increased and we want to be ready for those. @Danwhy has already recognised that there is room for streamlining the databases. Agreed for @Danwhy to spend 1 day on this at this time and more time can be invested at a later date if needed.

[reddog]

@Cleop Thanks for your notes,

@ellemindwave & @sarahwelton Are you able to provide further information on issues encountered during demos as mentioned by @Cleop? These should be captured as issue(s).

@Cleop & @Danwhy We have tools (Heroku dashboards) that will help give some indication if any particular action (database work for example) would have any meaningful impact. Have these reports been reviewed? What do they show?

Maybe the quickest / cheapest / easiest overall speed boost may be to simply up the Heroku dyno type. @ellemindwave Should we try that first?

@Danwhy can you expand on the suggested database streamlining? Are there any/many deviations to the regular Wagtail/Django database layout currently in place? Moving away from a standard database structure could add to site maintainability at later dates?

As mentioned elsewhere, there is no automatic performance increases expected with the move to AWS as covered by https://github.com/mindwaveventures/cms/issues/284

[reddog]

@Danwhy & @Cleop Is this issue now in progress? Are you able to provide some response to my queries at https://github.com/mindwaveventures/cms/issues/516#issuecomment-334706068 ?

[reddog]

@ellemindwave & @sarahwelton Are you able to further clarify previous comments about issues seen during demos? These should be raised as issues to ensure they're not left behind.

[Danwhy]

@reddog The database structure we're using is the standard wagtail setup, but the way we're accessing the data is slightly more complicated, due to the way we filter/aggregate the resources on the topic pages based on a large number of tags.

I'm quite sure that the way we're accessing the data is causing some of the slowness of the site, as it has got progressively slower as more resources have been added, and more fields (tags/locations/likes etc) have been added to the resources.

I've used django toolbar to have a look at the queries we're making, and there are a very high number. I'm working through the code now, and have found a few places where we're looping through the resources to render them, which we can refactor into much fewer queries, which will hopefully provide a significant speed increase.

[reddog]

@Danwhy Thanks for the update.

So, to confirm, it's not the database structure that is being refined but the queries you've created to get the data?

You mention that the site has been getting progressively slower. How many resources do we currently have?

Thanks.

[reddog]

@danwhy Are you able to provide a little further information as per https://github.com/mindwaveventures/cms/issues/516#issuecomment-335140459 ? Thanks.

[Danwhy]

That's right, the database itself is not changing at all, I've purely refactored the code. We currently have 111 resources, each of which we were making multiple database requests for in order to get information from other tables. We also have 319 tags which were not being efficiently loaded from the database. I've done some refactoring that has reduced the number of queries, which will hopefully cause a noticeable performance increase on the live site.

Before refactoring:

After refactoring:

We're still making more queries than we ideally would, but due to the diminishing returns in terms of performance I'm not going to spend any more time on this at the moment.

[reddog]

@danwhy Thanks very much for that extra information - really useful and helps give visibility to what the issue is, how it's come about, how it's being measured, how it's being tackled (for now), and what the impact is (of not fixing and separately after the fix).

If at all possible, could we have the first part of that conversation (the what the issue is, how it's come about, and current metrics) ahead of the work to fix it?

[reddog]

For future work on performance, homepage loading stats are:

• Pingdom, Stockholm, Sweden - 2.74s
• GTmetrix, Vancouver, Canada - 3.9s

Roughly the first 25% of that time is spent obtaining the homepage HTML. The rest of the time is spent retrieving other content from various places (48 requests in all).

[reddog]

Waterfall for request of https://www.good-thinking.uk/