Organograms - too slow

[davidread]

Page load time for the publication page was unacceptably slow. I reproduced this, clicking on:

https://test.data.gov.uk/organogram/manage/department-for-environment-food-and-rural-affairs then selected another publisher and then back to Defra.

The nginx logs show the load times for this page are really bad - about 1 in 2 or 3 times it is non-negligible: 15s, 14s, 26s, 13s, 12s, 25s

egrep \/organogram\/manage\/department-for-environment-food-and-rural-affairs /var/log/nginx/access.log.1

the last number shows the time that apache took to supply the response, in seconds.

It doesn't look specific to this publisher - looking through the logs I'm seeing examples of 14s for home-office, 8s for asset-protection-agency etc.

[davidread]

I guess this is due to you running ETL on-the-fly when you show this page. Why is this? Why is the result not stored more permanently?

[ratajczak]

This slowness is caused by API call to /api/3/action/organization_show?id=department-for-environment-food-and-rural-affairs I commented out one line with this call and this page loads quickly now. I don't think that we need it on this page, it's being used to render 'View' link, we know organogram dataset url convention so I can remove it. But this call is needed on publish and delete operations, they will stay affected by it slowness.

[rossjones]

If you don't need the packages in that call you can speed it up with &include_datasets=false

[ratajczak]

thanks @rossjones, thats useful

[davidread]

That's great! I tried a number of publishers on test.data.gov.uk and agree this is much quicker. Do you want to deploy to live and we can close this?

[ratajczak]

fixed and deployed