Closed arjunrajlab closed 3 months ago
Command for profiling with pprofile
:
python -m pprofile -o /tmp/profile.dat girder serve --host 0.0.0.0
Profile mongo, go to the mongo db docker image and:
exec mongosh
use girder
db.setProfilingLevel(2)
# Do what takes time here #
db.setProfilingLevel(0)
db.system.profile.find().limit(2).sort( { millis : -1 } ).pretty()
db.system.profile.drop()
Don't forget to db.setProfilingLevel(0)
to avoid filling the space with logs
When using this command:
workerClient.add_multiple_annotation_property_values(dataset_property_value_dict)
It seems to be very slow. The calculation of the properties is now super fast, which makes sense because we do them all at once, but the upload to the server is quite slow. It could be that this reformatting is slow:
But I suspect it is this call that is slow:
I talked with @manthey about it, and he suggested that it may be a problem with the database setting user access for each property entry. Or perhaps something else in that loop. But again, not sure.