Performance loss when using webId from a different server - Githubissues

nodeSolidServer / node-solid-server

Solid server on top of the file-system in NodeJS

https://solidproject.org/for-developers/pod-server

Other

1.78k stars 298 forks source link

Performance loss when using webId from a different server #1443

Closed bourgeoa closed 4 years ago

bourgeoa commented 4 years ago

see issue #1439 @megoth writed The reduced performance might be related to linkeddata/rdflib.js#419

I made some more tests adding webId (acl:agent) to acl rule (so that a rule as more than one webId - acl:agent)

the performance loss arises allready in https://solid.community running on NSS 5.2.3 (I don't know the rdflib version)
there is no performance loss if all acl:agent are on the same server

I practice it is not usable. It takes more than one minute, and some time timeout on reading acl doc. It does not seem to be be related to mashlib (to be confirmed) because I also have the problem using solid-ide.

michielbdejong commented 4 years ago

I'm also seeing a huge delay in test/integration/header-test.js, I'm having to set the mocha timeout to 10s there now in the update-deps branch (added a FIXME comment that links to this issue).

michielbdejong commented 4 years ago

Confirmed that when we removed all external webid's from https://solidos.solid.community/.acl, https://solidos.solid.community/ became like a hundred times faster.

timbl commented 4 years ago

The external webids are [i think] only damaging when they have Control access. See operational issue https://gitlab.com/solid.community/support/-/issues/15

michielbdejong commented 4 years ago

I reproduce this locally in my mashlib-dev now:

$ time curl 'https://localhost:8443/test1443/index.ttl'   -H 'authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiI1YjQ2NWRlZWU3NzE5MjI4N2VlYmYyZmY4NTIxM2VkOCIsImF1ZCI6Imh0dHBzOi8vbG9jYWxob3N0Ojg0NDMiLCJleHAiOjE1OTc4Mjg2NjksImlhdCI6MTU5NzgyNTA2OSwiaWRfdG9rZW4iOiJleUpoYkdjaU9pSlNVekkxTmlJc0ltdHBaQ0k2SWxZMFQwOVFiR042T1d0ckluMC5leUpwYzNNaU9pSm9kSFJ3Y3pvdkwyeHZZMkZzYUc5emREbzRORFF6SWl3aVlYVmtJam9pTldJME5qVmtaV1ZsTnpjeE9USXlPRGRsWldKbU1tWm1PRFV5TVRObFpEZ2lMQ0poZW5BaU9pSTFZalEyTldSbFpXVTNOekU1TWpJNE4yVmxZbVl5Wm1ZNE5USXhNMlZrT0NJc0luTjFZaUk2SW1oMGRIQnpPaTh2Ykc5allXeG9iM04wT2pnME5ETXZjSEp2Wm1sc1pTOWpZWEprSTIxbElpd2laWGh3SWpveE5UazVNRE0wTmpBM0xDSnBZWFFpT2pFMU9UYzRNalV3TURjc0ltcDBhU0k2SWpZeFpXSTFNRE13TkdZNVpUSmpZakVpTENKdWIyNWpaU0k2SWxobWVFTmZPVGwyY25aRWFscG1lSGgxY0hsQ056SlZVWFJqUjNZMVZFTnpOSGx4YTNGRmJVUmFhVUVpTENKaGRGOW9ZWE5vSWpvaWJXTXhSMnRCTUhoa2NtOWxWR1Y2Y0ROaFpsZElVU0lzSW1OdVppSTZleUpxZDJzaU9uc2lZV3huSWpvaVVsTXlOVFlpTENKbElqb2lRVkZCUWlJc0ltVjRkQ0k2ZEhKMVpTd2lhMlY1WDI5d2N5STZXeUoyWlhKcFpua2lYU3dpYTNSNUlqb2lVbE5CSWl3aWJpSTZJblJYWWpGNU1WbHdXRmszWVZKdk1qbFRlbmQ1VkVkS1ZXVTFTV04yYTBSMlZIZFdRWGxwYlhaNkxVVllabGhYZDBoTmFEbHNjRjlHVjA1VFQzQlZkbkppVDFkQ1oybDVlVXhoUjJnMlgzaHpXRkUzYlVWc1ZWUkdVMFJpU1c5WVYwbDZRVzV2VUc5UFdHbHBOMEU0VEZGV1IzQlhlV1pJTjJ0cVNVSlVWRlZmYURWTExUSlFWMEV0Y1d4bVoyMHhSV1V0TTJodVdXOWxia0ZMZGxOMFpFeEpUbmxOVFhsNlFtVnpUVWxQZUUxa2VGaHhWSGhaV2t4UE4zcENka1ZEVlhRMVIwWlROSEExTTBoS05FZFhkM2d3TUdSaFpraGZlVkU1TVVkNlNGTlJibkZMVDJvMFdXRmxiMnhQYkZSVVMxRlZkRGhIU1VsU2NqTjZkM05EVGkxaFJubGhVV1J6T0hSQkxWSmZNbE5hYVc1eE1qWmhXbUZ6VldneGFuWnZkVFJXUjFWbk5WcE1VRWR2U0RodVVWaEdhbE5oVkhjMWExQk1NMDVNWW13MFJVOTNSVEZhWlcxaVkxSkpZMHBTV1Y5UVpHbEZVU0o5ZlgwLm1TYXNhX0RXUXdmLUphOVdxLTlpWERCYjFBcUVVTjJyc2V5Rl9XNkl3UUNtUkI1TFRVYWp6Rktvb3NQMEtseEFyM0RPNE9MYlhDLU5lZjdaeVpKR2h2N3I2UEJ2dXNTVXVVZ1c1OTNmUjdFRVRjRmFfcGxGOHgyeWItMTBhT29pdGdZSVJNWEJwb08wUU53b05nNUF1TlEwSy16VUI3cmIyZ3BSTEZtaWwzX2F1cE1FMFh5amp4eVBCSlZDbmxzcWdnczZfSjkxeE5WZUtMSzNDVE15UmoyeWlwM29SNUVSYjFhR1JCWmNyajBaTmpwQnNNUTRkTHpzRlNWck5JX3ZsRnVvQWFBQ0JUaG1UdjNyQlo2bVBNQzlUZmlFTFZtMF8tY0RQaWtHeXFvOERjcWhDck85STUxdER1azFtZk1MRTNCa3Q5UzJfYWk3RGVqa21FUWJXdyIsInRva2VuX3R5cGUiOiJwb3AifQ.ZENBlF9X9iSQMl_bpERPaM1EMWIOPpJc4dJq6_Fgk3LG_9fuXyIt2DrUyafnpOrXOcKuxrfDYheCCGJZBnavL0pgWG7thUvNp-3M68USAi_MkfmhICMo3G63Nhd1JFVQwXLddqQ8T1c33bjqlUJUer4KiH4Vr7hsWBjOdUmVvOQUUS0949MrxWhz3teoY7kO8t5iQR9ueAq6MNxFKvhUgSKx5SCSJGlnz6V1wRy42357VzgQE6XI-b1WR2-H3c-1G7URsNRxL09VkLI9ACu5OB3vjCrkD8f-3y8hss-_tEz2bkeagvLO6sgxnC3VlS_F8Ohml1y-1FI5DMB9ZwX8nw'   -H 'Referer: https://localhost:8443/test1443/index.ttl'   -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36'   --compressed -k

real    1m9.399s
user    0m0.016s
sys 0m0.011s

The slowness is probably in https://github.com/solid/acl-check/blob/master/src/acl-check.js#L64 as @timbl mentioned in https://gitter.im/solid/solidos?at=5f3bd7c9a8c178017658a369

michielbdejong commented 4 years ago

@timbl you're right, it's fetching the remote owner profile 18 times. So even if it's not timing out, it just takes a long time.

timbl commented 4 years ago

If it used an rdflib store.load() then it would cache automatically of course. Would there be other problems with deleting the old stuff to prevent it growing indefinitely.

michielbdejong commented 4 years ago

Performance looks a lot better in NSS v5.4.4, we'll see how hit holds up in production and reopen this issue if necessary.