A data management system that allows users to share, annotate, organize and analyze large collections of datasets. It provides support for extensible metadata annotation using JSON-LD and a distribute analytics event bus for automatic curation of uploaded data.
Describe the bug
A clear and concise description of what the bug is.
When clicking on a parent collection, the level underneath (in this case, 1.6million datasets takes minutes to load.
The file tree and event log are turned off, and nothing is currently writing to the Clowder instance. This is just the system itself attempting to load the records stored in it. I also tried with traefik to spread the load and increase the memory to 4GB in Docker Compose, but the same lag occurs. It appears that when clicking on the top level collection instead of just loading the pagingated datasets, perhaps every dataset is loaded into memory?
To Reproduce
Steps to reproduce the behavior:
Go to Parent Collection
Click on Parent Collection
Scroll down to '....'
See error
There is a bad gateway error sometimes, or it may just take up to 5 minutes to show the first page of datasets.
Expected behavior
A clear and concise description of what you expected to happen.
I expected little or no lag.
Screenshots
If applicable, add screenshots to help explain your problem.
The docker CPU percentage will remain above 100 to 300% for several minutes and either time out, or eventually render the page.
Desktop (please complete the following information):
OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]
Linux on a virtual machine.
I'm using Chrome:latest
Smartphone (please complete the following information):
Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]
Additional context
Add any other context about the problem here.
Describe the bug A clear and concise description of what the bug is.
When clicking on a parent collection, the level underneath (in this case, 1.6million datasets takes minutes to load. The file tree and event log are turned off, and nothing is currently writing to the Clowder instance. This is just the system itself attempting to load the records stored in it. I also tried with traefik to spread the load and increase the memory to 4GB in Docker Compose, but the same lag occurs. It appears that when clicking on the top level collection instead of just loading the pagingated datasets, perhaps every dataset is loaded into memory?
To Reproduce Steps to reproduce the behavior:
Go to Parent Collection
Click on Parent Collection
Scroll down to '....'
See error There is a bad gateway error sometimes, or it may just take up to 5 minutes to show the first page of datasets.
Expected behavior A clear and concise description of what you expected to happen.
I expected little or no lag.
Screenshots If applicable, add screenshots to help explain your problem. The docker CPU percentage will remain above 100 to 300% for several minutes and either time out, or eventually render the page.
Desktop (please complete the following information):
Additional context Add any other context about the problem here.