search

onepanelio / onepanel

The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.
https://docs.onepanel.ai/
Apache License 2.0
709 stars 69 forks source link

How to run OpenPanelio on own private server ( not cloud) #734

Closed arasharchor closed 3 years ago

arasharchor commented 3 years ago

If I have understood correctly, to used OpenPanelio, the data has to be saved on one of the following cloud servers, Microsoft Azure, Amazon EKS, Google GKE.

I have my own data server and I have no plans to put my data on cloud. Is it still possible to use OpenPanelio? If yes, in the docs, I could not find how to do so.

rushtehrani commented 3 years ago

I have my own data server and I have no plans to put my data on cloud. Is it still possible to use OpenPanelio? If yes, in the docs, I could not find how to do so.

@smajida We are working on updated docs for multi-node on-premises deployments. We are aiming to have those ready by end of next week.

arasharchor commented 3 years ago

@rushtehrani I was wondering whether there is any news on this?

rushtehrani commented 3 years ago

I was wondering whether there is any news on this?

@arasharchor We are still working on this - there are still issues with networking and a few other things that we are still trying to figure out. I will update #732 as we make progress.

arasharchor commented 3 years ago

@rushtehrani Thanks a lot for your prompt reply. We are working in TUM university and having working with CVAT for a while, but it is not scalable and the tool gets slow after we add long videos. Would openpanelio help us on this regard? Our server's OS is Ubuntu 16.04 and another one 18.04 The one with 16.04 version has no GPUs and we use it for annotations without GPU support.

rushtehrani commented 3 years ago

We are working in TUM university and having working with CVAT for a while, but it is not scalable and the tool gets slow after we add long videos. Would openpanelio help us on this regard?

@arasharchor Can you provide more detail around how it's getting slower? Is auto annotation slow or are you talking about CVAT responsiveness when annotating?

arasharchor commented 3 years ago

@rushtehrani

I am talking about CVAT responsiveness. I am not using any auto annotation features for now. I am annotating 100 aerial images with the size of 6000x8000px (20MB each) with more than 1000 objects and about 3000points. I would like to be able to annotate aerial images easily similar to Qgis software.

Current Behaviour

I have to wait for a long time until the image is loaded. Then takes long if I want to zoom in or out to edit points. Browser is asking to stop the page: image

image image image image

Possible Solution

There is a solution for this case by making image pyramid. Also for the annotations, there is a way of working with tiled json in aerial imagery. There is a sample for loading large images into the website here: https://sg.geodatenzentrum.de/web_dop_viewer/viewer.html#lat=48.9156&lon=9.0070&zoom=5&webatlasde=false&dop40=true&layer_4=false&layer_3=true&layer_2=true&layer_1=false

Steps to Reproduce (for bugs)

Load 100 large images with more than 6000x6000px size and populate with more than 1000 objects.

My system: Ubuntu 18.04 - CPU only

I do appreciate it if you could let me know if there is a solution for my use case.

rushtehrani commented 3 years ago

@arasharchor have you discussed this with the CVAT team? Have you also tested this on a larger machine to see if it resolves/reduces the issues?

I did find this issue that seems to have been addressed. What version of CVAT are you using?

Edit: Saw that you created an issue in CVAT, I’ll sync follow up in that issue in the coming days to see if there is anything we can do to help.

arasharchor commented 3 years ago

@rushtehrani Yes, I discussed this with CVAT team and as you can see I have also mentioned OnePanel in that issue to see if it would be a possible solution.

According to CVAT team, this is an known issue. I will work on this to see if I can solve this problem because I normally work with large images.

rushtehrani commented 3 years ago

Multi-node MicroK8s deployment on Linux machines is now available in the newly released v0.20.0. See updated quick start for instructions.