SkalskiP / make-sense

Free to use online tool for labelling photos. https://makesense.ai
GNU General Public License v3.0
3.16k stars 523 forks source link

saving my continuous progress so i won't have to start over. #169

Open DR2495 opened 3 years ago

DR2495 commented 3 years ago

saving my continuous progress so I won't have to start over.

iamfaith commented 3 years ago

I suggest make-sense should add the concept of projects. We can click new project to start our work and save for next time.

SkalskiP commented 3 years ago

Hi 👋 , @DR2495 & @iamfaith! We could do that, but as long as make-sense will not have backend, files related to that project would need to be downloaded to your machine, for example in .zip format. And could be reuploaded next time. Do you feel it is a valuable option?

I am also very sorry for the late reply. Work factors have meant that I have had no time to develop MS.

rasyidf commented 3 years ago

Hi @SkalskiP , I found something interesting here : Paint.js

SkalskiP commented 3 years ago

Hello, once again @rasyidf 👋. That is a true, very interesting project. But... what is the relation of paint.js with that issue? 🧐

rasyidf commented 3 years ago

I guess I missed the details before,

If we want to fully utilize the PWA capability to just install the webapp locally, then integrate file handler.

Web.dev file handling

we, as a user can

on our desktop,

SkalskiP commented 3 years ago

@rasyidf I was thinking about it. We could do something similar. But to make it work similarly - load a whole project from a single file, we would need that file to be quite big. We would need to use some kind of zip structure and hide all images inside along with labels and metadata required.

rasyidf commented 3 years ago

I see, so basically you want the project behave like open Document format. read here => Wikipedia Open Doc.

make a zipped folder, with Metadata and every assets.

SkalskiP commented 3 years ago

Yes but instead of using XML we would use JSON to store labels and metadata. We are already saving zip in TypeScript so I do not think it would be an issue. One thing that is worrying is the size of the saved file. It can be easily a few hundred MB. So for some users with slower machines it can be an issue :/

rasyidf commented 3 years ago

If the make-sense is installed offline. how about making it folder based project with something like .sense/ subfolder with it's workspaced settings. it's opinionated but can be easily predicted by program

something like this : project.json

{
    "name": "pet-behaviour",
    "version": "2.3.0",
    "description": "Pet Behaviour Recognition",
    "keywords": ["veterinary", "cats", "pets", "dogs", "YOLOv5"],
    "author": "John Doe",
    "engine": {
      "model": "YOLOv5",
      "weight" : "yolov5.weights"
    },
    "paths": {
      "images": [
        "images/*"
      ],
      "annotations": [
        "labels/*"
      ],
      "masks": [
        "masks/*"
      ],
      "code": [
        "src/*"
      ]
    }
  }

then, we can make the folder structured as described, then load it and the images will be lazy loaded, after every setup finalized.

SkalskiP commented 3 years ago

@rasyidf that's an interesting suggestion, but I really wouldn't like to have a separate solution for people using make-sense online and offline. We have around 6000 people visiting https://www.makesense.ai/ every week. And I think that we should build new features mostly for those people.

AxelAli commented 3 years ago

I believe we could use either indexedDB (similar to MySQL) or https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage We could store the pending annotations by item name/hash into indexedDB and that would persist by the user side without the need for any storage on our servers (this being particularly important in our client-side system)

SkalskiP commented 3 years ago

Hello, @AxelAli 👋 ! Many thanks for your interest in our project and for your ideas. However, both of them have some drawbacks that will probably stop us from using them.

AxelAli commented 2 years ago

Hi @SkalskiP , IndexedDB lives in the user's browser, no installs required. Here you can see an example demo (Try adding a note, changing the URL to Google or something, and open the same link in another tab), notes will persist. Data isn't stored anywhere, and it's persisted even without internet Regarding images , we can either do something like this, or store them in base64 and then drop them in indexedDB https://hacks.mozilla.org/2012/02/storing-images-and-files-in-indexeddb/

SkalskiP commented 2 years ago

Hi @AxelAli! I have to admit that these are very interesting ideas. 💪 I'll have to look into that. I'm curious to see how efficient this solution is. We have to remember that projects have very often 500 or 1000 images. But thank you very much for this idea. I am also sorry that I reacted so negatively. I was simply not aware of the existence of this feature. 🙏

Let me do the research.