Proposal: Fancy folder structure for DB4S module

[MKleusberg]

These are actually a number of proposals and I will try to order them by urgency:

1) Consolidate the two types of JSON responses for directory browsing. Here's what we produce now:

[
  {
    "url": "https://dev2.dbhub.io:5550/justinclift",
    "last_modified": "14 Mar 17 21:53 UTC"
  },
  {
    "url": "https://dev2.dbhub.io:5550/mkleusberg",
    "last_modified": "27 Jan 17 14:23 UTC"
  }
]

and

[
  {
    "database_name": "issue946.db",
    "version": 1,
    "url": "https://dev2.dbhub.io:5550/mkleusberg/issue946.db?version=1",
    "size": 8192,
    "last_modified": "27 Jan 17 14:23 UTC"
  },
  {
    "database_name": "pr921.db",
    "version": 1,
    "url": "https://dev2.dbhub.io:5550/mkleusberg/pr921.db?version=1",
    "size": 20480,
    "last_modified": "08 Jan 17 23:08 UTC"
  }
]

Here's what I would like to have:

[
  {
    "type": "folder",
    "name": "justinclift",
    "url": "https://dev2.dbhub.io:5550/justinclift",
    "last_modified": "14 Mar 17 21:53 UTC"
  },
  {
    "type": "folder",
    "name": "mkleusberg",
    "url": "https://dev2.dbhub.io:5550/mkleusberg",
    "last_modified": "27 Jan 17 14:23 UTC"
  }
]

and

[
  {
    "type": "database",
    "name": "issue946.db",
    "version": 1,
    "url": "https://dev2.dbhub.io:5550/mkleusberg/issue946.db?version=1",
    "size": 8192,
    "last_modified": "27 Jan 17 14:23 UTC"
  },
  {
    "type": "database",
    "name": "pr921.db",
    "version": 1,
    "url": "https://dev2.dbhub.io:5550/mkleusberg/pr921.db?version=1",
    "size": 20480,
    "last_modified": "08 Jan 17 23:08 UTC"
  }
]

The idea is to basically have the same format for directories and database files so it's easier to parse the data and -- potentially -- easier to generate it too.

2) When browsing the root dir (https://dev2.dbhub.io:5550/) we're currently serving a list of all user names. Maybe it might be a better idea to hand out a list of virtual directories like this:

[
  {
    "type": "folder",
    "name": "Your databases",
    "url": "https://dev2.dbhub.io:5550/mkleusberg",
    "last_modified": "14 Mar 17 21:53 UTC"
  },
  {
    "type": "folder",
    "name": "Starred databases",
    "url": "https://dev2.dbhub.io:5550/stars",
    "last_modified": "27 Jan 17 14:23 UTC"
  },
  {
    "type": "folder",
    "name": "Recommended databases",
    "url": "https://dev2.dbhub.io:5550/recommendations",
    "last_modified": "27 Jan 17 14:23 UTC"
  }
]

And when asking for one of the sub directories, serve a list of databases fitting that category. This way it's easier to browse the databases that actually matter to you.

3) Maybe add categories, too, as virtual directories. See issue #1.

4) Add actual directories, too.

5) Maybe add an icon url attribute to the json objects for having a way to set different icons for databases and folders, or even for private vs. public databases or the starred databases folder vs. the personal databases folder.

[MKleusberg]

Maybe just one more thought about this: one main benefit seems to be that this puts most of the logic on this to the server side. This allows us to add all sorts of features (e.g. adding a notifications directory for "hey please look at this version of that database" situations when collaborating in a team) just on the fly without people having to update their local DB4S client every time.

[justinclift]

Ahhh, interesting stuff. :smile:

For 1. Consolidate the two types of JSON responses for directory browsing, yep that should be really simple.

For 2., excellent. I've been struggling to think of what to a) do with the root dir, and b) how to give the user options for looking at stuff-thats-not theirs. This seems like exactly the right thing to do both. 😺

3. No objection here. We'll need to think about the right way of doing it. At a guess, it'll build upon the above two, so I'm kind thinking we get those working first then see what the good way forward for this is.

4. Yep, folder (and sub-folder, etc) support is already included and working in a lot of the internal functions. It'll take more work to get it fully worked through and into the webUI in a good way though.

5. No objections here. We'd probably want to have commonly used icons in DB4S for efficiency, but it shouldn't hurt to allow pulling custom/special/etc icons over the network too where useful.

[justinclift]

Putting most of the stuff server side, yeah I hadn't really thought it through that far. My head is kind of in some of the implementation details at the moment rather than on the big picture. :wink: The concept you're conveying sounds decent though. :smile:

[MKleusberg]

One other thought:

6. Add dynamic(?) virtual folders. This is what I'm thinking of here: https://dev2.dbhub.io:5550/search?query=bla&orderby=size would perform a server side search and return the results as a directory listing that can be browsed client side. This one obviously needs special support on the client side, too.

[justinclift]

Ahhh, interesting idea. We'll need to figure out what we want to be searchable, to keep the server side resources reasonable (at least at the start). :smile:

eg potential scopes:

• across names of all public databases? (shouldn't be hard)
• string match inside all public databases? (probably need an Elasticsearch cluster backend for that to work)
• string match inside just the requesting user's databases? (depending on database size, this might be feasible)
• probably more potential scopes to think of too. :smile:

[MKleusberg]

Probably just names and meta data like user name and description at first :smiley: For the content we probably need some proper UI, too, for displaying all the results and narrowing them down.

[justinclift]

New issues for discussing the webUI front page concept: https://github.com/sqlitebrowser/dbhub.io/issues/36