search

solid / solid-spec

Solid specification draft 0.7.0
Creative Commons Zero v1.0 Universal
1.13k stars 103 forks source link

Specify exactly what globbing patterns are allowed #147

Open RubenVerborgh opened 5 years ago

RubenVerborgh commented 5 years ago

The spec does not specify what globbing patterns it allows. Is it just a single * at the end, or possibly more?

It does say that:

the aggregation process is not currently recursive, therefore it will not apply to children containers.

Inspired by https://github.com/solid/solid/issues/253#issuecomment-476838647 (@melvincarvalho)

RubenVerborgh commented 5 years ago

Related: do we have any more history on globbing besides https://github.com/solid/solid-spec/commit/c26141e57db9f1fcf1fd36428aa6540db79d0dcd ? @dmitrizagidulin

melvincarvalho commented 5 years ago

I've been around solid a long time, and I've probably looked at every app ever made. I've never seen anything other than /* used to get the contents of a directory.

I think that is how it was supposed to be. Basically we organized things like putting stuff in a per monthly folder and so on. It would just be a pain to get every item from every day, which could be 100s. Also the queries change a bit.

I was a bit perplexed when I saw that globbing could be used as /foo* type queries. Because why would you do that? It seems overkill, and using it 'just because you can'. Better to design your app's data not to need it.

I might have seen @bblfish once make an argument around generic globbing, but unsure anything came of it. @deiu may also have some back ground on this.

I do think there is a path towards deprecating globbig, for more efficient methods, but I'd like to see that over a long period of time. And simple server side testing is not going to catch all the issues. I dont mind turning off globbing if everyone agrees its not needed anymore because we have something better.

melvincarvalho commented 5 years ago

Recursive? I dont think it has ever been recursive, has it?

RubenVerborgh commented 5 years ago

@melvincarvalho Thanks, this is super important. Because currently, NSS uses glob, which supports much more complex patterns (and that was a major reason for my concerns). So it seems that NSS is implementing a superset of globbing as defined by the spec.

Questions:

  1. Do you have any objection to stating that a glob pattern can only contain one star?
  2. Do you have any objection to stating that this star can only occur at the end?
  3. Do you have any objection to stating that this star can only occur after a final slash?
melvincarvalho commented 5 years ago

Do you have any objection to stating that a glob pattern can only contain one star?

No

Do you have any objection to stating that this star can only occur at the end?

No

Do you have any objection to stating that this star can only occur after a final slash?

No

This is really how I thought globbing worked. The idea is that how you fetch stuff influences how you design the data layout of your app. Once you get beyond simple apps, say for a social network or chat that grows over time, you need to archive things daily, monthly, yearly etc. Your ability to fetch from this kind of folder will influence the design of your data shapes and also your app.

It may be self evident that you could have 100s of items in a monthly LDPC, and that works well with globbing. That was the need for which is was introduced.

Any other need, I dont see right now, and looks to me like feature creep. Id genuinely be interested if there's a use case out there.

RubenVerborgh commented 5 years ago

Thanks so much. PR forthcoming (which I will leave open for a bit until everyone agrees on the above answers).

This changes a lot 🙂 I have always seen the complex implementation of globbing (with multiple stars on arbitrary places and thus recursion). I'm opposed to including that; way less opposed to a "concatenate all in folder", which is what the current "globbing" does.

melvincarvalho commented 5 years ago

Ah right you should run this by @timbl because he uses the yyyy/mm/dd pattern so I could see he might want yyyy/*/*

However I've always used the yyyymmdd/ pattern for this very reason. A single glob is a nice to have, but 2 seems self indulgent! :)

RubenVerborgh commented 5 years ago

@melvincarvalho Thanks, invited you and @timbl to review https://github.com/solid/solid-spec/pull/148.

dmitrizagidulin commented 5 years ago

@RubenVerborgh

Related: do we have any more history on globbing

I'm not sure. I've never used it.