Resurrect pgfs - Githubissues

pgfs.py was an experimental Python script that mounted the database as a file system. The root directory contained all schemas as sub-directories, which contained all relations as sub-directories, then all rows' primary keys as sub-directories, then all columns as text files:

/pgfs/{schema_name}/{relation_name}/{pk_value}/{column_name}

It was writable too, you could edit fields with your preferred text editor. It was written in python2 and used the fuse module. In python3 the fuse package seems to be in disarray, and resurrecting the project has ended in tears. Plus now that we're using Go, there's better tools in the bag. The bazil/fuse library looks really nice.

This pattern is pretty much a requirement. For example, right now I'm doing the big meta type_id flattening refactor. All the database code has been refactored, but all the widget code is actually in the database. Since the IDE won't work until the widgets are refactored, we have neither chicken nor egg, the IDE is bricked. The only option is to use a third-party database editor to refactor the IDE widgets.

There needs to be a reliable way to edit any field in the database, and I don't know of one that's suited for editing a field that is a 200 line Javascript code block. update widget.widget set html='{ 200 lines of HTML }' where name='ascii_art' isn't really acceptable. It's pretty amazing that there's no built-in pattern for editing a field's value in PostgreSQL other than UPDATE.

So, resurrect pgfs.

There are some really cool ideas in this space:

Beyond just providing a basic schema/table/pk/column scheme, also make a bundle-centric file system:

/{bundle_name}/{commit_id | 'HEAD' | 'CHECKOUT' }/{schema}/{table}/{pk}/{column_name}

Allow columns to be decorated with a mimetype:

create table column_mimetype (
    id uuid,
    column_id meta.column_id,
    mimetype endpoint.mimetype
);

endpoint.mimetype_extention contains the associated file extension(s) for a given mimetype. Append the extension onto each field's filename, so file names have the appropriate extension, like /widget/widget/12345/post_js.js. Then editors will recognize the file type and do proper syntax highlighting etc.

Implement server-side row text identifiers (#67) so, when possible, rows can be accessed by name rather than by pk, e.g. /widget/widget/row_list/post_js.js.

Should this be built into the aquameta daemon? I think so, use go func to run two processes, one for the HTTP server and one for the file system server, and make both optional.

aquametalabs / aquameta

Resurrect pgfs #260