Prototype for 'external' table type

[jamesaoverton]

This code should not be merged, but maybe it will be helpful.

It adds support for an 'external' value in the table.type column. When a table is 'external', VALVE will not try to read it, or create it, or truncate it, or load it. The entries about external tables may still be useful for other tools that use VALVE, such as Nanobot.

[jamesaoverton]

A few more thoughts about this feature:

These tables are "external" from VALVE's point of view, but from Nanobot's point of view "external" doesn't make sense. So it would be good to find another way to say "VALVE should ignore this table".

In my prototype I'm using the 'path' column for more information. If the 'path' is an HTTP URL then I want to fetch the file. If the file extension is ".owl" then I want to run it through LDTab. If the file extension is ".sql" then I want to execute that SQL script.

So instead of the check here for table.type != 'external', we could instead check that table.path ends with ".tsv" or ".csv" -- otherwise VALVE would not load the table, i.e. treat it as "external".

[lmcmicu]

A few more thoughts about this feature:

These tables are "external" from VALVE's point of view, but from Nanobot's point of view "external" doesn't make sense. So it would be good to find another way to say "VALVE should ignore this table".

I'm not sure why it matters that 'external' doesn't make sense from nanobot's point of view, since this is a valve feature. But if this is really an issue, then as an alternative we could use something like 'unmanaged'.

In my prototype I'm using the 'path' column for more information. If the 'path' is an HTTP URL then I want to fetch the file. If the file extension is ".owl" then I want to run it through LDTab. If the file extension is ".sql" then I want to execute that SQL script.

So instead of the check here for table.type != 'external', we could instead check that table.path ends with ".tsv" or ".csv" -- otherwise VALVE would not load the table, i.e. treat it as "external".

FYI, Valve already allows some flexibility with regard to the table table:

Positional arguments:
  SOURCE                The location of the valve configuration entrypoint. Can
                        be one of (A) A URL of the form `postgresql://...` or
                        `sqlite://...` indicating a database connection where
                        the valve configuration can be read from a table named
                        "table"; (B) The filename (including path) of the table
                        file (usually called table.tsv).

[jamesaoverton]

This should be covered by #87.