malloydata / malloy

Malloy is an experimental language for describing data relationships and transformations.
http://www.malloydata.dev
MIT License
1.99k stars 76 forks source link

expand support for postgres datatypes #255

Closed anikaks closed 2 years ago

anikaks commented 2 years ago

boolean type for sure not currently supported. @lloydtabb is on this but documenting for posterity.

jonathanswenson commented 2 years ago

Added some types here: #259

but there are several others that I didn’t know what to do with that are referenced in the pg_catalog schema (referenced in #251)

bytea
ARRAY (in caps for some reason)
pg_ndistinct
pg_mcv_list
pg_dependencies
interval
inet
pg_node_tree
bytea
pg_lsn
anyarray
regproc
regtype

wrote some simple explores that let me play with them that might be useful (the “missing” type is a comment at the end of the line): Any commented out line was NOT working as of #259 but the others were at least possible to query / load the schema for a preview.

explore: pg_tables is table('pg:pg_catalog.pg_tables') {}
explore: pg_authid is table('pg:pg_catalog.pg_authid') {}
-- explore: pg_statistic is table('pg:pg_catalog.pg_statistic') {} // array
--  explore: pg_type is table('pg:pg_catalog.pg_type') {} // regproc
--  explore: pg_foreign_table is table('pg:pg_catalog.pg_foreign_table') {} // array
--  explore: pg_statistic_ext_data is table('pg:pg_catalog.pg_statistic_ext_data') {} // pg_ndistinct
--  explore: pg_user_mapping is table('pg:pg_catalog.pg_user_mapping') {} // array
--  explore: pg_subscription is table('pg:pg_catalog.pg_subscription') {} // array
--  explore: pg_attribute is table('pg:pg_catalog.pg_attribute') {} // array
--  explore: pg_proc is table('pg:pg_catalog.pg_proc') {} // regproc
--  explore: pg_class is table('pg:pg_catalog.pg_class') {} // array
--  explore: pg_attrdef is table('pg:pg_catalog.pg_attrdef') {} // pg_node_tree
--  explore: pg_constraint is table('pg:pg_catalog.pg_constraint') {} // regproc
 explore: pg_inherits is table('pg:pg_catalog.pg_inherits') {} // empty for me
--  explore: pg_index is table('pg:pg_catalog.pg_index') {} // array
--  explore: pg_operator is table('pg:pg_catalog.pg_operator') {} // regproc
 explore: pg_opfamily is table('pg:pg_catalog.pg_opfamily') {}
 explore: pg_opclass is table('pg:pg_catalog.pg_opclass') {}
--  explore: pg_am is table('pg:pg_catalog.pg_am') {} // regproc
 explore: pg_amop is table('pg:pg_catalog.pg_amop') {}
--  explore: pg_amproc is table('pg:pg_catalog.pg_amproc') {} // regproc
--  explore: pg_language is table('pg:pg_catalog.pg_language') {} // array
--  explore: pg_largeobject_metadata is table('pg:pg_catalog.pg_largeobject_metadata') {} // array
--  explore: pg_aggregate is table('pg:pg_catalog.pg_aggregate') {} // regproc
--  explore: pg_statistic_ext is table('pg:pg_catalog.pg_statistic_ext') {} // array
--  explore: pg_rewrite is table('pg:pg_catalog.pg_rewrite') {} // pg_node_tree
--  explore: pg_trigger is table('pg:pg_catalog.pg_trigger') {} // array
--  explore: pg_event_trigger is table('pg:pg_catalog.pg_event_trigger') {} // array
 explore: pg_description is table('pg:pg_catalog.pg_description') {}
 explore: pg_cast is table('pg:pg_catalog.pg_cast') {}
 explore: pg_enum is table('pg:pg_catalog.pg_enum') {} // empty for me
--  explore: pg_namespace is table('pg:pg_catalog.pg_namespace') {} // array
--  explore: pg_conversion is table('pg:pg_catalog.pg_conversion') {} // regproc
 explore: pg_depend is table('pg:pg_catalog.pg_depend') {}
--  explore: pg_database is table('pg:pg_catalog.pg_database') {} // array
--  explore: pg_db_role_setting is table('pg:pg_catalog.pg_db_role_setting') {} // array
--  explore: pg_tablespace is table('pg:pg_catalog.pg_tablespace') {} // array
 explore: pg_auth_members is table('pg:pg_catalog.pg_auth_members') {}
 explore: pg_shdepend is table('pg:pg_catalog.pg_shdepend') {}
 explore: pg_shdescription is table('pg:pg_catalog.pg_shdescription') {}
 explore: pg_ts_config is table('pg:pg_catalog.pg_ts_config') {}
 explore: pg_ts_config_map is table('pg:pg_catalog.pg_ts_config_map') {}
 explore: pg_ts_dict is table('pg:pg_catalog.pg_ts_dict') {}
--  explore: pg_ts_parser is table('pg:pg_catalog.pg_ts_parser') {} // regproc
--  explore: pg_ts_template is table('pg:pg_catalog.pg_ts_template') {} // regproc
--  explore: pg_extension is table('pg:pg_catalog.pg_extension') {} // array
--  explore: pg_foreign_data_wrapper is table('pg:pg_catalog.pg_foreign_data_wrapper') {} // array
--  explore: pg_foreign_server is table('pg:pg_catalog.pg_foreign_server') {} // array
--  explore: pg_policy is table('pg:pg_catalog.pg_policy') {} // array
 explore: pg_replication_origin is table('pg:pg_catalog.pg_replication_origin') {} // empty for me
--  explore: pg_default_acl is table('pg:pg_catalog.pg_default_acl') {} // array
--  explore: pg_init_privs is table('pg:pg_catalog.pg_init_privs') {} // array
 explore: pg_seclabel is table('pg:pg_catalog.pg_seclabel') {} // empty for me
 explore: pg_shseclabel is table('pg:pg_catalog.pg_shseclabel') {} // empty
 explore: pg_collation is table('pg:pg_catalog.pg_collation') {}
--  explore: pg_partitioned_table is table('pg:pg_catalog.pg_partitioned_table') {} // array
--  explore: pg_range is table('pg:pg_catalog.pg_range') {} // regproc
--  explore: pg_transform is table('pg:pg_catalog.pg_transform') {} // regproc
 explore: pg_sequence is table('pg:pg_catalog.pg_sequence') {} // empty
 explore: pg_publication is table('pg:pg_catalog.pg_publication') {} // empty
 explore: pg_publication_rel is table('pg:pg_catalog.pg_publication_rel') {} // empty
--  explore: pg_subscription_rel is table('pg:pg_catalog.pg_subscription_rel') {} // pg_lsn
--  explore: pg_largeobject is table('pg:pg_catalog.pg_largeobject') {} // bytea
lloydtabb commented 2 years ago

This is on its way and will keep things moving. We need an 'opaque' internal type for types we won't operate on.

https://github.com/looker-open-source/malloy/pull/262

anikaks commented 2 years ago

Closing in favor of more specific type issues, following https://github.com/looker-open-source/malloy/pull/262