Unable to write

[GlenMWadleigh]

I am attempting the quickstart. I am able to connect to the database but when I try to execute the write_and_read.rb I get the following error:

$ ruby demos/write_and_read.rb C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/session.rb:136:inexec_params': ERROR: function write_message(character varying, character varying, character varying, jsonb, jsonb, bigint) does not exist (PG::UndefinedFunction) LINE 1: SELECT write_message($1::varchar, $2::varchar, $3::varchar, ... ^ HINT: No function matches the given name and argument types. You might need to add explicit type casts. from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/session.rb:136:in execute' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/put.rb:86:inexecute_query' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/put.rb:67:in insert_message' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/put.rb:42:incall' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/write.rb:52:in write_message_data' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/write.rb:28:inblock (2 levels) in write' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/write.rb:26:in each' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/write.rb:26:inblock in write' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/session.rb:150:in transaction' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/session.rb:150:intransaction' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/write.rb:24:in write' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-message_store-2.3.0.1/lib/message_store/write.rb:55:incall' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-messaging-2.5.0.1/lib/messaging/write.rb:66:in call' from C:/Users/name/quickstart/gems/ruby/2.6.0/gems/evt-messaging-2.5.0.1/lib/messaging/write.rb:45:incall' from demos/write_and_read.rb:34:in <main>'

The line numbers don't exactly line up because I have added print statements to help me track down the issue. I am new to Ruby so I might be overlooking something obvious in the setup.

[stulzer]

I confirm the issue on the setup. Didn't stop to dig into it, but I am also getting the same error when following the quickstart guide.

Labs/eventide-start/gems/ruby/2.7.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/session.rb:117:in `exec_params': ERROR:  function write_message(character varying, character varying, character varying, jsonb, jsonb, bigint) does not exist (PG::UndefinedFunction)
LINE 1: SELECT write_message($1::varchar, $2::varchar, $3::varchar, ...
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

I am not new to Ruby, but totally new to eventide. I was also just following the quickstart guide and stumble upon the same issue. Thought that was the a problem with postgresql on version 12.1, then I downgraded to 11.6 and the issue persists.

[GlenMWadleigh]

After reviewing similar errors messages around the web and attempting the other eventide examples and getting the same errors I realized this error is occurring because the Postgres database is not set up correctly.

I tried the automated code (message-db-master) to set up the database and it keeps failing to execute completely. I am not sure how or why this is happening.

On Fri, Feb 14, 2020 at 12:01 PM Rubens Stulzer notifications@github.com wrote:

I confirm the issue on the setup. Didn't stop to dig into it, but I am also getting the same error when following the quickstart guide.

Labs/eventide-start/gems/ruby/2.7.0/gems/evt-message_store-postgres-2.4.0.2/lib/message_store/postgres/session.rb:117:in `exec_params': ERROR: function write_message(character varying, character varying, character varying, jsonb, jsonb, bigint) does not exist (PG::UndefinedFunction) LINE 1: SELECT write_message($1::varchar, $2::varchar, $3::varchar, ... ^ HINT: No function matches the given name and argument types. You might need to add explicit type casts.

I am not new to Ruby, but totally new to eventide. I was also just following the quickstart guide and stumble upon the same issue. Thought that was the a problem with postgresql on version 12.1, then I downgraded to 11.6 and the issue persists.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/eventide-examples/quickstart/issues/3?email_source=notifications&email_token=AORHIMZ72LWJCW5WFTS2WDTRC3L6TA5CNFSM4KT4VES2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELZ4OWY#issuecomment-586401627, or unsubscribe https://github.com/notifications/unsubscribe-auth/AORHIM4KLRRLAZBU5ZUPSRLRC3L6TANCNFSM4KT4VESQ .

[GlenMWadleigh]

After reviewing similar errors messages around the web and attempting the over eventide examples and getting the same errors I realized this error is occurring because the Postgres database is not set up correctly. I tried the automated code to set up the database and it keeps failing to execute completely. I am not sure how or why it is failing.

[sbellware]

@GlenMWadleigh Is there an error that you're seeing when trying to create the database?

Also, is this a plain old install of Postgres? Anything special about it?

[sbellware]

Note: If the instance of Postgres is on some "special" cloud version of Postgres that isn't stock Postgres, it shouldn't be expected to work. Heroku is one of those "special" clouds where Postgres is effectively like Postgres, but not exactly stock Postgres. Anyone's third party container or image of a Postgres install is also as non-deterministic. So, just a heads up... variants, specializations, and handicapped versions of Postgres should not be expected to work like stock Postgres.

[sbellware]

Also, just for sake of thoroughness, if you change the message_store role name to something other than what's defined in the Message DB DDL, of course the set up of the database will fail. We've seen people try to do this without accounting for how often that role name must be referenced throughout the setup scripts. If you want to change the role name (and there isn't much justification for doing this, but nonetheless), make sure it's changed throughout the SQL scripts that define the message store.

[GlenMWadleigh]

@sbellware, the current error I am getting is this:

$ sh recreate-database.sh

Uninstalling Database Version: 1.1.6 = = = DATABASE_NAME is not set. Using: message_store.

Deleting database

---

» message_store database psql: warning: extra command-line argument "pager=off" ignored psql: warning: extra command-line argument "-q" ignored psql: warning: extra command-line argument "-c" ignored psql: warning: extra command-line argument "DROP DATABASE IF EXISTS message_store;" ignored

I set up a local copy of Postgresql and I do not believe there is anything unusual about the settings. The role may be an issue. I did encounter so password issues and set the identity of the user in settings and gave them a password.

[sbellware]

@GlenMWadleigh I've never seen the psql command report that command line arguments are ignored. It's even ignoring the actual SQL commands themselves.

I have a feeling that this isn't an issue with Postgres itself rather than an issue with the shell environment. Specifically, the way that the script is being parsed.

Is there something specialized about the way that you run bash that would account for variation in the way that scripts are parsed?

[sbellware]

@GlenMWadleigh PS: What do you mean by "set the identity of the user in settings and gave them a password"?

I'm not sure what the "identity" is that you're referring to. Also, I'm not clear what settings you might be referring to.

[ntl]

@GlenMWadleigh can you try running the installation script in tracing mode, which causes each command that the script executes to be printed? Example:

bash -x ./database/uninstall.sh
bash -x ./database/install.sh

Can you run database/install.sh with the -x flag (that turns on tracing mode) and then paste your output?

Here is my output, as a reference:

~/Projects/message-db/message-db (/home/ntl/Projects/message-db/message-db)
> bash -x ./database/install.sh 
+ set -e
++ script_dir
++++ dirname ./database/install.sh
+++ cd ./database
+++ pwd
++ val=/home/ntl/Projects/message-db/message-db/database
++ echo /home/ntl/Projects/message-db/message-db/database
+ base=/home/ntl/Projects/message-db/message-db/database
+ echo

+ echo 'Installing Database'
Installing Database
++ cat /home/ntl/Projects/message-db/message-db/database/VERSION.txt
+ echo 'Version: 1.1.6'
Version: 1.1.6
+ echo '= = ='
= = =
+ '[' -z ']'
+ database=message_store
+ echo 'DATABASE_NAME is not set. Using: message_store.'
DATABASE_NAME is not set. Using: message_store.
+ export DATABASE_NAME=message_store
+ DATABASE_NAME=message_store
+ '[' -z ']'
+ export 'PGOPTIONS=-c client_min_messages=warning'
+ PGOPTIONS='-c client_min_messages=warning'
+ echo

+ echo 'Creating User'
Creating User
+ echo '- - -'
- - -
+ create-user
++ script_dir
++++ dirname ./database/install.sh
+++ cd ./database
+++ pwd
++ val=/home/ntl/Projects/message-db/message-db/database
++ echo /home/ntl/Projects/message-db/message-db/database
+ base=/home/ntl/Projects/message-db/message-db/database
+ echo '» message_store role'
» message_store role
+ psql postgres -q -f /home/ntl/Projects/message-db/message-db/database/roles/message-store.sql
+ echo

+ echo 'Creating Database'
Creating Database
+ echo '- - -'
- - -
+ create-database
+ echo '» message_store database'
» message_store database
+ createdb message_store
+ echo

+ echo 'Creating Schema'
Creating Schema
+ echo '- - -'
- - -
+ create-schema
+ echo '» message_store schema'
» message_store schema
+ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/schema/message-store.sql
+ echo

+ echo 'Creating Extensions'
Creating Extensions
+ echo '- - -'
- - -
+ create-extensions
++ script_dir
++++ dirname ./database/install.sh
+++ cd ./database
+++ pwd
++ val=/home/ntl/Projects/message-db/message-db/database
++ echo /home/ntl/Projects/message-db/message-db/database
+ base=/home/ntl/Projects/message-db/message-db/database
+ echo '» pgcrypto extension'
» pgcrypto extension
+ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/extensions/pgcrypto.sql
+ echo

+ echo 'Creating Table'
Creating Table
+ echo '- - -'
- - -
+ create-table
++ script_dir
++++ dirname ./database/install.sh
+++ cd ./database
+++ pwd
++ val=/home/ntl/Projects/message-db/message-db/database
++ echo /home/ntl/Projects/message-db/message-db/database
+ base=/home/ntl/Projects/message-db/message-db/database
+ echo '» messages table'
» messages table
+ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/tables/messages.sql
+ echo

+ source /home/ntl/Projects/message-db/message-db/database/install-functions.sh
++ set -e
++ echo 'Creating Types'
Creating Types
++ echo '- - -'
- - -
++ create-types
++ '[' -z x ']'
++ database=message_store
+++ script_dir
+++++ dirname /home/ntl/Projects/message-db/message-db/database/install-functions.sh
++++ cd /home/ntl/Projects/message-db/message-db/database
++++ pwd
+++ val=/home/ntl/Projects/message-db/message-db/database
+++ echo /home/ntl/Projects/message-db/message-db/database
++ base=/home/ntl/Projects/message-db/message-db/database
++ echo '» message type'
» message type
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/types/message.sql
++ echo

++ echo 'Creating Functions'
Creating Functions
++ echo '- - -'
- - -
++ create-functions
++ '[' -z x ']'
++ database=message_store
+++ script_dir
+++++ dirname /home/ntl/Projects/message-db/message-db/database/install-functions.sh
++++ cd /home/ntl/Projects/message-db/message-db/database
++++ pwd
+++ val=/home/ntl/Projects/message-db/message-db/database
+++ echo /home/ntl/Projects/message-db/message-db/database
++ base=/home/ntl/Projects/message-db/message-db/database
++ echo '» message_store_version function'
» message_store_version function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/message-store-version.sql
++ echo '» hash_64 function'
» hash_64 function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/hash-64.sql
++ echo '» acquire_lock function'
» acquire_lock function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/acquire-lock.sql
++ echo '» category function'
» category function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/category.sql
++ echo '» is_category function'
» is_category function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/is-category.sql
++ echo '» id function'
» id function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/id.sql
++ echo '» cardinal_id function'
» cardinal_id function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/cardinal-id.sql
++ echo '» stream_version function'
» stream_version function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/stream-version.sql
++ echo '» write_message function'
» write_message function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/write-message.sql
++ echo '» get_stream_messages function'
» get_stream_messages function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/get-stream-messages.sql
++ echo '» get_category_messages function'
» get_category_messages function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/get-category-messages.sql
++ echo '» get_last_stream_message function'
» get_last_stream_message function
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/functions/get-last-stream-message.sql
++ echo

+ source /home/ntl/Projects/message-db/message-db/database/install-indexes.sh
++ set -e
++ echo 'Creating Indexes'
Creating Indexes
++ echo '- - -'
- - -
++ create-indexes
++ '[' -z x ']'
++ database=message_store
+++ script_dir
+++++ dirname /home/ntl/Projects/message-db/message-db/database/install-indexes.sh
++++ cd /home/ntl/Projects/message-db/message-db/database
++++ pwd
+++ val=/home/ntl/Projects/message-db/message-db/database
+++ echo /home/ntl/Projects/message-db/message-db/database
++ base=/home/ntl/Projects/message-db/message-db/database
++ echo '» messages_id index'
» messages_id index
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/indexes/messages-id.sql
++ echo '» messages_stream index'
» messages_stream index
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/indexes/messages-stream.sql
++ echo '» messages_category index'
» messages_category index
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/indexes/messages-category.sql
++ echo

+ source /home/ntl/Projects/message-db/message-db/database/install-views.sh
++ set -e
++ echo 'Creating Views'
Creating Views
++ echo '- - -'
- - -
++ create-views
++ '[' -z x ']'
++ database=message_store
+++ script_dir
+++++ dirname /home/ntl/Projects/message-db/message-db/database/install-views.sh
++++ cd /home/ntl/Projects/message-db/message-db/database
++++ pwd
+++ val=/home/ntl/Projects/message-db/message-db/database
+++ echo /home/ntl/Projects/message-db/message-db/database
++ base=/home/ntl/Projects/message-db/message-db/database
++ echo '» stream_summary view'
» stream_summary view
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/views/stream-summary.sql
++ echo '» type_summary view'
» type_summary view
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/views/type-summary.sql
++ echo '» stream_type_summary view'
» stream_type_summary view
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/views/stream-type-summary.sql
++ echo '» type_stream_summary view'
» type_stream_summary view
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/views/type-stream-summary.sql
++ echo '» category_type_summary view'
» category_type_summary view
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/views/category-type-summary.sql
++ echo '» type_category_summary view'
» type_category_summary view
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/views/type-category-summary.sql
++ echo

+ source /home/ntl/Projects/message-db/message-db/database/install-privileges.sh
++ set -e
++ echo 'Granting Privileges'
Granting Privileges
++ echo '- - -'
- - -
++ grant-privileges
++ '[' -z x ']'
++ database=message_store
+++ script_dir
+++++ dirname /home/ntl/Projects/message-db/message-db/database/install-privileges.sh
++++ cd /home/ntl/Projects/message-db/message-db/database
++++ pwd
+++ val=/home/ntl/Projects/message-db/message-db/database
+++ echo /home/ntl/Projects/message-db/message-db/database
++ base=/home/ntl/Projects/message-db/message-db/database
++ echo '» schema privileges'
» schema privileges
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/privileges/schema.sql
++ echo '» messages table privileges'
» messages table privileges
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/privileges/table.sql
++ echo '» sequence privileges'
» sequence privileges
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/privileges/sequence.sql
++ echo '» functions privileges'
» functions privileges
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/privileges/functions.sql
++ echo '» views privileges'
» views privileges
++ psql message_store -q -f /home/ntl/Projects/message-db/message-db/database/privileges/views.sql
++ echo

+ echo '= = ='
= = =
+ echo 'Done Installing Database'
Done Installing Database
++ cat /home/ntl/Projects/message-db/message-db/database/VERSION.txt
+ echo 'Version: 1.1.6'
Version: 1.1.6
+ echo

[sbellware]

@GlenMWadleigh Also, are you running a current version of Postgres and psql?

[sbellware]

@stulzer That failure means that the write_message function either doesn't exist or that it can't be found within the Postgres user's search path.

Did you change anything about the Postgres user/role?

[stulzer]

@sbellware Indeed I've changed the user on the settings/message_store_postgres.json, just switched back to message_store and voila, it works now. ;)

I thought initially that the settings/message_store_postgres.json was some sort of config/database.yml from Rails. And I've changed the user to match what I generally do on database.yml. And since I got the exact same message from this issue I've made my comment.

I think it worth to mention on the start guide, to not change the user or dbname there.

So @GlenMWadleigh did you changed something on the settings/message_store_postgres.json file? If you did, try to checkout and get back to the default values to try again.

[sbellware]

@stulzer (and @GlenMWadleigh) It's deadly important to consider that Rails is an anomaly in the way that it approaches database operations. It treats the storage tier as merely an incidental detail of having a models directory.

Architecturally, a Rails database is only ever an implementation detail of the models directory. It's not its own database, in and of itself. It's literally just a part of the Rails application. It's an example of Rails disregarding all the rules of sane and sensible software development, operations, and maintenance.

And ultimately, that's fine. Rails is not a framework for enabling long-term consideration of software sustainability. It's a framework for rushing a prototype, bypassing all sensible guidelines.

Why this matters: Nothing else treats a database with as much cavalier disregard as Rails does. And it's ok that Rails does this. It's a Rails database; it belongs to Rails, and Rails can choose to do whatever it wants with its own artifacts.

But it's an anomaly in the database operations world. It's a very popular anomaly, but that's just a reflection of demographics rather than design principles. There's no shortage of examples of technologists rushing toward a minimal implementation of a prototype disregarding the design principles that make their projects survivable, operable, and safe long term.

Message DB isn't Rails. It reflects data operations expectations of data operators, not just applications developers. It's not built to reflect Rails' shortcuts. The architectures that it supports are not survivable if the kinds of shortcuts typical of web prototyping are pursued outside of the web prototyping context.

All this to say that the tools for administering a database are the tools provided by the database. In the case of Postgres, the fundamental tool is psql. And at an even more fundamental level, the libpq library.

Message DB is of Postgres, not Rails. Its administration and operations regime is that of Postgres, not Rails.

Administration of a Postgres database is intended to be done via psql, and Message DB respects this, and must respect this. If it did not, it would not and should not be taken seriously as a Postgres solution.

All that said, we can now put Eventide's message_store_postgres.settings file into proper perspective:

It's the runtime configuration file for a specific client implementation of Message DB, written in Ruby, and named "Eventide". It has absolutely nothing to do with Message DB itself. It has to do only with establishing a runtime Message DB connection from an Eventide implementation.

The reason that Rails is an anomaly - and "abomination" is an appropriate word here - is that it uses the same connection configuration for both runtime applicative connection to its database, as well as the administrative connection. This is an otherwise horrific idea and would disqualify Message DB from consideration by - for lack of a better term - "serious" data operators and data security auditors.

In the end, you should not presume to be able to apply the presumptions and habits of Rails to something that is - dare I say - not as immature in its approach as Rails.

The message_store security role created by the Message DB installation script is an integral part of the database. It's not just an incidental detail the way that the database user might be with a Rails database. You should no more expect the database to remain operable by changing the message_store role than if you have changed the name of the messages table or the write_message function.

In Postgres, a role is an organizational unit to which permissions grants are applied. The fact that a Postgres role can also use ascribed an additional quality that allows it to be used as a login is entirely secondary and incidental (though obviously vital).

You can monkey with the message_store role if you want, that's part of the reason why the message store database definition is provided as raw Postgres DDL scripts, in addition to allowing experienced Postgres data operators the opportunity to audit Message DB and certify it for their environments. But if you do, you must ensure that its constraints remain consistent with the rest of the database definition.

Message DB and Eventide aren't built for the level of understanding that beginners are presently at, the way that Rails is. It's built to give beginners an opportunity to go beyond the level of understanding where Rails resides. And the Eventide and Message DB community is managed to facilitate and support that transition, and to provide stewardship of the process as the most solemn of the Eventide community's duties.

Eventide doesn't shield developers from more mature data operations regimes. It doesn't presume that developers are incapable of leveling up. It invites them to go further into the knowledge and rigors, and provides a path and a community to facilitate it.

And that also means that it's not as approachable to beginners the way that Rails is. And that's on purpose. The kinds of architectures that Message DB supports aren't appropriate to the same audience that Rails is. But it does provide a pathway of upward mobility toward evented, service-oriented, and autonomous systems. And a great deal of attention has been invested into the training curriculum that takes a person deeper into the world of evented and autonomous architecture.

But the limiting force that has to be overcome is all the things that Rails has conditioned us to presume and believe. Especially with regard to data operations. All the safety edges and cushioned corners that Rails provides that helps beginners successfully enter the world of software development are no longer applicable in the world of architectures that things like Eventide are intended for.

But there's a hell of a lot more power available when we're no longer tethered to the safety measures imposed by things like Rails, useful as they may be at certain times in the evolution of a software lifetime.

I hope this helps, and puts not only data operations considerations into broader perspective, but also Eventide itself, and Message DB.

[GlenMWadleigh]

Thank you so much for all the suggestions! It will take me a bit of time to go through all this but I really appreciate the feedback and assistance.

[sbellware]

@GlenMWadleigh (and @stulzer) No problem. I hope it helps.

The conversation on the Eventide Slack is a lot richer and a lot more immediate. If you're so included to add another Slack org to your life, sign up at: https://eventide-project-slack.herokuapp.com/