Installing on fresh installation of chronicle-app produces error

wongjustin99 commented 2 years ago

After installing using the commands in the README, I attempted to run:

chronicle-etl --extractor imessage --since "2022-02-07" --transformer imessage

It was them prompted with:

Plugin specified by job not installed. Do you want to install chronicle-imessage and start the job? (Y/n)

This is a little odd, since I just installed it. However, when going with "Y" or "n", I both run into a stack trace:

/Users/userboy/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.2/lib/chronicle/etl/job_definition. rb:42:in `validate!': Job definition is invalid (Chronicle::ETL::JobDefinitionError)

wongjustin99 commented 2 years ago

I am also seeing a similar issue with the shell history plugin, so it might be an issue with the "plugins installed" code in general.

Is the registry somehow out of date with the adapters' current names?

hyfen commented 2 years ago

Thanks for reporting this! I moved this issue to the main chronicle-etl issue because it's a problem in the way plugins/dependencies are loaded. Someone else hit this issue a few days ago too but the error handling was swallowing up too much information to give a provide a useful error.

I just released a version of chronicle-etl (v0.4.3) that does a slightly better job of handling exceptions and messaging when plugins fail to load. Can you give it a shot and let me know if you're able to get it working? Running with --verbose will give the full stack trace: chronicle-etl --extractor imessage --since "2022-02-07" --transformer imessage --verbose

Also, what shows up when you run $ chronicle-etl plugins:list?

wongjustin99 commented 2 years ago

No prob!

Looks like installing the plugin fails now too, after updating the gem.

/Users/user/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/registry/plugin_registry.rb:43:in `rescue in activate': Plugin 'imessage' couldn't be loaded (Chronicle::ETL::PluginLoadError)

Plugins list:

~/dotfiles on master !2 ?1 ❯ chronicle-etl plugins:list                                                                                                                                                                          took 5s at 01:22:05
Installed plugins:
  NAME     DESCRIPTION                                    LATEST VERSION
  imessage Connectors for iMessage                        0.2.2
  shell    Extract shell command history from Bash or Zsh 0.2.0

When trying to run the extractor:

~/dotfiles on master !2 ?1 ❯  chronicle-etl --extractor imessage --since "2022-02-07" --transformer imessage --verbose                                                                                                                   at 01:24:03
Plugin specified by job not installed.
Do you want to install chronicle-imessage and start the job? Yesn) Y
[✖] Installing imessage... Error
/Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/registry/plugin_registry.rb:43:in `rescue in activate': Plugin 'imessage' couldn't be loaded (Chronicle::ETL::PluginLoadError)
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/registry/plugin_registry.rb:35:in `activate'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/registry/plugin_registry.rb:56:in `install'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/plugins.rb:22:in `block in install'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/plugins.rb:20:in `each'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/plugins.rb:20:in `install'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/jobs.rb:127:in `install_missing_plugins'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/jobs.rb:55:in `start'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
        from /Users/myuser/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-hollaback-0.2.1/lib/thor/hollaback.rb:68:in `run'

hyfen commented 2 years ago

Ahh, in fixing the previous issue, I forgot to remove a require 'pry' in the imessage plugin. I think your stack trace above is missing the last few dozen lines right? Was the original error about.

I released a fixed version. You can run: gem install chronicle-imessage -v 0.2.3. Let me know if that takes care of the issue.

A few takeaways:

I need to distinguish between plugins that aren't installed and those that can't be loaded successfully
I need to set up tests/CI for the plugins 🤣

wongjustin99 commented 2 years ago

Thank you!

Progress now. I am able to get a job running.

Running into a slightly different error now - though this one may be specific to imessage?

I wonder if I to let it access some permission ?

Beginning job
/Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-imessage-0.2.3/lib/chronicle/imessage/imessage_extractor.rb:61:in `load_my_phone_contact': undefined method `fetch' for nil:NilClass (NoMethodError)

          phone_number: @config.my_phone_number || local_contacts.my_phone_contact.fetch(:phone_number),
                                                                                  ^^^^^^
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-imessage-0.2.3/lib/chronicle/imessage/imessage_extractor.rb:51:in `prepare_data'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-imessage-0.2.3/lib/chronicle/imessage/imessage_extractor.rb:22:in `prepare'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/runner.rb:34:in `prepare_job'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/runner.rb:13:in `run!'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/jobs.rb:114:in `run_job'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/jobs.rb:58:in `start'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-hollaback-0.2.1/lib/thor/hollaback.rb:68:in `run'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:116:in `invoke'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:243:in `block in subcommand'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-hollaback-0.2.1/lib/thor/hollaback.rb:71:in `block in run'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/hollaback-0.1.1/lib/hollaback/sequence.rb:34:in `call'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-hollaback-0.2.1/lib/thor/hollaback.rb:71:in `run'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/lib/chronicle/etl/cli/main.rb:35:in `start'
        from /Users/goose/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/chronicle-etl-0.4.3/exe/chronicle-etl:5:in `<top (required)>'
        from /Users/goose/.rbenv/versions/3.1.1/bin/chronicle-etl:25:in `load'
        from /Users/goose/.rbenv/versions/3.1.1/bin/chronicle-etl:25:in `<main>'

hyfen commented 2 years ago

Great, the plugin is loading successfully and this is now just a normal, boring bug! I created a new issue: https://github.com/chronicle-app/chronicle-imessage/issues/3 and should be pretty straightforward to fix.

chronicle-app / chronicle-etl

Installing on fresh installation of chronicle-app produces error #33