Local plugin is not loaded from local path

[Lyneals]

Describe the bug

Plugin is not loaded from the local path using the following connection statement:

connection "eol" {
     plugin = "local/steampipe-plugin-eol"
}

Plugins directory tree:

plugins
├── local
│   └── steampipe-plugin-eol
│       └── steampipe-plugin-eol.plugin
└── versions.json

Steampipe plugin list output:

+-----------+---------+-------------+
| Installed | Version | Connections |
+-----------+---------+-------------+
|           |         |             |
+-----------+---------+-------------+

+------------------------------------------------------------+-------------+---------------+
| Failed                                                     | Connections | Reason        |
+------------------------------------------------------------+-------------+---------------+
| hub.steampipe.io/plugins/local/steampipe-plugin-eol@latest | eol         | Not installed |
+------------------------------------------------------------+-------------+---------------+

Logs trace (grep eol):

2024-03-15 07:07:01.197 UTC [INFO]  steampipe [46da]: plugin 'local/steampipe-plugin-eol' is not installed
2024-03-15 07:07:01.197 UTC [INFO]  steampipe [46da]: connection 'eol' requires plugin 'local/steampipe-plugin-eol' which is not loaded and has no instance config
2024-03-15 07:07:01.759 UTC [INFO]  plugin 'local/steampipe-plugin-eol' is not installed
2024-03-15 07:07:01.759 UTC [INFO]  connection 'eol' requires plugin 'local/steampipe-plugin-eol' which is not loaded and has no instance config
2024-03-15 07:07:01.759 UTC [TRACE] loaded config map: eol
2024-03-15 07:07:01.894 UTC [TRACE] setting user search path to [public eol steampipe_internal]
2024-03-15 07:07:01.895 UTC [TRACE] user search path sql: [LOCK TABLE pg_user IN SHARE ROW EXCLUSIVE MODE; ALTER USER "steampipe" SET SEARCH_PATH TO "public","eol","steampipe_internal";]
2024-03-15 07:07:01.896 UTC [TRACE] connection eol is in error - marking for deletion
    Connection state: eol
        1 plugin required by 1 connection is missing. To install, please run: steampipe plugin install local/steampipe-plugin-eol
2024-03-15 07:07:01.931 UTC [TRACE] steampipe [46da]: could not find schema in state map or connection is not Ready eol

Steampipe version (steampipe -v)

steampipe -v
Steampipe v0.22.0

To reproduce

Follow https://steampipe.io/docs/develop/writing-plugins#installing-and-testing-your-plugin to install a plugin locally.

Expected behavior Plugins prefixed by local/ should be loaded as local plugins

Additional context

It was working fine on an older laptop. Source if needed: https://github.com/Lyneals/steampipe-plugin-eol

[kaidaguerre]

Hi @Lyneals , sorry you've hit this issue.

I'll look into that and see what I can do

[Remy-Mollandin-SK5]

Hello @kaidaguerre , thanks for the response (same guy, different account here).

I could make it work by adding my local plugin to the versions.json file:

{
  "plugins": {
    "local/steampipe-plugin-eol": {
      "name": "local/steampipe-plugin-eol",
      "version": "0.1.0",
      "struct_version": 20230502
    }
  },
  "struct_version": 20220411
}

I believe it's related to recent changes that check the versions before loading a plugin: bb26c0ee810ec4f7239a08754dc0790d2978837d

    pluginVersion, ok := c.PluginVersions[imageRef]
    if !ok {
        // just log it
        log.Printf("[WARN] addPlugin called for plugin '%s' which is not installed", imageRef)
        return nil
    }

Either we can expect local development to maintain their versions.json file or we need to bypass this test for local plugins.

edit:

steampipe plugin list
+----------------------------+---------+-------------+
| Installed                  | Version | Connections |
+----------------------------+---------+-------------+
| local/steampipe-plugin-eol | 0.1.0   | eol         |
+----------------------------+---------+-------------+

[gerrymiller]

I'm still having this issue with Steampipe v0.22.1:

╰─$ steampipe -v         
Steampipe v0.22.1

I have a local plugin loaded in ~/.steampipe/

plugins
├── local
│   └── steampipe-plugin-teamwork
│       └── steampipe-plugin-teamwork.plugin
├── hub.steampipe.io
│   └── plugins
│       └── turbot
│           └── aws@latest
│               └── docs
│               └── steampipe-plugin-aws.plugin
│               └── version.json
│           └── steampipe@latest
│               └── docs
│               └── steampipe-plugin-steampipe.plugin
│               └── version.json
└── versions.json

However, when I execute steampipe plugin list I still get

+--------------------------------------------------+---------+-------------+
| Installed                                        | Version | Connections |
+--------------------------------------------------+---------+-------------+
| hub.steampipe.io/plugins/turbot/aws@latest       | 0.133.0 | aws         |
| hub.steampipe.io/plugins/turbot/steampipe@latest | 0.10.0  | steampipe   |
+--------------------------------------------------+---------+-------------+

+------------------------------------------------+-------------+---------------+
| Failed                                         | Connections | Reason        |
+------------------------------------------------+-------------+---------------+
| hub.steampipe.io/plugins/local/teamwork@latest | teamwork    | Not installed |
+------------------------------------------------+-------------+---------------+

I tried adding a local version.json to my ~/.steampipe/plugins/local/steampipe-plugin-teamwork directory, but no joy:

{
    "name": "local/steampipe-plugin-teamwork@latest",
    "version": "0.1.0",
    "struct_version": 20230502
}

When I try to add a section to ~/steampipe/plugins/versions.json, the section disappears every time I run steampipe plugin list:

Before

{
  "plugins": {
    "hub.steampipe.io/plugins/turbot/aws@latest": {
      "name": "hub.steampipe.io/plugins/turbot/aws@latest",
      "version": "0.133.0",
      "image_digest": "sha256:707173ae5c7bc57daee5b5355b3189be0d6764c859ba633e0a7d802e415f1610",
      "binary_digest": "sha256:db13f495a3ac712ff0033cd0e485cf3bbdaab7f3689ec928f5359a50543041cb",
      "binary_arch": "arm64",
      "installed_from": "us-docker.pkg.dev/steampipe/plugins/turbot/aws:latest",
      "last_checked_date": "2024-03-28T21:43:20Z",
      "install_date": "2024-03-16T18:04:35Z",
      "struct_version": 20230502
    },
    "hub.steampipe.io/plugins/turbot/steampipe@latest": {
      "name": "hub.steampipe.io/plugins/turbot/steampipe@latest",
      "version": "0.10.0",
      "image_digest": "sha256:dc2cf4b807c94bad49564845967b739ef6bbcc0a4ff54f0341adfa10b85cddff",
      "binary_digest": "sha256:d695f13d7d958781f111f2e4b6a677821a239432b2c8dd446049d384b39a7b6f",
      "binary_arch": "arm64",
      "installed_from": "us-docker.pkg.dev/steampipe/plugins/turbot/steampipe:latest",
      "last_checked_date": "2024-03-28T21:43:20Z",
      "install_date": "2023-12-29T01:46:48Z",
      "struct_version": 20230502
    },
    "local/steampipe-plugin-teamwork@latest": {
      "name": "local/steampipe-plugin-teamwork@latest",
      "version": "0.1.0",
      "struct_version": 20230502
    }
  },
  "struct_version": 20220411
}

After

{
  "plugins": {
    "hub.steampipe.io/plugins/turbot/aws@latest": {
      "name": "hub.steampipe.io/plugins/turbot/aws@latest",
      "version": "0.133.0",
      "image_digest": "sha256:707173ae5c7bc57daee5b5355b3189be0d6764c859ba633e0a7d802e415f1610",
      "binary_digest": "sha256:db13f495a3ac712ff0033cd0e485cf3bbdaab7f3689ec928f5359a50543041cb",
      "binary_arch": "arm64",
      "installed_from": "us-docker.pkg.dev/steampipe/plugins/turbot/aws:latest",
      "last_checked_date": "2024-03-28T21:43:20Z",
      "install_date": "2024-03-16T18:04:35Z",
      "struct_version": 20230502
    },
    "hub.steampipe.io/plugins/turbot/steampipe@latest": {
      "name": "hub.steampipe.io/plugins/turbot/steampipe@latest",
      "version": "0.10.0",
      "image_digest": "sha256:dc2cf4b807c94bad49564845967b739ef6bbcc0a4ff54f0341adfa10b85cddff",
      "binary_digest": "sha256:d695f13d7d958781f111f2e4b6a677821a239432b2c8dd446049d384b39a7b6f",
      "binary_arch": "arm64",
      "installed_from": "us-docker.pkg.dev/steampipe/plugins/turbot/steampipe:latest",
      "last_checked_date": "2024-03-28T21:43:20Z",
      "install_date": "2023-12-29T01:46:48Z",
      "struct_version": 20230502
    }
  },
  "struct_version": 20220411
}

[ramirezj]

From what I can see, @gerrymiller, its not made it into a release yet: https://github.com/turbot/steampipe/compare/v0.22.1...main

I know Jon was doing some subsequent testing

[kaidaguerre]

sorry for the delay in this one, found another small issue with the fix, hoping the get a patch out tomorrow.

[gerrymiller]

Thanks. I have v0.22.2, and it's still not working. Do you know if the patch made it into that version? I've updated Steampipe to the latest available.

[gerrymiller]

Hi @kaidaguerre wondering if you had an update on this...thank you!