search

apache / openwhisk-composer

Apache OpenWhisk Composer provides a high-level programming model in JavaScript for composing serverless functions
https://openwhisk.apache.org/
Apache License 2.0
68 stars 21 forks source link

Cannot find module 'redis-commands' #85

Closed GitHubDiom closed 1 year ago

GitHubDiom commented 2 years ago

Hi, community

The error occurs when I am trying to run a set of parallel actions.

{ "error": "Internal error: Cannot find module 'redis-commands'\nRequire stack:\n- /nodejsAction/runner.js\n- /nodejsAction/src/service.js\n- /nodejsAction/app.js" }

Here is my basic env info, the /guest/image itself can be run normally

$ wsk action list
actions
/guest/concur-images                                                   private nodejs:14
/guest/image                                                           private python:3

$ node -v
v10.24.0

I also wrote some scripts for building and invoking the parallel actions as below: concur-images.js

const composer = require('openwhisk-composer');

const workflow = 
    composer.par(
        composer.action("image"),
        composer.action("image"),
        composer.action("image"),
)
module.exports = workflow

build_concur_images.sh

docker stop redis
docker rm redis 

function_name="concur-images"

redis_container_id=$(docker run -itd --name redis -p 6379:6379 redis)

redis_container_ip=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' $redis_container_id)
echo "IP = $redis_container_ip"
echo '{
    "$composer": {
        "openwhisk": {
            "ignore_certs": true
        },
        "redis": {
            "uri": "redis://'${redis_container_ip}':6379"
        }
    }
}' > ${function_name}-input.json

compose $function_name.js > $function_name.json
deploy $function_name $function_name.json -w

invoke-concur-image.sh

wsk action invoke -r /guest/concur-images -P concur-images-input.json

And I just run ./invoke-concur_image.sh after ./build_concur_images.sh, then the error happends.


$ ./build_concur_images.sh 
redis
redis
IP = 172.17.0.2
ok: created action /_/concur-images
$ ./invoke-concur_image.sh 
{
    "error": "Internal error: Cannot find module 'redis-commands'\nRequire stack:\n- /nodejsAction/runner.js\n- /nodejsAction/src/service.js\n- /nodejsAction/app.js"
}

Thank you

HongyuHe commented 1 year ago

Hi @GitHubDiom, have you solved the issue by any chance? I encountered the same problem.

GitHubDiom commented 1 year ago

yeah, changing nodejs image version (7f7c181) solved this issue { "kind": "nodejs:14", "default": true, "image": { "prefix": "openwhisk", "name": "action-nodejs-v14", "tag": "7f7c181" }, ... }

HongyuHe commented 1 year ago

yeah, changing nodejs image version (7f7c181) solved this issue { "kind": "nodejs:14", "default": true, "image": { "prefix": "openwhisk", "name": "action-nodejs-v14", "tag": "7f7c181" }, ... }

Hi @GitHubDiom, thanks a lot for the suggestion! I've changed the nodejs runtime to 7f7c181:

Thursday 16 February 2023  12:32:00 -0700 (0:00:00.044)       0:01:22.147 *****
changed: [invoker1] => (item={'prefix': 'openwhisk', 'name': 'action-nodejs-v14', 'tag': '7f7c181'})
changed: [invoker0] => (item={'prefix': 'openwhisk', 'name': 'action-nodejs-v14', 'tag': '7f7c181'})
changed: [invoker0] => (item={'prefix': 'openwhisk', 'name': 'action-nodejs-v16', 'tag': 'nightly'})
changed: [invoker1] => (item={'prefix': 'openwhisk', 'name': 'action-nodejs-v16', 'tag': 'nightly'})
changed: [invoker0] => (item={'prefix': 'openwhisk', 'name': 'action-python-v3.7', 'tag': 'nightly'})
changed: [invoker1] => (item={'prefix': 'openwhisk', 'name': 'action-python-v3.7', 'tag': 'nightly'})
changed: [invoker0] => (item={'prefix': 'openwhisk', 'name': 'action-swift-v5.1', 'tag': 'nightly'})

However, it gives me the following error now:

$ wsk -i action invoke parallel -P parallel_params.json --result
{
    "error": "Internal error: DEL can't be processed. The connection is already closed."
}

Logs:

Activation: 'parallel' (4576b5566aac4a47b6b5566aacaa47e9)
[
    "2023-02-16T19:52:57.808228566Z stdout: Entering composition",
    "2023-02-16T19:52:57.808314957Z stdout: barrierId: 514a3457-15e1-494d-a108-6eabe88c20d2, spawning: 1",
    "2023-02-16T19:52:57.809953984Z stderr: AbortError: LPUSH can't be processed. The connection is already closed.",
    "2023-02-16T19:52:57.809979327Z stderr:     at handle_offline_command (/node_modules/redis/index.js:851:15)",
    "2023-02-16T19:52:57.809988773Z stderr:     at RedisClient.internal_send_command (/node_modules/redis/index.js:885:9)",
    "2023-02-16T19:52:57.809996803Z stderr:     at RedisClient.lpush (/node_modules/redis/lib/commands.js:58:25)",
    "2023-02-16T19:52:57.810004616Z stderr:     at eval (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:22:12)",
    "2023-02-16T19:52:57.810012999Z stderr:     at new Promise (<anonymous>)",
    "2023-02-16T19:52:57.810020612Z stderr:     at RedisClient.t.<computed> [as lpushAsync] (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:21:86)",
    "2023-02-16T19:52:57.810028855Z stderr:     at m (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:30:39)",
    "2023-02-16T19:52:57.810036515Z stderr:     at Object.parallel (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:75:105)",
    "2023-02-16T19:52:57.81004431Z  stderr:     at $ (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:94:78)",
    "2023-02-16T19:52:57.810052458Z stderr:     at eval (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:99:59) {",
    "2023-02-16T19:52:57.810060251Z stderr:   command: 'LPUSH',",
    "2023-02-16T19:52:57.810067251Z stderr:   code: 'NR_CLOSED',",
    "2023-02-16T19:52:57.810074299Z stderr:   args: [ 'composer/fork/514a3457-15e1-494d-a108-6eabe88c20d2', 42 ]",
    "2023-02-16T19:52:57.810081747Z stderr: }"
]

Do you know by any chance how to resolve it?

Abhishekghosh1998 commented 1 year ago

I too faced the same issue.

{
"error": "Internal error: Cannot find module 'redis-commands'\nRequire stack:\n- /nodejsAction/runner.js\n- /nodejsAction/src/service.js\n- /nodejsAction/app.js"
}

The problem is probably due to the absence of the redis-commands module in the nodejs:14 image with tag 1.20.0 in the runtime.json file in the openwhisk-deploy-kube:

https://github.com/apache/openwhisk-deploy-kube/blob/e202f469181716ba15e0676245680057d294d951/helm/openwhisk/runtimes.json#L5-11

             ...
             {
                "kind": "nodejs:14",
                "default": true,
                "image": {
                    "prefix": "openwhisk",
                    "name": "action-nodejs-v14",
                    "tag": "1.20.0"
                },
                ....

To get around this, as mentioned by @GitHubDiom I used the nodejs:14 image with tag 7f7c181 

             ...
             {
                "kind": "nodejs:14",
                "default": true,
                "image": {
                    "prefix": "openwhisk",
                    "name": "action-nodejs-v14",
                    "tag": "7f7c181"
                },
                ....

Using the above image got rid of the redis-commands module not found error, but it caused another issue:

"result": {
            "error": "Internal error: DEL can't be processed. The connection is already closed."
        },

Digging into the logs:

"logs": [                                                                                                                                                                                                                           
        "2023-10-23T17:26:31.864809433Z stdout: Entering composition",                                                                                                                                                                  
        "2023-10-23T17:26:31.864852233Z stdout: barrierId: 38701345-55a1-4c0b-bf9b-1071374bbd2a, spawning: 2",                                                                                                                          
        "2023-10-23T17:26:31.865529558Z stdout: AbortError: LPUSH can't be processed. The connection is already closed.",                                     
        "2023-10-23T17:26:31.865540192Z stdout: at handle_offline_command (/node_modules/redis/index.js:851:15)",                                             
        "2023-10-23T17:26:31.865555121Z stdout: at RedisClient.internal_send_command (/node_modules/redis/index.js:885:9)",                                   
        "2023-10-23T17:26:31.865559223Z stdout: at RedisClient.lpush (/node_modules/redis/lib/commands.js:58:25)",                                            
        "2023-10-23T17:26:31.865563019Z stdout: at eval (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:27:12)",                                                                                          
        "2023-10-23T17:26:31.865567206Z stdout: at new Promise (<anonymous>)",                                                 
        "2023-10-23T17:26:31.865570984Z stdout: at RedisClient.t.<computed> [as lpushAsync] (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:26:86)",
        "2023-10-23T17:26:31.865580893Z stdout: at m (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:35:39)",                                                                                             
        "2023-10-23T17:26:31.865585186Z stdout: at Object.parallel (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:80:105)",
        "2023-10-23T17:26:31.865588915Z stdout: at $ (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:99:78)",                                                                                             
        "2023-10-23T17:26:31.865592807Z stdout: at eval (eval at initializeActionHandler (/nodejsAction/runner.js:56:23), <anonymous>:104:59) {",             
        "2023-10-23T17:26:31.865597794Z stdout: command: 'LPUSH',",                                                            
        "2023-10-23T17:26:31.865601253Z stdout: code: 'NR_CLOSED',",                                                                                          
        "2023-10-23T17:26:31.865604729Z stdout: args: [ 'composer/fork/38701345-55a1-4c0b-bf9b-1071374bbd2a', 42 ]",                                          
        "2023-10-23T17:26:31.865608309Z stdout: }"                                                                                                            
    ],

It turns out that the above error arises if Openwhisk-Composer cannot access Redis. #74

I tried to install Redis (without password) on my Kubernetes pod as follows:

$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo update
$ helm install my-redis bitnami/redis --set auth.enabled=false
$ kubectl get svc --all-namespaces
NAMESPACE     NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
default       kubernetes          ClusterIP   10.96.0.1        <none>        443/TCP                      4d4h
default       my-redis-headless   ClusterIP   None             <none>        6379/TCP                     6h32m
default       my-redis-master     ClusterIP   10.104.247.215   <none>        6379/TCP                     6h32m
default       my-redis-replicas   ClusterIP   10.101.110.216   <none>        6379/TCP                     6h32m
kube-system   kube-dns            ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP       4d4h
openwhisk     owdev-apigateway    ClusterIP   10.102.148.42    <none>        8080/TCP,9000/TCP            2m51s
openwhisk     owdev-controller    ClusterIP   10.104.205.202   <none>        8080/TCP                     2m51s
openwhisk     owdev-couchdb       ClusterIP   10.106.166.250   <none>        5984/TCP                     2m51s
openwhisk     owdev-kafka         ClusterIP   None             <none>        9092/TCP                     2m51s
openwhisk     owdev-nginx         NodePort    10.106.115.191   <none>        80:31031/TCP,443:31001/TCP   2m51s
openwhisk     owdev-redis         ClusterIP   10.98.128.197    <none>        6379/TCP                     2m51s
openwhisk     owdev-zookeeper     ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   2m51s

Note that now my system has two redis pods:

  1. my-redis: installed by me
  2. owdev-redis: installed as part of OpenWhisk itself.

my-redis-parameter.json:

{
  "$composer": {
      "openwhisk": {
          "ignore_certs": true
      },
      "redis": {
          "uri": "redis://my-redis-master.default.svc.cluster.local:6379"
      }
  }
}

owdev-redis-parameter.json:

{
    "$composer": {
        "redis": {
            "uri": "redis://owdev-redis.openwhisk.svc.cluster.local:6379"
        },
    "openwhisk": {
            "ignore_certs": true
        }
    }
}

Then we can use either of the Redis setup for our parallel composition:

# creation of the compostion
$ compose parallel.js > parallel.json
$ deploy -i parallel parallel.json -w

Invocation using my-redis

wsk -i action invoke parallel --result --param-file my-redis-parameter.json

Invocation using owdev-redis:

wsk -i action invoke parallel --result --param-file owdev-redis-parameter.json