Update connector to new version

[stvdo]

Good afternoon,

I use i2 Analyze version 4.4.2 with the docker container version 2.6.0. I had 2 working connectors, with version 0.1.0. I make some changes and changed the version to 1.0.0.

Finally it seems the the dist folder was missing in the connector-images/connector/app. I copied it manual, now it works. Normally the dist folder should be included in the tgz file I asume and copied automatically into the app/dist folder of the connector.

1. I created 2 packages anpr-connector-1.0.0.tgz and car-registration-connector-1.0.0.tgz and uploaded those to the connector- images folder while the config was not deployed yet (today).
2. I changed the version and tag in the connector-version.json from 0.1.0 to 1.0.0 and deployed the config.
3. I got the error that only 1 tgz file was allowed, so I removed the old version and deployed the config again.
4. I received the error below and ran the command 'manage-environment -t connectors'

   ----------------------------------------------------------------------

   Waiting for Connector to be live on https://car-registration-connector-1-0-0.eia:3443/config

   ----------------------------------------------------------------------

   curl: (7) Failed to connect to car-registration-connector-1-0-0.eia port 3443: Connection refused Could not connect to https://car-registration-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 1). Waiting... curl: (60) SSL: no alternative certificate subject name matches target host name 'car-registration-connector-1-0-0.eia' More details here: https://curl.se/docs/sslcerts.html

5. After this I still had the error, so I did a clean deploy of the config.

   ----------------------------------------------------------------------

   Waiting for Connector to be live on https://anpr-connector-1-0-0.eia:3443/config

   ----------------------------------------------------------------------

   curl: (7) Failed to connect to anpr-connector-1-0-0.eia port 3443: Connection refused Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 1). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 2). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 3). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 4). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 5). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 6). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 7). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 8). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 9). Waiting... curl: (6) Could not resolve host: anpr-connector-1-0-0.eia Could not connect to https://anpr-connector-1-0-0.eia:3443/config Connector is NOT live (attempt: 10). Waiting... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Could not resolve host: anpr-connector-1-0-0.eia

6. In the logs of the anpr connector I saw this error. Error: Cannot find module './data/cars.json' Require stack:
   • /opt/app-root/src/dist/data-service.js
   • /opt/app-root/src/dist/index.js at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1028:15) at Function.Module._load (node:internal/modules/cjs/loader:873:27) at Module.require (node:internal/modules/cjs/loader:1100:19) at require (node:internal/modules/cjs/helpers:108:18) at Object. (/opt/app-root/src/dist/data-service.js:6:14) at Module._compile (node:internal/modules/cjs/loader:1198:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10) at Module.load (node:internal/modules/cjs/loader:1076:32) at Function.Module._load (node:internal/modules/cjs/loader:911:12) at Module.require (node:internal/modules/cjs/loader:1100:19) { code: 'MODULE_NOT_FOUND', requireStack: [ '/opt/app-root/src/dist/data-service.js', '/opt/app-root/src/dist/index.js' ] }
     • Terminal will be reused by tasks, press any key to close it.

[Anthony-Johnson-i2]

Hi @Jonny-Cornwell-i2

Can you have a look at Steven's steps here and see if anything stands out as to why the dist folder was missing when he updated his connector versions and why he had to manually copy it?

Cheers

[stvdo]

One other question. I tried to deploy it on our demo environment in the cloud. This is running on container version 2.7.0.

But here I get permission errors.

1. I created 2 packages anpr-connector-1.0.0.tgz and car-registration-connector-1.0.0.tgz and uploaded those to the connector- images folder (with FileZilla via SSH) while the config was not deployed yet (today).
2. I connected via VSCode to the container (via SSH)
3. I changed the version and tag in the connector-version.json from 0.1.0 to 1.0.0 and deployed the config. Than I got this error.

I tried it with 'sudo deploy -c {config name}' but this doesn't help. Any tips for this?

----------------------------------------------------------------------

Building connectors

----------------------------------------------------------------------

tar: dist/schema/law-enforcement-schema.js: Cannot open: Permission denied tar: dist/data/account_organisation.json: Cannot open: Permission denied tar: dist/data/account_person.json: Cannot open: Permission denied tar: dist/data/address - Copy.json: Cannot open: Permission denied tar: dist/data/address.json: Cannot open: Permission denied tar: dist/data/cars.json: Cannot open: Permission denied tar: dist/data/persons_addresses.json: Cannot open: Permission denied tar: dist/data/persons_cars.json: Cannot open: Permission denied tar: dist/data/persons.json: Cannot open: Permission denied tar: dist/data/transaction_link.json: Cannot open: Permission denied tar: dist/schema/law-enforcement-schema.js.map: Cannot open: Permission denied tar: dist/schema/law-enforcement-schema-charting-schemes.xml: Cannot open: Permission denied tar: dist/schema/law-enforcement-schema.xml: Cannot open: Permission denied tar: Exiting with failure status due to previous errors

[Jonny-Cornwell-i2]

Hi,

Sorry you are having issues! With a focus on the anpr connector first it looks like it is failing to load the data/cars.json file. Indicating that possible the path is wrong or it has not been included with the tgz. Is there any chance providing a list of files in the tgz by running npm pack --dry-run? This will list the files that would go into the tgz without generating the tgz. e.g.

npm notice 
npm notice 📦  dist-test@0.1.0
npm notice === Tarball Contents === 
npm notice 1.1kB dist/index.js        
npm notice 821B  dist/schema.js       
npm notice 23B   data/outside-src.json
npm notice 518B  package.json         
npm notice 19B   dist/sample.json     
npm notice 421B  config/server.json   
npm notice 4B    config/settings.json 
npm notice 825B  dist/index.js.map    
npm notice 585B  dist/schema.js.map   
npm notice 2.3kB README.md            
npm notice 524B  .env.sample          
npm notice 4B    dist/sample.txt      
npm notice 2.5kB dist/schema.xml      
npm notice === Tarball Details === 
npm notice name:          dist-test                               
npm notice version:       0.1.0                                   
npm notice filename:      dist-test-0.1.0.tgz                     
npm notice package size:  4.0 kB                                  
npm notice unpacked size: 9.6 kB                                  
npm notice shasum:        95bc4e8375c56aae51c573567534187ed1bf3076
npm notice integrity:     sha512-ENKFhj7Db3qDa[...]rP8uqwcK2MiXQ==
npm notice total files:   13                                      
npm notice 

Also as you alluded to you, should not need to copy the dist folder manually at all assuming you have executed npm run build on the connector followed by an npm pack to generate the tgz.

Cheers

[stvdo]

Morning Jonny,

Thanks for your reply!

The cars.json is included in the output. So that shouldn't be the problem. I'll keep an eye on it next time and try to compare all the files before copying this manually.

[stvdo]

About the other issue on our demo environment. Is it possible to unzip the tar with sudo to prevent the errors below or is there another way to fix this?

----------------------------------------------------------------------

Building connectors

----------------------------------------------------------------------

tar: dist/schema-charting-schemes.xml.bak: Cannot open: File exists tar: dist/schema.xml.bak: Cannot open: File exists tar: dist/data-service.js: Cannot open: File exists tar: dist/index.js: Cannot open: File exists tar: dist/schema/law-enforcement-schema.js: Cannot open: Permission denied tar: dist/result-building.js: Cannot open: File exists tar: dist/schema.js: Cannot open: File exists tar: dist/data/account_organisation.json: Cannot open: Permission denied tar: dist/data/account_person.json: Cannot open: Permission denied tar: dist/data/address - Copy.json: Cannot open: Permission denied tar: dist/data/address.json: Cannot open: Permission denied tar: dist/data/cars.json: Cannot open: Permission denied tar: package.json: Cannot open: File exists tar: dist/data/persons_addresses.json: Cannot open: Permission denied tar: dist/data/persons_cars.json: Cannot open: Permission denied tar: dist/data/persons.json: Cannot open: Permission denied tar: config/server.json: Cannot open: File exists tar: config/settings.json: Cannot open: File exists tar: dist/data/transaction_link.json: Cannot open: Permission denied tar: dist/data-service.js.map: Cannot open: File exists tar: dist/index.js.map: Cannot open: File exists tar: dist/schema/law-enforcement-schema.js.map: Cannot open: Permission denied tar: dist/result-building.js.map: Cannot open: File exists tar: dist/schema.js.map: Cannot open: File exists tar: README.md: Cannot open: File exists tar: .env.sample: Cannot open: File exists tar: dist/schema/law-enforcement-schema-charting-schemes.xml: Cannot open: Permission denied tar: dist/schema/law-enforcement-schema.xml: Cannot open: Permission denied tar: dist/schema-charting-schemes.xml: Cannot open: File exists tar: dist/schema.xml: Cannot open: File exists tar: Exiting with failure status due to previous errors vscode ➜ /analyze-containers-270 $ sudo deploy -c config-demo sudo: deploy: command not found vscode ➜ /analyze-containers-270 $

[stvdo]

Good morning @Jonny-Cornwell-i2 ,

This morning I changed the .devcontainer/devcontainer.json and commented out the remoteUser. I had hoped that this would solve the permission errors, but than I get the message that the command is not found.

root ➜ /analyze-containers-270 $ deploy -c config-demo bash: deploy: command not found

Locally I don't have those permission issues, only on our demo environment. Local: Ubuntu 22.04.2 LTS, i2 Analyze 4.4.2, analyze-containers 2.6.0 Demo environment: Ubuntu 22.10, i2 Analyze 4.4.2, analyze-containers 2.7.0

[Anthony-Johnson-i2]

Hi Steven @sdorresteijn88

It is probably better to ask your container permissions related questions over on analyze-containers as the folks looking at that will be more familiar with this area.

Cheers

[Jonny-Cornwell-i2]

Morning Jonny,

Thanks for your reply!

The cars.json is included in the output. So that shouldn't be the problem. I'll keep an eye on it next time and try to compare all the files before copying this manually.

Good stuff, looks like it has mostly got what I would expect in the tgz, there could be an issue with the path causing this error Error: Cannot find module './data/cars.json'. In the data-service.ts module are you able to provide the code to how you are determining the path for the cars.json file?

[stvdo]

Now I use "import * as cars from './data/cars.json';" in the data-service.ts. So it seems to be the right path.