583: access services in local network

[BeckerFrank]

See: https://github.com/eclipse-mylyn/org.eclipse.mylyn/issues/583

I coorect the setup for the bridged Network and did a test with an other computer in my local network. As a rsult I also have updated the README.md and made some changes in other places.

Please report problems and Questions in this pull Request.

[BeckerFrank]

@gnl42: I found some small things that should be changed, but this version is working.

I don't think we should try to set up two domains, just one that works better with certificates. I will make the changes and update the pull request.

[gnl42]

Thanks for doing the work.

I'm trying it out now and will let you know if I find anything

[gnl42]

Hi @BeckerFrank

it keeps failing for me with:

TASK [multipass_create_vm_setup : add etc hosts if not included "mylynadm01"] ***
task path: /Users/George/ci/org.eclipse.mylyn/org.eclipse.mylyn.releng/multipass/roles/multipass_create_vm_setup/tasks/test_and_update_remote_host_etc.yml:11
Saturday 20 July 2024  14:40:00 -0700 (0:00:02.531)       0:02:55.163 *********
Saturday 20 July 2024  14:40:00 -0700 (0:00:02.531)       0:02:55.163 *********
fatal: [localhost -> mylynadm01]: FAILED! => {
    "msg": "The task includes an option with an undefined variable. The error was: 'inst_network_ip' is undefined\n\nThe error appears to be in '/Users/George/ci/org.eclipse.mylyn/org.eclipse.mylyn.re
leng/multipass/roles/multipass_create_vm_setup/tasks/test_and_update_remote_host_etc.yml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending
line appears to be:\n\n\n- name: add etc hosts if not included \"{{ act_name.name }}\"\n  ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote tem
plate expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"
}

I'm doing:

 multipass delete --purge --all ; sudo cp /dev/null /var/db/dhcpd_leases && cp /dev/null ~/.ssh/known_hosts
 ansible-playbook create_vm.yml -vvv --ask-become-pass --extra-vars="docker_run_mode=container" | tee step3.log

step3.log

[BeckerFrank]

Hi @BeckerFrank

it keeps failing for me with:

TASK [multipass_create_vm_setup : add etc hosts if not included "mylynadm01"] ***
task path: /Users/George/ci/org.eclipse.mylyn/org.eclipse.mylyn.releng/multipass/roles/multipass_create_vm_setup/tasks/test_and_update_remote_host_etc.yml:11
Saturday 20 July 2024  14:40:00 -0700 (0:00:02.531)       0:02:55.163 *********
Saturday 20 July 2024  14:40:00 -0700 (0:00:02.531)       0:02:55.163 *********
fatal: [localhost -> mylynadm01]: FAILED! => {
    "msg": "The task includes an option with an undefined variable. The error was: 'inst_network_ip' is undefined\n\nThe error appears to be in '/Users/George/ci/org.eclipse.mylyn/org.eclipse.mylyn.re
leng/multipass/roles/multipass_create_vm_setup/tasks/test_and_update_remote_host_etc.yml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending
line appears to be:\n\n\n- name: add etc hosts if not included \"{{ act_name.name }}\"\n  ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote tem
plate expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"
}

I'm doing:

 multipass delete --purge --all ; sudo cp /dev/null /var/db/dhcpd_leases && cp /dev/null ~/.ssh/known_hosts
 ansible-playbook create_vm.yml -vvv --ask-become-pass --extra-vars="docker_run_mode=container" | tee step3.log

step3.log

@gnl42: Two Points.

1. the additional_launch_parms parameter in the inventory.yml assume that you do the installation on a Mac OS device. For other platforms you may have not en0 for the network connection to the DHCP server. Reason is that the multipass network command depends both on the platform and the driver in use.
2. In global_setup.yml there is a new variable network_ip_matcher with the default of 192.168.2.* addresses. If you have an other IP range change that line or add an extra playbook parameter local_network_ip_matcher with your values

[gnl42]

Yippee! It works :)

Couple of questions:

1. Why steps 3,4, and 5? In the past you've mentioned that you basically only do step 6.
2. Any reason for not use the network_ip_matcher values in /etc/hosts by default?
3. Could the regex expression for network_ip_matcher be derived from the ethernet interface used?

I still have to manually add the entries to '/etc/hosts' if they don't exist initially.

Thanks a lot. I want to upgrade the unit tests to JUnit 5 and now I have something I can work against and compare

[BeckerFrank]

Hi @gnl42 , I added corrections for your comments and test the whole setup with Multipass 1.14

I use the following commands

• ansible-playbook create_vm.yml --ask-become-pass --extra-vars="local_network_ip_matcher='^192.168.2..*'"
• ansible-playbook mylyn_setup.yml

[gnl42]

Thanks. I'll try out the changes and let you know

[gnl42]

One more change at least: The multipass launch -n mylynadm01 --disk 5G --memory 1G --cpus 1 --network br-eth0 --cloud-init first_setup/cloud-init.yaml" fails with launch failed: Requested disk (5368709120 bytes) below minimum for this image (10737418240 bytes)

============ I'm still having some bizarre IP issues under ubuntu. /etc/hosts gets update with some unusual entries and I have to run the ansible-playbook create_vm.yml twice in order for the VMs to be created. The first run fails because mp is trying to use a wrong IP address, /etc/hosts looks like:

192.168.64.3 mylyn-local traefik.mylyn.local mylyn-local.mylyn.local mylyn.local jenkins.mylyn.local hello.mylyn.local artifactory.mylyn.local gitlab.mylyn.local
192.168.64.3 mylyn-local traefik.mylyn.local mylyn-local.mylyn.local mylyn.local jenkins.mylyn.local hello.mylyn.local artifactory.mylyn.local gitlab.mylyn.local
192.168.64.4 mylynwrk01 mylynwrk01.mylyn.local
10.209.22.169 mylynadm01
192.168.0.240 mylynmstr01 mylyn.local traefik.mylyn.local jenkins.mylyn.local artifactory.mylyn.local gitlab.mylyn.local

The 10.209 is the mpbr0 network.

During the second run the IP addresses are corrected by the script and after some editing, my working /etc/hosts looks like:

#10.209.22.69 mylyn-local traefik.mylyn.local mylyn-local.mylyn.local mylyn.local jenkins.mylyn.local hello.mylyn.local artifactory.mylyn.local gitlab.mylyn.local
10.209.22.169 mylynadm01
192.168.0.240 mylynmstr01 mylyn.local traefik.mylyn.local jenkins.mylyn.local artifactory.mylyn.local gitlab.mylyn.local

============ Another issue I've run into with the tests is that the url from the test data ends with a '/', but the code is not expecting the slash so the tests blows up. I added code to handle the case, but I don't know who is wrong?

============ When I hit the https://mylyn.local/bugzilla-5_2-rest-api/ I get:

Software error:

The ./data/params.json file does not exist. You probably need to run checksetup.pl. at Bugzilla/Config.pm line 341.
Compilation failed in require at /var/www/html/index.cgi line 15.
BEGIN failed--compilation aborted at /var/www/html/index.cgi line 15.

============ When I run the unit tests as 'JUnit Plug-in Test' against mylyn.local they fail with:

org.eclipse.core.runtime.CoreException: Unexpected error: Timeout waiting for connection from pool at
org.eclipse.mylyn.internal.jenkins.core.JenkinsCorePlugin.toCoreException(JenkinsCorePlugin.java:69) at 
org.eclipse.mylyn.internal.jenkins.core.JenkinsServerBehaviour.getPlans(JenkinsServerBehaviour.java:272) at org.eclipse.mylyn.jenkins.tests.integration.JenkinsIntegrationTest.testPlanParameters(JenkinsIntegrationTest.java:63) at 

java.base/java.lang.reflect.Method.invoke(Method.java:568) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at 
java.base/java.util.ArrayList.forEach(ArrayList.java:1511) Caused by: org.eclipse.mylyn.internal.jenkins.core.client.JenkinsException: Timeout waiting for connection from pool at org.eclipse.mylyn.internal.jenkins.core.client.JenkinsOperation.run(JenkinsOperation.java:247) 
at org.eclipse.mylyn.internal.jenkins.core.client.RestfulJenkinsClient.getJobsFromFolder(RestfulJenkinsClient.java:341) at 

[BeckerFrank]

One more change at least: The multipass launch -n mylynadm01 --disk 5G --memory 1G --cpus 1 --network br-eth0 --cloud-init first_setup/cloud-init.yaml" fails with launch failed: Requested disk (5368709120 bytes) below minimum for this image (10737418240 bytes)

Strange on my Macbook I have

multipass info mylynadm01
Name:           mylynadm01
State:          Running
Snapshots:      0
IPv4:           192.168.64.2
Release:        Ubuntu 24.04 LTS
Image hash:     2c47dbf04477 (Ubuntu 24.04 LTS)
CPU(s):         1
Load:           0.00 0.00 0.00
Disk usage:     1.8GiB out of 4.8GiB
Memory usage:   219.9MiB out of 953.0MiB
Mounts:         --

OK I change this to 10G with the next commit

============ I'm still having some bizarre IP issues under ubuntu. /etc/hosts gets update with some unusual entries and I have to run the ansible-playbook create_vm.yml twice in order for the VMs to be created. The first run fails because mp is trying to use a wrong IP address, /etc/hosts looks like:

192.168.64.3 mylyn-local traefik.mylyn.local mylyn-local.mylyn.local mylyn.local jenkins.mylyn.local hello.mylyn.local artifactory.mylyn.local gitlab.mylyn.local
192.168.64.3 mylyn-local traefik.mylyn.local mylyn-local.mylyn.local mylyn.local jenkins.mylyn.local hello.mylyn.local artifactory.mylyn.local gitlab.mylyn.local
192.168.64.4 mylynwrk01 mylynwrk01.mylyn.local
10.209.22.169 mylynadm01
192.168.0.240 mylynmstr01 mylyn.local traefik.mylyn.local jenkins.mylyn.local artifactory.mylyn.local gitlab.mylyn.local

The 10.209 is the mpbr0 network.

During the second run the IP addresses are corrected by the script and after some editing, my working /etc/hosts looks like:

#10.209.22.69 mylyn-local traefik.mylyn.local mylyn-local.mylyn.local mylyn.local jenkins.mylyn.local hello.mylyn.local artifactory.mylyn.local gitlab.mylyn.local
10.209.22.169 mylynadm01
192.168.0.240 mylynmstr01 mylyn.local traefik.mylyn.local jenkins.mylyn.local artifactory.mylyn.local gitlab.mylyn.local

Can you please delete all mylyn-local, mylynmstr01, mylynadm01 and mylynwrk01 from /etc/hosts and recreate the environment?

============ Another issue I've run into with the tests is that the url from the test data ends with a '/', but the code is not expecting the slash so the tests blows up. I added code to handle the case, but I don't know who is wrong?

============ When I hit the https://mylyn.local/bugzilla-5_2-rest-api/ I get:

Software error:

The ./data/params.json file does not exist. You probably need to run checksetup.pl. at Bugzilla/Config.pm line 341.
Compilation failed in require at /var/www/html/index.cgi line 15.
BEGIN failed--compilation aborted at /var/www/html/index.cgi line 15.

Can you make sure that mylyn.setup has the following lines (work for me)

        bugzilla_instances:
          - { git_branch: "release-4.4.13", servmame: "bugzilla-4_4", port1: 8090, port2: 8490, image_tag: "4.4", extra_info: '"default": "true"' }
#          - { git_branch: "release-4.4.13", servmame: "bugzilla-4_4-custom-wf", port1: 8091, port2: 8491, image_tag: "4.4-custom_wf", extra_info: '"custom_wf":"true"' }
#          - { git_branch: "release-4.4.13", servmame: "bugzilla-4_4-custom-wf-and-status", port1: 8092, port2: 8492, image_tag: "4.4-custom_wf_a_s", extra_info: '"custom_wf_and_status":"true"' }
#          - { git_branch: "release-4.4.13", servmame: "bugzilla-4_4-bugaliases", port1: 8093, port2: 8493, image_tag: "4.4-bugaliases", extra_info: '"use_bug_alias":"true"' }
#          - { git_branch: "5.2", servmame: "bugzilla-5_2", port1: 8094, port2: 8494, image_tag: "5.2", extra_info: '' }
          - { git_branch: "5.2", servmame: "bugzilla-5_2-rest", port1: 8095, port2: 8495, image_tag: "5.2-rest", extra_info: '"rest_enabled":"true", "default": "true", "testdataVersion": "Version1"' }
          - { git_branch: "5.2", servmame: "bugzilla-5_2-rest-api", port1: 8096, port2: 8496, image_tag: "5.2-rest-api", extra_info: '"rest_enabled": "true", "api_key_enabled": "true", "testdataVersion": "Version1"' }

============ When I run the unit tests as 'JUnit Plug-in Test' against mylyn.local they fail with:

org.eclipse.core.runtime.CoreException: Unexpected error: Timeout waiting for connection from pool at
org.eclipse.mylyn.internal.jenkins.core.JenkinsCorePlugin.toCoreException(JenkinsCorePlugin.java:69) at 
org.eclipse.mylyn.internal.jenkins.core.JenkinsServerBehaviour.getPlans(JenkinsServerBehaviour.java:272) at org.eclipse.mylyn.jenkins.tests.integration.JenkinsIntegrationTest.testPlanParameters(JenkinsIntegrationTest.java:63) at 

java.base/java.lang.reflect.Method.invoke(Method.java:568) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at 
java.base/java.util.ArrayList.forEach(ArrayList.java:1511) Caused by: org.eclipse.mylyn.internal.jenkins.core.client.JenkinsException: Timeout waiting for connection from pool at org.eclipse.mylyn.internal.jenkins.core.client.JenkinsOperation.run(JenkinsOperation.java:247) 
at org.eclipse.mylyn.internal.jenkins.core.client.RestfulJenkinsClient.getJobsFromFolder(RestfulJenkinsClient.java:341) at 

OK, I focused on Bugzilla. Will look into the Jenkins Setup

[gnl42]

Almost perfect for me :) The only setup issue I have now is having to run the setup_vm script twice in a row. First run fails with: Failed to connect to the host via ssh: ssh: connect to host 10.209.22.169 port 22: No route to host which was the IP used the previous setup.

===== Bugzilla failures still:

Failures: 

  testGetTaskData(org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest): null

Errors: 

  testStdWorkflow(org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest): Task data at "E:\workspaces\mylyn-main\git\org.eclipse.mylyn\mylyn.tasks\connectors\bugzilla\org.eclipse.mylyn.bugzilla.tests\target\work\data\.metadata\.mylyn\tasks\bugzilla-https%3A%2F%2Fmylyn.local%2Fbugzilla%2D4%5F4\offline\424.zip" not found

  testAliasRetrieval(org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest): alias in use: Bug45 has already taken the aliasFritz. Please choose another alias.

  testAliasMultiRetrieval(org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest): alias in use: Bug45 has already taken the aliasFritz. Please choose another alias.

4 out of 150 tests failed

==== The params.json complaint is gone, but i did get a 'Bad Gateway' error when I first tried the url. When I tried it later, it worked fine

==== Using @EnabledIf with JUnit5.

The problem I have with using properties is they are fine for scripts but for individual runs they are a pain to have set up all the time.

==== I figured out that the message Status ERROR: org.eclipse.mylyn.bugzilla code=13 MylynStatus, really means:

ENTRY org.eclipse.mylyn.bugzilla 4 13 2024-08-05 17:52:49.874
!MESSAGE Empty password not allowed to login to https://mylyn.local/bugzilla-4_4.

Empty password not allowed for Authentication credentials.

Please validate credentials via Task Repositories view.

and the Status ERROR: org.eclipse.mylyn.tasks.core code=0 Failed to find unmatched container for repository ["https://mylyn.local/bugzilla-4_4"](https://mylyn.local/bugzilla-4_4) is another inconsistency in the code about how URL strings should end, in this case the code needs a trailing "/" to be added

Thanks George

[gnl42]

Could you merge?