search

intel / intelRSD

Intel® Rack Scale Design Reference Software
http://intel.com/IntelRSD
101 stars 55 forks source link

Feasibility of running a compute simulator on a single node or VM #4

Closed mmnelemane closed 8 years ago

mmnelemane commented 8 years ago

I have a question on the minimum requirements for a basic integration with openstack.

Is it feasible to run just the psme-compute-simulator as a service on a single node or on a VM to be interpreted by an openstack controller as a potential compute node ? What are the minimum set of services that need to be run in order to be able to interact with PSME APIs ?

I dont expect to create instances on the node but would like to test the APIs in a single node environment and use this for development. Any pointers on this will be helpful.

maciejro commented 8 years ago

Hi @mmnelemane,

Firstly, some clarification: probably you mean PSME Compute Agent Simulator which is a data source for another agent exposing REST API called PSME Rest Server (it connects to all available agents on the OS to gather information from them). PSME Compute Agent Simulator is exposing its data (read from XML file) through GAMI protocol. If I understood your question correctly, the answer is that we do not have any agent that contains OS within which you can run a Nova Compute service.

PSME Compute Agent Simulator is used for Deep Discovery feature (more information in Intel® Rack Scale Design Pooled System Management Engine (PSME) User Guide). The XML file is autogenerated based on information gathered from the host OS. This agent is responsible only for exposing data, but user is unable to make any interaction with it. Non-stub agent called PSME Compute Agent read data about host/system/node via IPMI protocol and manages it.

To check PSME REST API you can run PSME Rest Server with many (at least one) "Intel" or "Stub" agents running on the same host (if address 'localhost' is set in configuration files) where PSME Rest Server will be running. They can be run on any host (even there where they were compiled).

Basic integration with OpenStack: You have to run node with attached disk or iSCSCI Target with any OS that have preinstalled and preconfigured Nova Compute service. It will connect to OpenStack Cloud Controller just after OS start. More information about planned complex OpenStack solution can be found here.

mmnelemane commented 8 years ago

@maciejro : I am running psme-rest-server and psme-compute-simulator as per the instructions from the PSME User Guide. The response to curl request is below:

curl http://127.0.0.1:8888/redfish/v1/ | python -m json.tool

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   699  100   699    0     0  1225k      0 --:--:-- --:--:-- --:--:--  682k
{
    "@odata.context": "/redfish/v1/$metadata#ServiceRoot.ServiceRoot",
    "@odata.id": "/redfish/v1",
    "@odata.type": "#ServiceRoot.1.0.0.ServiceRoot",
    "Chassis": {
        "@odata.id": "/redfish/v1/Chassis"
    },
    "Description": "Service Root description",
    "EthernetSwitches": {
        "@odata.id": "/redfish/v1/EthernetSwitches"
    },
    "EventService": {
        "@odata.id": "/redfish/v1/EventService"
    },
    "Id": "RootService",
    "Links": {},
    "Managers": {
        "@odata.id": "/redfish/v1/Managers"
    },
    "Name": "Service Root",
    "Oem": {
        "Intel_RackScale": {
            "@odata.type": "#Intel.Oem.ServiceRoot",
            "ApiVersion": "1.2.0"
        }
    },
    "RedfishVersion": "1.0.0",
    "Services": {
        "@odata.id": "/redfish/v1/Services"
    },
    "Systems": {
        "@odata.id": "/redfish/v1/Systems"
    },
    "UUID": "b5fb5162-693a-11e6-811f-27b5d20a3a34"
}

So I believe, the services are working. However other APIs respond with "Unauthorized" error.

For eg: 

curl http://127.0.0.1:8888/redfish/v1/Systems | python -m json.tool

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    85  100    85    0     0   200k      0 --:--:-- --:--:-- --:--:-- 85000
{
    "error": {
        "@Message.ExtendedInfo": [],
        "code": "Unauthorized",
        "message": "Unauthorized"
    }
}

What authorization am I missing here ? I did a few configuration changes like:

In /etc/psme/psme-rest-server-configuration.json,

client-cert-required : false
rmm-present : false
use-ssl: false

In /etc/psme/psme-compute-simulator-configuration.json,

assetConfiguration {
  input : /etc/psme/deep_discovery.xml
  schema : /etc/psme/deep_discovery.xsd
}

The deep_discovery.* files were copied from the source path intelRSD/PSME/agent-simulator/compute/. Are these files even required when we dont have a PODM running ?

As I can recollect reading from the documents, the data for the APIs are fetched from the host when we run the simulator. Where would these data files be stored or fetched from ?

bobpiece commented 8 years ago

What authorization am I missing here ?

Read about following curl options: --insecure, --user Your request should be something like this: curl https://localhost:8443/redfish/v1/Systems --insecure --user user:password

maciejro commented 8 years ago

Hi @mmnelemane,

What authorization am I missing here ?

In /etc/psme/psme-rest-server.json file you have to set "use-ssl" to "true" in "connectors" section for port 8443 since all resources under /redfish/v1/* namespace are required to available via HTTPS protocol. "client-cert-required" and "rmm-present" shall be set to "false".

After that you can use following cURL command to retrieve data:

curl --insecure https://127.0.0.1:8443/redfish/v1/Systems

The deep_discovery.* files were copied from the source path intelRSD/PSME/agent-simulator/compute/. Are these files even required when we dont have a PODM running ?

Yes, because simulator uses those file for exposing fake data. No information are read from host OS by simulator agent. Have in mind that "schema" field is optional.

mmnelemane commented 8 years ago

with "use-ssl" set to "true", psme-rest-server expects /etc/psme/certs/server.key and /etc/psme/certs/server.crt to be present. Can these be any key and certificate generated with a common tool (eg: gnutls certtool) ? Doing that however doesn't allow me to start connector on port8443. Should I use any specific tool in the RSD source to generate the key and certificate ?

maciejro commented 8 years ago

Hi @mmnelemane,

First of all, sorry for the delayed response.

Can you answer the following questions?

You can also look on our script which is a good example of how the certificate is generated. This script is available here

Itxaka commented 8 years ago

@maciejro Same issue happening to me.

Steps: Using the script provided at https://github.com/01org/intelRSD/blob/master/PSME/lui/OS/rootfs/usr/bin/psme-gen-serv-cert.sh OS: Fedora 24, updated. Gnutls version: gnutls-3.4.15-1

maciejro commented 8 years ago

Hi @Itxaka,

Can you provide any logs from running this script?

Itxaka commented 8 years ago

@maciejro it doesnt really generates a lot of logs:

$ ./psme-gen-serv-cert.sh
generating certificate
Signature ok
subject=/C=EX/O=Example Inc/CN=zeus
Getting Private key

it generates srv_cert and srv_key which then I copied to /etc/psme/certs and renamed to server.key/server.crt

maciejro commented 8 years ago

Hi @Itxaka,

Could you execute ldd on psme-rest-server and give us output? Can you also attach logs from psme-rest-server (ex. journalctl -u psme-rest-server)? That would be very helpful for us!

Itxaka commented 8 years ago

@maciejro 

$ ldd psme-rest-server 
    linux-vdso.so.1 (0x00007ffd6c76e000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb4efdbf000)
    libossp-uuid++.so.16 => /lib64/libossp-uuid++.so.16 (0x00007fb4efbaf000)
    libmicrohttpd.so.10 => /home/itxaka/projects/intelRSD/PSME/build/lib/libmicrohttpd.so.10 (0x00007fb4ef99a000)
    libjsoncpp.so.1 => /home/itxaka/projects/intelRSD/PSME/build/lib/libjsoncpp.so.1 (0x00007fb4ef765000)
    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fb4ef52e000)
    libcurl.so.4 => /lib64/libcurl.so.4 (0x00007fb4ef2b3000)
    libgnutls.so.30 => /lib64/libgnutls.so.30 (0x00007fb4eef7f000)
    libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fb4eebf7000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fb4ee8ee000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fb4ee6d7000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fb4ee314000)
    /lib64/ld-linux-x86-64.so.2 (0x0000562f00ca7000)
    libfreebl3.so => /lib64/libfreebl3.so (0x00007fb4ee111000)
    libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007fb4edef0000)
    libidn.so.11 => /lib64/libidn.so.11 (0x00007fb4edcbb000)
    libssh2.so.1 => /lib64/libssh2.so.1 (0x00007fb4eda8d000)
    libpsl.so.5 => /lib64/libpsl.so.5 (0x00007fb4ed880000)
    libssl3.so => /lib64/libssl3.so (0x00007fb4ed635000)
    libsmime3.so => /lib64/libsmime3.so (0x00007fb4ed40e000)
    libnss3.so => /lib64/libnss3.so (0x00007fb4ed0e5000)
    libnssutil3.so => /lib64/libnssutil3.so (0x00007fb4eceb6000)
    libplds4.so => /lib64/libplds4.so (0x00007fb4eccb2000)
    libplc4.so => /lib64/libplc4.so (0x00007fb4ecaad000)
    libnspr4.so => /lib64/libnspr4.so (0x00007fb4ec86d000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fb4ec669000)
    libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fb4ec41c000)
    libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fb4ec135000)
    libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fb4ebf04000)
    libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fb4ebd00000)
    liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007fb4ebaf0000)
    libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007fb4eb89f000)
    libz.so.1 => /lib64/libz.so.1 (0x00007fb4eb689000)
    libp11-kit.so.0 => /lib64/libp11-kit.so.0 (0x00007fb4eb423000)
    libtasn1.so.6 => /lib64/libtasn1.so.6 (0x00007fb4eb210000)
    libnettle.so.6 => /lib64/libnettle.so.6 (0x00007fb4eafd8000)
    libhogweed.so.4 => /lib64/libhogweed.so.4 (0x00007fb4eadab000)
    libgmp.so.10 => /lib64/libgmp.so.10 (0x00007fb4eab18000)
    libssl.so.10 => /lib64/libssl.so.10 (0x00007fb4ea8a5000)
    libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fb4ea445000)
    libunistring.so.2 => /lib64/libunistring.so.2 (0x00007fb4ea115000)
    librt.so.1 => /lib64/librt.so.1 (0x00007fb4e9f0c000)
    libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fb4e9cfd000)
    libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fb4e9af8000)
    libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fb4e98de000)
    libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007fb4e96c0000)
    libffi.so.6 => /lib64/libffi.so.6 (0x00007fb4e94b8000)
    libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fb4e9290000)
    libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fb4e901d000)

For the logs, Im launching it manually, like this:

$ sudo ./psme-rest-server /etc/psme/psme-rest-server-configuration.json
2016-09-20 16:00:55.564247861 - INFO  - USR - [application-ref/src/app.cpp:load_configuration:42]MANUAL PSME BUILD; Built 13:43:42, 16-09-2016
2016-09-20 16:00:55.564263232 - DEBUG - USR - [common/configuration/src/configuration.cpp:add_file:280]Added file /etc/psme/psme-rest-server-configuration.json
2016-09-20 16:00:55.564274889 - WARN  - USR - [common/configuration/src/configuration.cpp:create_file_content_list:161]Cannot load default file
2016-09-20 16:00:55.564276132 - INFO  - USR - [common/configuration/src/configuration.cpp:create_file_content_list:167]Load file
2016-09-20 16:00:55.564294699 - INFO  - USR - [common/configuration/src/configuration.cpp:load_file:88]Loaded file /etc/psme/psme-rest-server-configuration.json
2016-09-20 16:00:55.564299828 - INFO  - USR - [common/configuration/src/configuration.cpp:create_file_content_list:174]Load internal defaults
2016-09-20 16:00:55.564326450 - WARN  - USR - [common/configuration/src/configuration.cpp:update_json_with_defaults:256]No configuration section <configuration>::server::security provided. Using internal defaults.
2016-09-20 16:00:55.564336945 - WARN  - USR - [common/configuration/src/configuration.cpp:update_json_with_defaults:256]No configuration section <configuration>::metadata-file provided. Using internal defaults.
2016-09-20 16:00:55.564358963 - INFO  - USR - [application-ref/src/app.cpp:load_configuration:57]JSON Schema load!
2016-09-20 16:00:55.565339593 - INFO  - app -  Reading Service UUID file: /etc/psme/service_uuid.json
2016-09-20 16:00:55.565425828 - INFO  - app -  Service UUID: 8a3ded44-7c06-11e6-a2a9-74d435eadd32
2016-09-20 16:00:55.565460799 - WARN  - app - Unable to open config file: /tmp/subscriptions
2016-09-20 16:00:55.568420681 - INFO  - app - Starting REST event service ...
2016-09-20 16:00:55.568438017 - INFO  - app - REST event service started.
2016-09-20 16:00:55.568989053 - INFO  - app - Starting REST server ...
2016-09-20 16:00:55.569185425 - INFO  - app - Loaded : /etc/psme/certs/server.key
2016-09-20 16:00:55.569204425 - INFO  - app - Loaded : /etc/psme/certs/server.crt
2016-09-20 16:00:55.569266710 - ERROR - app -  Cannot start connector on port 8443: Success
2016-09-20 16:00:55.569530140 - INFO  - app - Stopping REST event service ...
2016-09-20 16:00:56.568635881 - INFO  - app - REST event service stopped.
2016-09-20 16:00:56.568642590 - INFO  - app - Stopping REST event service ...
2016-09-20 16:00:56.568644716 - INFO  - app - Stopping REST server ...
2016-09-20 16:00:56.568646855 - INFO  - app - REST server stopped.
maciejro commented 8 years ago

Hi @Itxaka,

Few more questions and actions to take. That would be very helpful for us.

1) Was libmicrohttpd compiled with SSL support ? 2) Please execute following command and paste its output: ldd /home/itxaka/projects/intelRSD/PSME/build/lib/libmicrohttpd.so.10 3) Can you provide configuration summary of libmicrohhtpd compilation (during cmake execution) ? It should look something like:

…
config.status: creating src/testzzuf/Makefile
config.status: creating MHD_config.h
config.status: executing depfiles commands
config.status: executing libtool commands
configure: Configuration Summary:
  Operating System:  linux-gnu
  libgcrypt:         yes
  libcurl (testing): yes
  Target directory:  /home/m/repos/intelRSD-master/PSME/build.release.gcc.64bit
  Messages:          yes
  Basic auth.:       yes
  Digest auth.:      yes
  Postproc:          yes
  HTTPS support:     yes <-- most important
  epoll support:     yes
  libmicrospdy:      no
  spdylay (testing): no
…

Please try also installing the libmicrohttpd-devel package.

Itxaka commented 8 years ago

@maciejro Im very sorry, seems that when compiled manually (gmake + make all) it makes it miss something, probably the https support as you mentioned,

Using the setenv.sh + build64.sh scripts I can get it to build properly and the error is gone :)

May be something to look at, but at least it works now! Thanks!

maciejro commented 8 years ago

Hi @Itxaka ,

You're welcome! :) Please enjoy our solution!

Since @mmnelemane didn't respond to this issue for last 9 days, I'll close this issue. Fell free to open new issue if needed!