HPSIM Configuration and Troubleshooting - client side


Notes By : Bibith Dathan 




Checking Requirements for SysFaultMgmt :

=================================================

HPUX 11i v1 (11.11) –
    SysFaultMgmt Version A.04.00.04
    HP-UX 11i v1 September 2005 or later
    GOLDBASE11i - B.11.11.0612.459 Base Patches for HP-UX 11i v1, December 2006
    SysMgmtWeb version A.2.2.5 (HP-UX Web Based System Management User Interface)
    OpenSSL A.00.09.07e.012 or later
    WBEMServices A.02.05 WBEM Services CORE Product
    OnlineDiag B.11.11.18.xx

HPUX 11i v2 (11.23) –
    SysFaultMgmt Version B.05.00.05
    May 2005 HP-UX 11iv2 Operating Environment (OEUR) or later
    QPKBASE B.11.23.0712.070a Base Quality Pack Bundle for HP-UX 11i v2, December 2007
    OpenSSL A.00.09.07e.013 or later
    SysMgmtWeb version A.2.2.7 (HP-UX Web-Based System Management User Interface)
    PHKL_34795  IPMI patch (Dependency: PHKL_31500)
    WBEMServices A.02.05.08 WBEM Services CORE Product
    OnlineDiag version B.11.23.10.xx

HPUX 11i v2 (11.23) –
    SysFaultMgmt Version B.06.00.04
    May 2005 HP-UX 11iv2 Operating Environment (OEUR) or later
    QPKBASE B.11.23.0803.xxx Base Quality Pack Bundle for HP-UX 11i v2, March 2008
    OpenSSL A.00.09.07e.013 or later
    SysMgmtWeb version A.2.2.7 (HP-UX Web-Based System Management User Interface)
    WBEMServices A.02.05.08 WBEM Services CORE Product
    OnlineDiag version B.11.23.10.xx
    PHKL_34795  IPMI patch (Dependency: PHKL_31500)

   
HPUX 11i v3 (11.31) –
    SysFaultMgmt C.03.00.06
    HP-UX 11i v3 February 2007
    OpenSSL Version A.00.09.07e.013 or later
    Baseboard Management Controller (BMC) firmware version 70.59 or later
    WBEM Services Version A.02.07 or later
    EVM-EventMgr B.11.31
    SysMgmtBase B.00.02.03
    SysMgmtWeb version A.2.2.4 (HP-UX Web Based System Management User Interface)
    Online Diagnostics B.11.31.02.yy


Installation SysFaultMgmt Software :

==================================================

The SysFaultMgmt Software is located at different location.
It can be downloaded from http://www.hp.com or as internal depot bundled
with all providers and additional Software useful for RSP or it can be installed aspart of the HPUX OE Media.

For HPUX 11.23 patch PHKL_34795  is a prerequisite – install this patch prior to installing SFM. 

1) For the hp.com depot do:
#swinstall -s <depot_path> SysFaultMgmt


2) For the OE Media:
# swinstall -x autoselect_dependencies=true -x enforce_dependencies=true -s /tmp/cdrom SysFaultMgmt

Check with swverify that all software is properly installed.

For 11.31 install patch PHSS_37856 (HPUX System Fault Management Mar 2008).



WBEM & SFM check and setup :
==================================================
1) Check that the correct version of the cimserver is running:
# /opt/wbem/sbin/cimserver –v
A.02.07

2) Check that the SFMProvider Module is running fine:
# /opt/wbem/bin/cimprovider -l –s
MODULE                                  STATUS
……
SFMProviderModule                       OK
……

3) Check that the providers are properly registered with the SFM Provider Module
# /opt/wbem/bin/cimprovider -l -m SFMProviderModule
CPUProvider
CPUStatusProvider
EMSWrapperProvider
SFMIndicationProvider
EventIndicationConsumer
MemoryProvider
MemoryStatusProvider
DiskProvider
DiskStatusProvider
StateChangeIndicationProvider
ChassisProvider
CoolingStatusProvider
PowerStatusProvider
ThermalProvider
VoltageProvider
MPProvider
MPStatusProvider
FirmwareRevisionProvider
HPUX_ControlProvider
FMDProvider
HealthStateProvider
EMArchiveConsumer
EMEmailConsumer
emdprovider
SubscriptionConfigAssociationProvider
ThrottlingConfigInstanceProvider
WBEMToEMSConsumer
EnclosureProvider

4) Check that EMS is enabled:
# /etc/opt/resmon/lbin/monconfig


5) Check that SFM is used for hardware monitoring:
# /opt/sfm/bin/sfmconfig -w –q

EMS hardware monitors are disabled & SysFaultMgmt is monitoring devices.

If EMS is still enabled for monitoring, switch to SFM using the command:
# /opt/sfm/bin/sfmconfig –w -s

6) Check that the HP-defined filters are setup:
# /opt/sfm/bin/sfmconfig -m list
Filter Name               : General Filter
Filter Type               : HP Defined Filter
Filter Unique Identifier  : 1
Filter Query              : Select * from HP_DeviceIndication
Filter Query Language     : WQL
Filter Source Namespace   : root/cimv2
Filter Description        : General Device Indications.
Filter State              : Enabled Filter State
Filter Last Operation     : No Operation
--------------------------------------------------------


If the HP-defined filters are not listed:
#/opt/sfm/bin/sfmconfig –m enable – n “General Filter” –u 1 –t HP
Filter Name               : General Filter
Filter Type               : HP Defined Filter
Filter Unique Identifier  : 1
Filter Query              : Select * from HP_DeviceIndication
Filter Query Language     : WQL
Filter Source Namespace   : root/cimv2
Filter Description        : General Device Indications.
Filter State              : Enabled Filter State
Filter Last Operation     : Enable Filter
--------------------------------------------------------

7) Change the startup mode for the System Management Homepage so that the autostart
   url mode is set to OFF and start on boot mode is set to ON:

#/opt/hpsmh/lbin/hpsmh stop
#/opt/hpsmh/bin/smhstartconfig -a off -b on
#/opt/hpsmh/lbin/hpsmh start
#/opt/hpsmh/bin/smhstartconfig

HPSMH 'autostart url' mode.........: OFF
HPSMH 'start on boot' mode.........: ON
Start Tomcat when HPSMH starts.....: OFF


Setup System(s) for Remote Support Pack :
===================================================
After installation of SysFaultMgmt Web the systems needs to be discovered by SIM.
To do this, start the SIM:

https://<Sim_Server>:50000 and navigate to Options  Discover.
If you want to discover single IP addresses, select “Manual”.


Enter your IP Address and press “Add System”.

If you want to discover a range of IP addresses choose “Automatic” and press “New”

Enter a name for you discovery, uncheck the “Automatically execute discovery every:”
check box, enter you IP address range and press “OK”. Select your discovery task and press “Run Now”.

The discovery will take a while, please wait until the task is finished.

When the task is finished, navigate to “All Systems” and check that the system(s) are
listed and properly discovered.

If the system is not discovered properly, navigate to:
->Options ->Identify System ->“Run Now”

Wait for the status to be “Completed” and check if the system is now properly discovered.



Configuration Steps
================================================
Set System properties (single servers)

The next configuration step is to set the system properties for the system(s).
To do this for a single system, click on the system name in the “All Systems” section

->“Tools & Links”
->“Edit System Properties”
->enter / verify company and contact information
->add serial and part number if not already listed


Create System properties task (for multiple systems)

To enter the same company and contact information for several systems and
every new system discovered, select:
->Options
->System Properties
->Set System Properties

For the pull-down menu select the system type that will have the same contact –
if the contact for all systems is the same select “All Systems”.
Press “Apply” and “Next>”.

Verify the selection, change it if needed and press “Next>”.

On the next screen, scroll down to the “Contract and Warranty Information” section.

For the “System Site Information, either select a site from the pull-down menu or
press “Manage Sites…”  To add a new site press “New…” and fill out the required fields.

Proceed with the “Manage Contacts” section in the same way:

When all date has been entered, use the “Schedule” Button to continue.


Check contract status
=================================================
To verify the contract status of the system, navigate to “All Systems” and select your device:
->Options
->Contract and Warranty Data Collection
->Press “Run Now”
Wait for the task to be completed.


If the entitlement worked fine, check the status in the Remote Support Pack.
To do this, navigate to
->Options
->Remote Support Configurations and Server
Select the “Entitlement” tab

If the system is not listed, press the “Synchronize RSE” button.
If the entitlement status is not correct, select the system.
From the “Action List” select “Entitle checked” and press “Run Action”.



Creating WBEM subscription
====================================================
Creating WBEM subscription using root account :

To use the root account for WBEM subscription again navigate to “All Systems” and select those with the same root password.

From the menu select
->Configure
->Configure or Repair Agents

Select “Next”, enter the credentials and press again “Next”.
For “Step 3: Install Providers and Agents  (Optional)” change nothing and press “Next”.
For “Step 4: Configure or Repair Settings” select
     -- Create subscription to WBEM events
    -- Send a sample WBEM / WMI indication
-- Use an HP SIM WBEM certificate

And deselect
      -- Configure a non-administrative account for HP SIM to access WMI data
     -- everything listed for "Configure SNMP"
     -- Configure secure shell (SSH) access authentication

If you want to setup data collections later one, make sure
"Set Trust relationship to "Trust by Certificate"" is selected.

Press “Run Now” to start the subscription. Wait for the task to be finished:


Creating WBEM Subscription using a non-privileged user :
==============================================

setup an account for the WBEM user using “SAM”. Navigate to: 

->Accounts for Users and Groups
->Users

    -- From the menu select “Actions” => Add
    -- Enter the needed data and press ok.

Or use the CLI command:

# useradd –m <user name>
# passwd <user name>

Now add read and write authorizations for the new WBEM user to each of the namespaces:  
     -- root/cimv2
     -- root/PG_InterOp
     -- root/PG_Internal:

Using the command cimauth:
cimauth -a -u <username> -n root/cimv2 -R -W
cimauth -a -u <username> -n root/PG_InterOp -R -W
cimauth -a -u <username> -n root/PG_Internal -R –W
cimauth -a -u <username> -n root/Npar -R -W
cimauth -a -u <username> -n root/Vpar -R -W

To check that all authorizations are set properly execute:

# cimauth -l
<username>, root/cimv2, "rw"
<username>, root/PG_InterOp, "rw"
<username>, root/PG_Internal, "rw"

To be able to do a WBEM event subscription with a non privileged account this has to be enabled.
Checked the current settings using the command:

#cimconfig -l -c
enableAuditLog=false
sslClientVerificationMode=optional
enableSubscriptionsForNonprivilegedUsers=false
shutdownTimeout=30
authorizedUserGroups=
enableRemotePrivilegedUserAccess=true
enableHttpsConnection=true
enableNamespaceAuthorization=true
enableHttpConnection=false

If the enableSubscriptionsForNonprivilegedUsers is set to false, this needs to be changed to true:
#cimconfig -s enableSubscriptionsForNonprivilegedUsers=true -p
#cimserver –s
#cimserver

After this has been done, the credentials for the system have to be set first.
Therefore select the system (if more than one system has the same credentials, select all of them)
in the “All Systems” section and navigate to:
->Options
->Protocol Setting
->System Protocol Setting

For the WBEM settings select “Update values for this protocol” and “Use values specified below”,
than enter the user and password.
          
Select “Run Now” and wait for the task to be completed.

When this has been successfully completed, you need to subscribe to WBEM events.
To do again select the system(s) in the “All systems” section and choose
->Options
->Events
->Subscribe to WBEM Events

Select “Next” and on the second page confirm the task with “Run Now”.
Wait for the task to be completed – this may take a while especially for PA RISC systems.


To verify the WBEM subscription from the HPUX device use the following command:

# /opt/sfm/bin/evweb subscribe -L -b external

Filter Name            Handler Name            Query                    Destinat Destination Url

HPSIM_simserver_0        HPSIM_simserver            select * from HP_DeviceIndication    UNKNOWN

HPSIM_ simserver_1        HPSIM_ simserver        select * from HP_ThresholdIndication    UNKNOWN

HPSIM_TYPE_1_ simserver_0    HPSIM_TYPE_1_ simserver_client    SELECT * FROM CIM_AlertIndication          CIMXML   https://simserver:50004/cimom/listen1

HPWEBES_simserver_Filter_HP_AlertIndication HPWEBES_simserver_Handler_HP_AlertIndication select * from HP_AlertIndication    UNKNOWN



Verifying WBEM Subscription :
=================================================

To verify the WBEM subscription in WEBES start the System Event Analyzer either using:
->Start
->Programs
->Hewlett-Packard Service Tools
->System Event Analyzer
->System Event Analyzer

Or

Starting a web browser using the URL: https://simserver-ip:7906

Login with <any name>-adv to get privileged rights. Select each system and confirm the data.

NOTE: Do not change any data within WEBES.
If anything is missing/wrong go back to SIM and enter / change the data in SIM.




Verifying status on HPUX host :
==================================================
There are two types of providers , the EMS wrapper providers and native providers.For some hardware we still use EMS to detect the event but this is forwardedto an EMS wrapper provider which is part of SFM.And for some hardware components there are already native providers.

EMS wrapper provider:

Check which EMS Monitors are running using the command:
#ps -ef|grep -i stm

Send a test event using oner of these monitor
#/etc/opt/resmon/lbin/send_test_event –v <monitor name>
f.e. #/etc/opt/resmon/lbin/send_test_event –v disk_em

Note: Check that a proper Resource Class/Hardware is listed for this monitor!

Read through the evweb output if the test event is listed – the event # of a test event is always 103:
#/opt/sfm/bin/evweb eventviewer -L

EvArchNo  Severity    Event #        Event Category Archive Time   Summary
========= =========== ============== ============== ============== ==============
10        Information 103            Unknown        2007-07-07 12: This is a t...

To get more info about that event use the following command:
#/opt/sfm/bin/evweb eventviewer –E –n <EvArchNo>


#/opt/sfm/bin/evweb eventviewer –E –n 10


Event Time...........: Mon NOV 09 12:33:30 2008
Severity.............: 2
Provider.............: DiskIndicationProvider
Event #..............: 103
System...............: server.deu.hp.com

Summary:

     Disk at hardware path 0/2/1/0.0.0.0.0 : Test event

Description of Error:

This is a test message from the monitor to test the communication path from the monitor
to the notification mechanism.

Probable Cause / Recommended Action:

     No action required.
……………..

SFM native provider (not available for 11.11)
    # /opt/sfm/bin/sfmconfig -t -a
    Sending test event for fpl_em monitor.
    Sending test event for ia64_corehw monitor.
    Sending test event for memory monitor.
    Sending test event for processor monitor.
    Sending test event for backbone monitor.

Note: The number and type of monitors could vary depending on the OS and the Systemtype.


Verifying status in SIM
==================================================

To verify the receipt of test events in HP SIM navigate to “All Events”:

To check if they are as well transmitted to the RSP, navigate to:

->Events
->Shared
-<Service Events
->All HP Service Events   


HP Service Events (processed by WEBES to Hp)


HPUX directories and logs :
==============================================
The install directory for System Fault Manager is

/opt/sfm
The logfiles are located at

/var/opt/sfm/log

WBEM is installed in the directory
/opt/wbem

The logfiles are located at
/var/opt/wbem/log

The log file for the System Fault Manager is
/var/opt/sfm/log/sfm.log    11iv2 and 11iv3
/opt/opt/sfm/log/sfm.log    11iv1


Troubleshooting :
=================================================
System Fault Management
Check that the cimserver is running:

# ps -eaf | grep cimserver
    root  1673     1  0  Jul 15  ?         4:15 /opt/wbem/lbin/cimserver
    root  1674  1673  0  Jul 15  ?         0:01 /opt/wbem/lbin/cimservera
    root  2349     1  0  Jul 15  ?         0:00 /opt/wbem/lbin/cimserverd

If it is not start it, do a
#cimprovider

To restart the cimserver
#cimprovider –s          to stop the service
#cimprovider                to start the service


Check the version of the cimserver:

# /opt/wbem/sbin/cimserver –v
A.02.05.08

If it is not the correct version, install the correct one using swinstall.


Check the health of the WBEM services:

# osinfo
OperatingSystem Information
  Host: server.deu.hp.com
  Name: HP-UX
  Version: B.11.23
  UserLicense: Unlimited user license
  Number of Users: 1 users
  Number of Processes: 157 processes
  OSCapability: 64 bit
  LastBootTime: Jul 15, 2007  16:27:35 (00200)
  LocalDateTime: Jul 17, 2007  23:41:27 (00200)
  SystemUpTime: 198832 seconds = 2 days, 7 hrs, 13 mins, 52 secs


Check that the SFM Provider Module is in status “OK”

# cimprovider –ls | grep -i sfm
SFMProviderModule                       OK

Check that the diagnostic mode is set to SFM:
# /opt/sfm/bin/sfmconfig -w –q
EMS hardware monitors are disabled & SysFaultMgmt is monitoring devices.

To switch to SFM Mode:
# /opt/sfm/bin/sfmconfig -w –s

Send a event using the native provider:
# /opt/sfm/bin/sfmconfig -t -a
Sending test event for fpl_em monitor.
Sending test event for ia64_corehw monitor.
Sending test event for memory monitor.
Sending test event for processor monitor.
Sending test event for backbone monitor.

Send a test event using the wrapper provider:
# /etc/opt/resmon/lbin/send_test_event –v <monitor name>

Check that the event is listed in evweb:
# evweb eventviewer -L

EvArchNo  Severity    Event #        Event Category Archive Time   Summary     
========= =========== ============== ============== ============== ==============
5         Information 103            Processor      2008-07-20 14: This is a t...
4         Information 103            System Hard... 2008-07-19 18: This is a t...
3         Information 103            Processor      2008-07-19 18: This is a t...
2         Information 103            Memory         2008-07-19 18: This is a t...
1         Information 103            System Hard... 2008-07-19 18: This is a t...


Check whether the HP-Defined filters are enabled:

# /opt/sfm/bin/sfmconfig -m list -t ALL
Filter Name               : General Filter
Filter Type               : HP Defined Filter
Filter Unique Identifier  : 1
Filter Query              : Select * from HP_DeviceIndication
Filter Query Language     : WQL
Filter Source Namespace   : root/cimv2
Filter Description        : General Device Indications.
Filter State              : Enabled Filter State
Filter Last Operation     : No Operation
========================================================

If needed enable them using the command:
#/opt/sfm/bin/sfmconfig –m enable – n “General Filter” –u 1 –t HP
Filter Name               : General Filter
Filter Type               : HP Defined Filter
Filter Unique Identifier  : 1
Filter Query              : Select * from HP_DeviceIndication
Filter Query Language     : WQL
Filter Source Namespace   : root/cimv2
Filter Description        : General Device Indications.
Filter State              : Enabled Filter State
Filter Last Operation     : Enable Filter
========================================================

If the provider is not registered properly, enter the following command at the HP-UX prompt:
# cimprovider -ls | grep SFMProviderModule

If the following output is displayed, all the providers are registered properly:
SFMProviderModule        OK

If the output displayed is different from this output, the provider module is not registered. To
register the provider module, enter the following command at the HP-UX prompt:

# cimmof -nroot/PG-InterOp /opt/sfm/schemas/mof/SFMProvidersR.mof

If no errors are displayed, the provider module is registered successfully. If errors are displayed,
restart the CIMOM by entering the following command at the HP-UX prompt:

# cimserver

After the CIMOM restarts, enter the following command at the HP-UX prompt to register the
provider module:

# cimmof -nroot/PG-InterOp /opt/sfm/schemas/mof/SFMProvidersR.mof

After the provider module is registered, create a link between the SFM providers and the CIMOM
by entering the following command at the HP-UX prompt:

On Itanium-based systems, enter:
#ln -s /opt/sfm/lib/libsfmproviders.1  \ /opt/wbem/providers/lib/libsfmproviders.so

On PA-RISC-based systems, enter:
# ln -s /opt/sfm/lib/libsfmproviders.1  \
/opt/wbem/providers/lib/libsfmproviders.sl











Post a Comment

0 Comments