Related Manuals for HP zl
Summary of Contents for HP zl
-
Page 1: User Guide
HP 24-Port 4x Fabric Copper Switch User Guide November 2004 (First Edition) Part Number 377710-001... - Page 2 The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
-
Page 3: Table Of Contents
Intended Audience ... ix Typographical Conventions ... ix Contact Information ...x 1: Introducing the InfiniBand System ... 1 About the HP 24-Port 4x Fabric Copper Switch User Guide ...1 Maximize Server Networks ...1 What is InfiniBand? ...2 How Does InfiniBand Work? ...2 Possible Components ... - Page 4 Change Default User Name and Password ... 18 3: Understanding the Management Options ... 19 About the CLI ...19 Understanding the Command Modes ... 20 Using the CLI...21 Entering the Sub-Command Mode ... 21 Exiting Command Modes ... 21 Using Command Completion ... 21 Displaying Command History ...
- Page 5 Setting the System Clock ...41 Setting Time... 42 Synchronize the Clock to an NTP Server ... 42 Rebooting the System ...43 Reboot a System with a Single Controller... 43 6: Setting Access and Security ... 45 Understanding Access and Accounts...45 About User Accounts...
- Page 6 Change the Response Timeout of a SM... 65 Managing Synchronization Between SMs...66 Enable/Disable Database Synchronization ... 66 Set Configurations for the Master SM ... 67 Set Configurations for the Backup SM... 68 Adding a Subnet Manager ...70 Viewing Partitions ...71 About InfiniBand Multicast Groups ...72 Viewing Multicast Groups...72 View a Multicast Group Summary ...
- Page 7 TopspinOS Upgrades ... 93 About the System Image...93 What is a System Image?... 93 What is an Image File? ... 94 About Copying/Downloading the Image ...94 Card Status Requirements...95 Upgrade Procedure Overview...95 Set-Up the Hardware Connection ...95 Out-of-Band Connection... 95 In-Band Connection ...
- Page 8 View the GUID of an HCA ... 124 Determine Which HCA Port is Connected to an IB Port ... 124 View the GUID of an IB Switch... 126 Add an Attached Device to the Topology View... 126 View the Internal Chassis Topology...127 View Subnet Manager Details ...129...
- Page 9 Adding an SNMP Trap Receivers ... 147 Editing a Current SNMP Trap Receiver ... 148 14: Monitoring Through the CLI ... 149 About InfiniBand Events ...149 About Tracing ...149 Types of Traces... 150 Trace Levels... 150 About SNMP Traps ...151 Events Sent to Trap Receivers ...
- Page 10 viii...
-
Page 11: Intended Audience
Preface This document is a guide to the HP 24-Port 4x Fabric Copper Switch. Intended Audience The intended audience is the administrator responsible for installing, configuring, and managing your equipment. This administrator should have experience administering similar networking or storage equipment. -
Page 12: Contact Information
For HP technical support: In the United States, call 1-800-345-1518. In Canada, call 1-800-263-5868. In the United States and Canada, call 1-800-HP-INVENT (1-800-474-6836). This service is available 24 hours a day, 7 days a week. For continuous quality improvement, calls may be recorded or monitored. -
Page 13: 1: Introducing The Infiniband System
“How Does InfiniBand Work?” on page 2 About the HP 24-Port 4x Fabric Copper Switch User Guide The HP 24-Port 4x Fabric Copper Switch User Guide is specifically intended to demonstrate the processes involved in using and managing the InfiniBand •... -
Page 14: What Is Infiniband
Protocols InfiniBand requires a new set of protocols. For information on how to configure these protocols, refer to the HP Dual-port 4x Fabric Adapter User Guide. IPoIB The IP over IB (IPoIB) link driver provides standardized Internet Protocol encapsulation over InfiniBand fabrics. -
Page 15: Architectural Elements
devices. This protocol allows InfiniBand hosts to natively send SCSI commands as if the storage was direct attached. The SRP protocol is designed to operate using an RDMA communication service. An RDMA communication service provides communication between pairs of consumers; it uses messages for control information and RDMA operations for data transfers. -
Page 16: Understanding The Subnet Manager (Sm)
The Queue Pair (QP) is one of the primary architectural elements of InfiniBand. In InfiniBand, communication occurs between Queue Pairs, instead of between ports. A Queue Pair (QP) in an addressable entity, and consists of two Work Queues: 1). Send Work Queue and a 2). -
Page 17: Understanding Subnet Manager Routing
Local Identifiers (LIDs) • Global Unique Identifier (GUIDs) • Partition Key (P_Keys) • Configures switch forwarding tables. • Receives traps from Subnet Management Agents (SMAs). • Sweeps the subnet, discovering topology changes and managing changes as nodes are added and deleted. - Page 18 Distance - Distance is defined as the number of hops (InfiniBand switches or “switch elements”) between source and destination. Contention - A contention is declared for every switch port on the path that is already used for routing another LID associated with the same host port.
- Page 19 Therefore, it is possible to setup the cabling between switch elements to force the algorithm to prioritize certain paths. Depending on the network requirements, the prioritized paths can either be concentrated on a particular...
-
Page 21: 2: Getting Started
Overview of entire system installation on • Setup procedures for the InfiniBand Overview of Set-up Procedures Follow the steps below to configure the InfiniBand server switch system. 1. Determine your hardware topology. Install the Host Channel Adapter and drivers Install and power-on the InfiniBand Chassis 4. -
Page 22: Install Hcas And Drivers In Hosts
Install HCAs and Drivers in Hosts Refer to the HP Dual-port 4x Fabric Adapter Quick Setup Installation card and the HP Dual-port 4x Fabric Adapter User Guide. Install and Power on the InfiniBand Chassis Refer to the HP 24-Port 4x Fabric Copper Switch Hardware Quick Setup Installation card and the HP 24-Port 4x Fabric Copper Switch Hardware User Guide for installation and power instructions. -
Page 23: Configuring A Direct Serial-Console Connection
You are now ready to power down the chassis and mount it. Later, you can configure the box via Telnet, SSH, Chassis Manager or Element Manager. Configuring a Direct Serial-Console Connection Refer to the HP Serial Management Cable Guide for information regarding setting up the physical serial-console connection. Remote Telnet Login You can Telnet to the Management-Ethernet port on the box from a host on the same network as the Management-Ethernet port, or from any host with a route to the Management-Ethernet network. -
Page 24: Configuring An Infiniband Management Ip Address
To login via SSH: Use an SSH client (e.g. Putty) to port 22. Configuring an InfiniBand Management IP Address To configure an In-band InfiniBand management IP address: 1. At the Login: prompt, enter the username and password. The default is super and super. Login: super Password: super Topspin-360>... -
Page 25: Test Network Connectivity
Test Network Connectivity Refer to the HP 24-Port 4x Fabric Copper Switch Hardware User Guide for information regarding connecting network devices. After you install network cables, you can verify connectivity by pinging those connected devices from the CLI or pinging between attached hosts. -
Page 26: Verify Switch To Host Communication
= 0.000000/0.000000/0.000000 ms Topspin-360# Install the GUI (Element Manager) HP 24-Port 4x Fabric Copper Switches can be managed visually through the Element Manager, which provides a wide range of configuration, monitoring, and troubleshooting options. Refer to Manager installation. -
Page 27: Gui Management
2. Enter a CLI user name. The CLI password prompt ( 3. Enter the CLI user password. The prompt changes to indicate a successful login. The HP 24-Port 4x Fabric Copper Switch system is now ready to receive CLI commands. Remote SSH Login TopspinOS supports SSH2 for secure, encrypted login to the switch CLI. -
Page 28: Create A Partition Id (P_Key)
Create a Partition ID (P_Key) A default partition is configured automatically. The members of a default partition include all connected ports, and provide full membership. However, to create separation between traffic, you must configure specific partitions. Launch Element Manager, if you have not already done so. Select InfiniBand -->... -
Page 29: Specify Partition Members And The Membership Type
The configured p_keys will be needed in completing the configuration of the system. Configured partition keys must be mapped to any of the following components that exist: • Host Channel Adapters (HCAs). Refer to the HP Dual-port 4x Fabric Adapter User Guide. •... -
Page 30: Set User Levels And Passwords
Set User Levels and Passwords Change Default User Name and Password For security purposes, since multiple users exist on the system, it is highly recommended that you change the default passwords after initial configuration. “Understanding Usernames and Passwords” on page 46 1. -
Page 31: 3: Understanding The Management Options
About the CLI The Topspin system can be managed through the Command Line Interface. For more information regarding the CLI, refer to the HP 24-Port Fabric Copper Switch Command Line Reference Guide, or “Understanding the Command Modes” on page 20. -
Page 32: Understanding The Command Modes
The user-execute mode is the entry point to the privileged-execute mode and all CLI sessions begin in the user-execute mode. This mode provides commands for viewing some of the HP 24-Port 4x Fabric Copper Switch configuration and some user information. Guest users may only work in the user-execute mode. -
Page 33: Using The Cli
Sub-Command Mode The final mode is sub-command mode. Anything to do with InfiniBand, Ethernet, and Fibre Channel interface cards, ports, and gateways is done in this mode, including the Management-Ethernet ports. This mode is used to assign IP addresses to interface gateway ports, set connection speeds, set connection types, etc. -
Page 34: Displaying Command History
For example: Topspin-360(config)# fc ? - Configure FC SRP srp-global - Configure FC SRP-global parameters Topspin-360(config)# fc srp- ? enable - Enable FC SRP gateway-portmask-pol - Configure FC SRP-global gateway-portmask-policy - Configure FC SRP-global ITL lun-policy - Configure FC SRP-global lun-policy target-portmask-poli - Configure FC SRP-global target portmask policy Topspin-360(config)# fc srp- gate ? restricted... -
Page 35: Ending A Cli Session
To set the number of lines displayed on the terminal screen to 67 lines at a time and raise the time-out limit to 60 minutes 1. Enter the user-execute or privileged-execute mode. 2. Enter the page. Topspin-90# terminal length 67 3. -
Page 36: Quick Help
Quick Help You can enter the question mark (?) at the CLI prompt to display one of three types of user information. 1. Enter a question mark (?) at the CLI prompt at any time to display the commands you can execute. Only the commands appropriate to the current mode and user login are displayed. -
Page 37: The Chassis Window
Once you log into the Element Manager, the chassis window is displayed. This is the primary window in which you work. It graphically depicts the current configuration of the attached InfiniBand system chassis. Sample HP 24-Port 4x Fabric Copper Switch Chassis View Menu Bar Tool Bar... -
Page 38: About Selecting Items
Everything in the switch chassis display may be selected. You can select the chassis, interface cards, ports, management ports, and serial console ports. The only exception are the cards without ports. These are placeholders for later expansion and cannot be selected. -
Page 39: Using Element Manager
For information regarding upgrading the Element Manager, refer to page HP 24-Port 4x Fabric Copper Switch devices can be managed visually through the Element Manager, which provides a wide range of configuration, monitoring, and troubleshooting options. The Element Manager runs on multiple platforms, including Windows NT/2000/XP and Linux. To install the Element Manager: 1. - Page 40 If the folder does not exist, you will be prompted to create it. 7. Click the Next button. On Windows, the Choose Shortcut Folder window opens. Figure 3-1: Element Manager Installation, Choose Shortcut Folder Window On Linux, the Choose Link Folder opens. Figure 3-2: Element Manager Installation, Choose Link Folder Window 8.
-
Page 41: Starting The Element Manager
The default unrestricted community string is strings, refer to 4. Click A graphic representation of the switch chassis and current configuration is displayed. Start->Programs->Topspin Element Manager->TopspinEM window opens. This window is used to specify the IP address or DNS name of window opens. -
Page 42: Reading The Element Manager Status Colors
Any Network Manager running the Simple Network Management Protocol (SNMP) can manage the switch, if the Management Information Base (MIB) is installed correctly. By default, the switch GUI is a network manager and uses SNMP as the protocol to communicate between the chassis and the management workstation. - Page 43 receiver. You can have a maximum of six trap receivers. Entries in this list can be configured from menu. Health --> Trap Receivers • Community Strings - You can set community strings as a simple method of authentication between the system and the remote Network Manager. One unique community string is associated with each username and password.
-
Page 45: 4: Performing Admin Tasks Through The Gui
For example, if you intend to set the speed as 4x, it is imperative that you confirm you are using a 4x cable. Using a 1x cable on a 4x speed InfiniBand interface will cause serious performance issues. Launch Element Manager, if it is not already open. A graphic representation of your InfiniBand switch appears. Double-click an InfiniBand port. -
Page 46: Set Ib Interface Speed To Auto-Negotiate
Set the InfiniBand interface speed to auto-negotiate if you want the speed of the transmit port and the receiving port to automatically negotiate the highest possible speed. Launch Element Manager, if it is not already open. A graphic representation of your InfiniBand switch appears. Double-click an InfiniBand port. -
Page 47: View The Ib Interface Speed
View the IB Interface Speed View the current speed of the InfiniBand interfaces: Launch Element Manager, if it is not already open. A graphic representation of your InfiniBand switch appears. Double-click an InfiniBand port. The IB Port window appears. Check the Enable Auto-Negotiate box. -
Page 48: Setting The System Clock
Click the Apply button. Setting the System Clock Maintaining accurate time is important for statistics and auditing. The InfiniBand chassis provides an on-board system clock to save the time settings across reboots. Time is maintained in one of two ways: •... -
Page 49: Rebooting The System
4. Click 5. Click Rebooting the System Reboot a System with a Single Controller Card 1. Launch the Element Manager. 2. Choose the reboot option: slots. A pop-up window opens that allows you to select If changes have been made, you will be asked if you want to save the changes: 3. -
Page 51: 5: Performing Admin Tasks Through The Cli
Performing Admin Tasks Through the CLI This chapter describes the following administrative procedures that can be performed through the CLI. • “Setting the IB Interface Speed” on page 39 • “Notifying Users” on page 40 • “Setting the System Clock” on page 41 •... -
Page 52: Set Ib Interface Speed To Auto-Negotiate
Enter the following command: config interface ib [{IB switch card/IB switch port | all }speed 1x | 4x ] Example Topspin-360> enable Topspin-360# config interface ib all speed 4x Set IB Interface Speed to Auto-Negotiate Set the InfiniBand interface speed to auto-negotiate if you want the speed of the transmit port and the receiving port to automatically negotiate the highest possible speed. -
Page 53: Broadcasting Messages To All Users
Topspin-90> Setting the System Clock Maintaining accurate time is important for statistics and auditing. The switch chassis provides an on-board system clock to save the time settings across reboots. Time is maintained in one of two ways: • An on-board system clock •... -
Page 54: Setting Time
Topspin-360# copy running-config startup-config Synchronize the Clock to an NTP Server You can set the InfiniBand switch to synchronize the time with up to three NTP servers. To set the InfiniBand system clock in the CLI: 1. Log in to the CLI as the 2. -
Page 55: Rebooting The System
Rebooting the System Reboot a System with a Single Controller Enter the CLI reload command in privileged EXEC mode. The system prompts for you to verify the reload. If you had not already saved configuration changes, and the system detects the changes, it prompts you to save. -
Page 57: 6: Setting Access And Security
Setting Access and Security This chapter describes the following Access and Security features: • “Understanding Access and Accounts” on page 45 • “Managing Access and Accounts” on page 47 • “About Partitions” on page 53 • “Create Partitions (CLI)” on page 56 •... -
Page 58: Elements Of The Access System
In order for users to initiate an administration session, the User has to supply login credentials. The credentials supplied depend upon the interface being used. Elements of the Access System The Topspin access system associates the following key elements: • Username - Creates a unique username in the system. -
Page 59: Managing Access And Accounts
Understanding Permission Levels The following table displays the different access-levels. Managing Access and Accounts Setting or Changing a Password 1. Log in to the CLI as a super user. Use the default username and password if they have not already been changed (refer to 2. -
Page 60: Displaying User Information
7. Use the show user command to verify changes. Topspin-360# show user ========================================================================= ========================================================================= username : testuser password : $1$OHJt61CE$ANK02CcPqKnFoxJ0AKAtB. snmp-community : secret permission-level : unrestricted-rw admin-status : enabled num-logins : 4 num-unsuccessful-logins : 0 last-login : Mon Nov 17 22:43:25 2003 last-unsuccessful-login : Topspin-360# Displaying User Information... -
Page 61: Adding New Users
Adding New Users To add a new user account for both CLI and Element Manager access: 1. Log in as the unrestricted user. Only a user with unrestricted permissions may add new user accounts. Topspin-360# Login: super Password: xxxx Topspin-360> Enter the privileged-user mode. -
Page 62: Deleting A User Account
9. View the new user account information. Topspin-360(config)# show user ib-admin Topspin-360# ========================================================================= ========================================================================= username : ib-admin password : ib-passwd snmp-community : ib-commStr permission-level : ib-rw, ip-ethernet-ro, fc-rw admin-status : enabled num-logins : 0 num-unsuccessful-logins : 0 last-login : last-unsuccessful-login : Topspin-360# Deleting a User Account 1. -
Page 63: Changing Community Strings
Table 6-3: User Account Administrative Commands Command Task username user password Creates a new user account. passwd username user no Deletes a user account. username user Assigns a community string to a user account. community-string string username user Removes a community string from a user account. no-community-string username user privilege Assigns one or more permission level(s) to a user account. -
Page 64: Switching User Identity
• you created a new user account and you want to verify the access privileges • you have multiple administrative user-accounts and you want to switch to another administrative area To change your user identity: 1. Enter the user-execute or privileged-execute mode. -
Page 65: About Partitions
In the example above, keyword, and NOTE: When you change a user’s privileges, all the old privileges are removed and replaced with the new privilege(s). If the user had multiple privileges, include the other privileges on the command line when you make the change. Privileges are order-dependent. Enter them in the same order as shown in Table 6-2 on page Example The following example gives a user read-write access to InfiniBand and Ethernet configuration... -
Page 66: Partition Members
InfiniBand partitions are comparable to hardware-enforced security features of conventional I/O networking technologies, such as Ethernet VLANs and Fibre-Channel zones. Partition Members Without members, a Partition doesn’t have meaning to the system. Ports are added to the Partition, and become members of that Partition. Each port may be part of multiple partitions so that the system administrator can define overlapping partitions as the situation requires. - Page 67 Hex to Binary Conversions The following table is provided to assist in the creation of P_keys. When creating the Partition p_key, enter a hexidecimal value that is the equivalent of 16-bits in binary. For example, enter 80:00 (hex) to be 10000000000000000 (binary). The default Partition (which cannot be altered) is 7f:ff.
-
Page 68: Understanding How P_Keys Are Saved
If you have more than one InfiniBand switch in your fabric, refer to Synchronization” on page If you are configuring one InfiniBand switch, it will automatically be the master, and the partition configuration is saved persistently on the switch. Create Partitions (CLI) -
Page 69: Create A Partition Id (P_Key)
Create a Partition ID (P_Key) Default partitions are configured automatically. The members of a default partition include all connected ports, and provide full membership. However, to create separation between traffic, you must configure specific partitions. Create a partition using the following steps: Enter the following items at the global-configuration mode prompt: - the ib sm subnet-prefix command - the subnet-prefix of the IB subnet (use the show ib sm config subnet-prefix all command). -
Page 70: Specify Partition Members And The Membership Type
Click open the Subnet Manager folders in the left window. The Partitions folder appears. Click on the Partitions folder in the left window. The Partitions Summary window appears. Click the Add button. The Add Partition dialog box appears. Enter a Partition key (P_Key) to identify the new partition. For information regarding selecting values, refer to the Click the Add button. -
Page 71: About Ssh
The available partition members appear in the right-side window. Note that the “Available Members” refers only to members that are known to the Subnet Manager. This includes HCAs and Switches that are already plugged into the fabric as well as manually configured entries. -
Page 73: 7: Using The Subnet Manager Through The Gui
Using the Subnet Manager Through the GUI This chapter provides the following information: • “The Subnet Manager (SM)” on page 61 • “Viewing the Subnet Manager Configurations” on page 62 • “Changing the Subnet Manager Configurations” on page 64 • “Managing Synchronization Between SMs”... -
Page 74: Standby Subnet Manager
Standby Subnet Manager A standby subnet manager is a subnet manager (SM) that is currently quiescent, and not in the role of a master SM. Standby SMs are dormant managers, and can take over in case of failure of the master subnet manager. - Page 75 Information specific to that subnet manager appears. • View the subnet-prefix of the subnet manager. • View the Global Unique Identifier (GUID) of the subnet manager. • View the Status of the subnet manager. This is the operational status, as determined by self-detection. The values are notActive, discovering, or Master.
-
Page 76: Changing The Subnet Manager Configurations
Changing the Subnet Manager Configurations Change the Priority of a SM The priority number of a subnet manager tells the subnet manager how to interact with other subnet managers; the highest priority subnet manager becomes the master. The integer must be between 0 and 15, with the default being 0. Select InfiniBand >... -
Page 77: Change The Response Timeout Of A Sm
Highlight the subnet manager that you want to configure from the left-navigation bar. Click into the Sweep Interval field. Enter a value that indicates the number of seconds between sweeps of the network. The default is 10 seconds. The Apply and Reset buttons become active if a change is made. Click the Apply button to save the changes to the chassis. -
Page 78: Managing Synchronization Between Sms
Highlight the subnet manager that you want to configure from the left-navigation bar. Click into the Response Timeout field. Enter a value that indicates the number of microseconds allowed between the port reception of a subnet management packet and the transmission of the associated response. The Apply and Reset buttons become active when a change is made. -
Page 79: Set Configurations For The Master Sm
Click the Database Sync tab. Click the Enable box to enable database synchronization between the active and backup subnet managers. Set Configurations for the Master SM Select InfiniBand > Subnet Management. The Subnet Management window appears. Highlight the subnet manager that you want to configure from the left-navigation bar. -
Page 80: Set Configurations For The Backup Sm
The Subnet Management General tab appears. Click into the Master Poll Interval field to change the interval (in seconds) at which the master SM polls an active slave SM to verify synchronization. Click into the Master Poll Retries field to specify the number of unanswered polls that cause the slave to identify the master as dead. - Page 81 Click into the Max Backup SMs field to enter the maximum number of backup subnet managers with which the master subnet manager will synchronize. A backup subnet manager is automatically added whenever a new InfiniBand (IB) switch is connected to the IB fabric. The default is 1.
-
Page 82: Adding A Subnet Manager
Adding additional subnet managers (in addition to the one that is provided by default on the InfiniBand system) should only be done by experienced users. In the event that additional switch is added to an InfiniBand fabric, an additional subnet manager is added by default (one is the master, and one is the standby). -
Page 83: Viewing Partitions
Click the Add button. Viewing Partitions For complete partition information, refer to The partitions that are currently being managed by the subnet manager can be viewed by performing the following steps. Launch the Element Manager, if you have not already done so. Select InfiniBand >... -
Page 84: About Infiniband Multicast Groups
The Available Members window appears. About InfiniBand Multicast Groups An InfiniBand Multicast Group is a collection of Host Channel Adapter (HCA) ports that receive multicast packets sent to a single address. The configuration and members of a multicast group can be viewed through the Element Manager, but cannot be modified through these screens. -
Page 85: View Multicast Group Details
The Multicast Groups Summary page appears. View the Multicast Global ID (MGID), which is the 64-bit multicast GID address for the multicast group. View the Queue Key (Q_Key), which is the 16-bit Q-Key of this multicast group. The queue key is a construct that is used to validate a remote sender’s right to access. - Page 86 Click on a specific multicast group from the left-navigation tree. View the General tab, which is displayed by default. • • • • • • • • • • • Continue to Using the Members Tab The Members tab displays the multicast group members and the properties of those members. View the Q_Key for this multicast group.
-
Page 87: View The Subnet Manager Services
Services are mostly used by the DAPL protocol for Address Transferrable Services (ATS), but may also be used by the SRP protocol. Switch information may be reported for all the switches on a subnet or for a specific switch. View a Summary of the SM Services To view the services that are managed by the subnet manager: Launch the Element Manager, if you have not already done so. -
Page 88: View Details Of The Sm Services
Click on Services from the left-navigation tree. The Services window appears. View a Summary of the selected subnet managers services: • • • • View Details of the SM Services Details of the subnet managed Services are displayed in the Services window. Refer to Subnet Manager Services”... -
Page 89: Configure Subnet Manager Routing
View the Details of the subnet managed services: • • • • • • • • • • Configure Subnet Manager Routing For detailed information regarding Subnet Manager or InfiniBand routing, refer to Subnet Manager Routing” on page View the Service Name of the subnet managed service. View the 64-bit Service ID of the subnet managed service in the Service ID field. -
Page 90: Configure The Lid Mask Control (Lmc)
Configure the LID Mask Control (LMC) The Subnet Manager (SM) allows an administrator to define the LMC (Local Identifier Mask Control) value per subnet. Once the LMC value has been assigned, the SM routes different paths for each LID (an address assigned to a port) that is associated with the same host port. Launch Element Manager, if you have not already done so. - Page 91 The Routes window appears. Enter the Source LID and the Destination LID into the fields in the Filter Route section. Click the Show Route button.
-
Page 93: 8: Using The Subnet Manager Through The Cli
Using the Subnet Manager Through the CLI This chapter provides the following information: • “The Subnet Manager (SM)” on page 81 • “Viewing the Subnet Manager Configurations” on page 82 • “Changing the Subnet Manager Configurations” on page 83 • “Managing Synchronization Between SMs”... -
Page 94: Standby Subnet Manager
Standby Subnet Manager • A subnet manager (SM) that is currently quiescent, and not in the role of a master SM. Standby SMs are dormant managers, and can take over in case of failure of the master subnet manager. Viewing the Subnet Manager Configurations View a Summary of Subnet Management Enter the show ib sm configuration subnet-prefix [prefix | all] summary command. -
Page 95: Changing The Subnet Manager Configurations
Example Topspin-360# show ib sm config subnet-prefix ================================================================================ ================================================================================ subnet-prefix : fe:80:00:00:00:00:00:00 guid : 00:05:ad:00:00:01:38:82 priority : 10 sm-key : 00:00:00:00:00:00:00:00 admin-status : enable oper-status : master act-count : 216655 status : active sweep-interval : 10 response-timeout : 400 • View the subnet-prefix of the subnet manager. -
Page 96: Change The Sweep Interval Of A Sm
can run on the system and other SMs may run in your IB network, the priority attribute identifies the master SM. The integer must be between 0 and 15, with the default being 0. Enter the ib sm subnet-prefix prefix priority sm-priority command. Example Topspin-360>... -
Page 97: Set Configurations For The Master Sm
Set Configurations for the Master SM Enter the config ib sm subnet-prefix <prefix> master-poll-interval <1..60> command to change the interval (in seconds) at which the master SM polls an active slave SM to verify synchronization. The default is 3 seconds. Example Topspin-360>... - Page 98 Enter the ib sm db-sync subnet-prefix <prefix> poll-interval <1..30> command to change the interval at which the master subnet manager will send a synchronization session status request packet to an active session. The default is 3 seconds and the possible entries are 1...30. Example Topspin-360>...
-
Page 99: Adding A Subnet Manager
Adding additional subnet managers (in addition to the one that is provided by default on the InfiniBand system) should only be done by experienced users. In the event that an additional switch is added to an InfiniBand fabric, an additional subnet manager is added by default (one is the master, and one is the standby). -
Page 100: Viewing Multicast Groups
Viewing Multicast Groups View a Multicast Group Summary Enter the show ib sm multicast summary command. Example Topspin-360> enable Topspin-360# show ib sm multicast summary ============================================================================== Summary of Multicast-Groups on Device ============================================================================== subnet-prefix : fe:80:00:00:00:00:00:00 MGID : ff:12:40:1b:ff:f1:00:00:00:00:00:00:ff:ff:ff:ff multicast-group-members : port-GID : fe:80:00:00:00:00:00:00:00:05:ad:00:00:00:12:bf member-join-state : full-member proxy-join-status : false... -
Page 101: View Multicast Group Details
View Multicast Group Details Enter the show ib sm multicast {subnet-prefix prefix | all} command. Example Topspin-360# show ib sm multicast subnet-prefix all ================================================================================ Multicast-Groups Managed by Specific Subnet Manager ================================================================================ subnet-prefix : fe:80:00:00:00:00:00:00 MGID : ff:12:40:1b:ff:ff:00:00:00:00:00:00:00:00:00:01 q-key : 00:00:00:0b MLID : c002 t-class : 0 p_key : ff:ff... -
Page 102: Viewing The Sm Services
Services are mostly used by the DAPL protocol for Address Transferrable Services (ATS), but may also be used by the SRP protocol. Switch information may be reported for all the switches on a specific subnet or for a specific switch. View a Summary of the SM Services To view the services that are managed by the subnet manager: Enter the show ib sm service summary command. -
Page 103: Configure The Lid Mask Control (Lmc)
Locate the Source LID and Destination LID. Enter show ib sm switch-route subnet-prefix fe:80:00:00:00:00:00:00 src-lid <source-lid> dst-lid <destination-lid>. The following example displays a summary of the SM route switch element table for one source and destination. Example Topspin-90> show ib sm switch-route subnet-prefix... -
Page 105: 9: Using Image Files
Using Image Files This chapter describes the following Access and Security features: • “Types of Image Upgrades” on page 93 • “Upgrade Procedure Overview” on page 95 • Upgrade Procedure Steps (begins Types of Image Upgrades There are two different types of upgrades that may need to be downloaded to the Topspin system. TopspinOS Upgrades Downloads can be performed remotely using an tftp or an ftp server. -
Page 106: What Is An Image File
The user simply executes one CLI command to install an image file. Refer to the install command in the HP 24-Port 4x Fabric Copper Switch Command Line Interface Reference Guide. The TopspinOS stores up to three images on a disk: the uninstalled image, the current system (or “installed”) image, and the recovery image. -
Page 107: Card Status Requirements
6. Reboot the system Set-Up the Hardware Connection For detailed information about the hardware, refer to the HP Dual-port 4x Fabric Adapter User Guide and the HP 24-Port 4x Fabric Copper Switch Hardware User Guide. There are two types of hardware connections that can be used to download a new image to the InfiniBand system: Out-of-Band or In-Band. -
Page 108: Verify The Installed Image Version
Verify the Installed Image Version Check the Image Version Through the GUI 1. In the Element Manager, select The File Management Window appears. Note the version number of the installed image by looking in the File Name column. The installed system image has a .img extension. Image files that are not installed files (for example, the .img files) can be deleted at this time to make room for the latest version by using the delete command. -
Page 109: Copy/Download The Image Through The Gui
• “Copy/Download an Image Through the CLI” on page 98 Copy/Download the Image Through the GUI Images must be installed using the CLI; however, images can be copied from a remote or local location through the GUI. Copy an Image from a Remote Location This section describes using the Element Manager to copy an image file from a remote location before installing it as the active image. -
Page 110: Copy/Download An Image Through The Cli
Copy an Image from a Local Location This section describes using the Element Manager to copy an image file from a locally available location before installing it as the active image. 1. In the Element Manager, select Note the uninstalled image files(.img) and installed system-images. Do not modify or delete the installed system-image. - Page 111 Directory management is automatically performed on the InfiniBand system, so do not include path information for files on the chassis. 1. (Optional) If you are using In-Band Management, configure the ethernet management interface (if you have not already done so). Topspin-90>...
-
Page 112: Activate An Image
Directory management is automatically performed on the InfiniBand system, so do not include path information for files on the chassis. (Optional) If you are using In-Band Management, configure the ethernet management interface (if you have not already done so). Topspin-270> enable Topspin-270# config Topspin-270(config)# interface mgmt-ether Topspin-270(config-if-mgmt-ethernet)#ip address 10.3.102.8. -
Page 113: Specify A New Boot Image
Topspin-360# install image:Topspin-360-TopspinOS-1.1.3-build497.img *****************operation completed successfully Topspin-360# The name of the image file will change after it is installed. Repeat the install procedure on any cards that did not have an oper-status of “up” at the time of image installation. View a card’s status by using the show card command. -
Page 114: Specify A New Boot Image Through The Cli
Specify a New Boot Image Through the CLI 1. Find the name of your new system image by listing all the image files on the Topspin chassis. Enter the dir image command. Topspin-360# dir image ========================================================================= Existing Boot-Images on System ========================================================================= slot date-created -------------------------------------------------------------------------... -
Page 115: Deleting Image Files
“Deleting Images Through the CLI” on page 103 Deleting Images Through the GUI For information regarding deleting images through the Chassis Manager web GUI, refer to the HP 24-Port 4x Fabric Copper Switch Chassis Manager User Guide. 1. In the Element Manager, select Note the uninstalled image files(.img) and installed system-images. -
Page 117: 10: Using Configuration Files
Using Configuration Files This chapter describes: • “Understanding Configuration Files” on page 105 • “Listing Configuration Files” on page 106 • “Export a Configuration File” on page 106 • “Import a Configuration File” on page 108 Understanding Configuration Files A configuration file is a text file that stores a list of CLI commands. About the Startup-Config The main configuration file is called necessary to completely configure a box from a factory, default state. -
Page 118: Listing Configuration Files
Listing Configuration Files List Config Files Through the CLI To list the configuration files currently stored on the InfiniBand system, enter the config For example: Topspin-360# dir config ============================================================================ Existing Configurations on System ============================================================================ slot date-created ---------------------------------------------------------------------------- Tue Jan 14 23:19:55 2003 Tue Jan 14 00:11:04 2003 Topspin-360# List Config Files Through the GUI... -
Page 119: Export A Config File Through The Cli
Export a Config File Through the CLI To copy a configuration file to a remote FTP server through the CLI: Enter the copy command, as well as the necessary ftp information in one string: • • • • • • •... -
Page 120: Import A Configuration File
(Optional) If you are copying the config file to an ftp server: (Optional) If you are copying the config file to a local drive: Click the Copy button. Import a Configuration File Download configuration files from either a FTP or TFTP server to quickly replicate a desired configuration. - Page 121 • • • • • Topspin-360# copy ftp://bob:secret@10.0.0.5/random directory path/startup.cfg sampledirectory:startup.cfg Download a Config File From a TFTP Server To download a configuration file from a TFTP server through the CLI: Enter the copy command, as well as the necessary ftp information in one string: •...
-
Page 123: 11: Using Log Files
Using Log Files This chapter describes: • “Understanding Log Files” on page 111 • “Listing Current Log File Names” on page 112 • “Viewing a Log File Through the CLI” on page 113 • “Viewing a Log File Through the GUI” on page 114 •... -
Page 124: Listing Current Log File Names
• CONF - configuration changes. No user action is required • INFO - general information. No user action is required • WARN - abnormal condition. User intervention may be required • ERROR - abnormal condition. User intervention is required • FATAL - abnormal condition. -
Page 125: Viewing A Log File Through The Cli
Viewing a Log File Through the CLI Display Entire Log Set terminal parameters, if you have not already done so. Refer to on page Enter the show logging command. Topspin-360# show logging Nov 19 22:08:49 topspin-cc kernel: THH kernel module initialized successfully Nov 19 22:09:17 topspin-cc ts_sma.x[746]: [INFO]: IB SMA v0.2 Nov 19 22:09:18 topspin-cc notifier.x[770]: [INFO]: Notifier v0.02 Nov 19 22:09:19 topspin-cc watchd_mgr.x[789]: [INFO]: Watchdog Manager v2.00... -
Page 126: Show Details Of A Specific Log
Show Details of a Specific Log The more command displays the contents of a specified log file, including the active log-file. Enter more syslog:file name Topspin-360# more syslog:ts_log Nov 19 22:08:49 topspin-cc kernel: THH kernel module initialized successfully Nov 19 22:09:17 topspin-cc ts_sma.x[746]: [INFO]: IB SMA v0.2 Nov 19 22:09:18 topspin-cc notifier.x[770]: [INFO]: Notifier v0.02 Nov 19 22:09:19 topspin-cc watchd_mgr.x[789]: [INFO]: Watchdog Manager v2.00 Nov 19 22:09:20 topspin-cc ip_mgr.x[812]: [INFO]: IP Manager v0.04... -
Page 127: Filtering Logs
The Log Viewer window appears. Filtering Logs Launch Element Manager, if you have not already done so. Select Health --> Log Viewer. The Log Viewer window appears. Click the filter button at the top of the window. The Log Filter window appears. Determine the type of filter that you want to use: •... - Page 128 • • Filter Logs by Time Follow the steps in Click the From date radio button. Select the Month and Day from the drop-down menus. Enter the Hour, Minute, and second in the fields that follow the date. Click the To radio button. Select the Month and Day from the drop-down menus.
-
Page 129: Configuring Remote Logging
Filter Logs by Category Follow the steps in Click on the technology for which you want to capture logs from the Category column. Click the Apply button. Filter Logs by Text String Follow the steps in Enter the text string that you want to use as your log filter into the Message field. (Optional) Check the Case Sensitive box to further restrict the events that are captured by the log. -
Page 131: 12: Viewing The Ib Network Through The Gui
Viewing the IB Network Through the GUI This chapter provides the following information: • “About the Device Manager (DM)” on page 119 • “Display the Device Manager” on page 119 • “About the Topology View” on page 121 • “Display the InfiniBand Topology” on page 122 •... -
Page 132: View I/O Controller Units
View the Option ROM field to determine the presence or absence of Option ROM. View the Controllers field for a list of each slot on your Server Switch that can potentially contain a controller, and to identify whether or not a controller resides in that slot. -
Page 133: View I/O Controller Units Services
14. View the maximum depth of the per-channel RDMA Read Queue in the RDMA Read Queue Depth field. 15. View the maximum size, in bytes, of send messages in the Send Message Size field. 16. View the maximum size, in bytes, of the outbound RDMA transfers that the IOC initiates in the RDMA Transfer Size field. -
Page 134: Display The Infiniband Topology
Display the InfiniBand Topology View the Topology Launch Element Manager, if you have not already done so. Select InfiniBand --> Topology. The Specify Devices dialog box opens, if this is your first time viewing the Topology. To add a device, refer to Click the OK button. -
Page 135: View The Name Of An Hca
View the Name of an HCA To easily view the name of a Host Channel Adapter, use the Topology view. Open the Topology view, if you have not already done so. Refer to page 122. View the name of the HCA that is displayed with the HCA icons. “View the Topology”... -
Page 136: View The Guid Of An Hca
Viewing the ports in this way prevents you from having to view the physical hardware to determine which ports are connected. Open the Topology view, if you have not already done so. Refer to page View the HCA port and follow the connection to the IB switch. • 122. 122. - Page 137 • The slot/port number of the IB switch is labelled close to the IB switch icon. Port 1 of the the HCA is connected to Port 16/4 of the IB switch...
-
Page 138: View The Guid Of An Ib Switch
View the GUID of an IB Switch To easily associate an InfiniBand (IB) switch with the Global Unique Identifier (GUID), use the Topology view. Open the Topology view, if you have not already done so. Refer to page View the switch GUID information that is displayed with the InfiniBand switch icon(s). -
Page 139: View The Internal Chassis Topology
The Specify Devices dialog box appears. Click the Add button to add another connected InfiniBand device to the Topology view. The Add dialog box appears. Enter the DNS name or the Management-Ethernet port IP address of the InfiniBand device in the Device Name or IP Address field. - Page 140 The external Topology view appears. TS360 Icon Double-click the InfiniBand switch icon. The icon will appear different, depending on the type of IB switch you are using in your network.
-
Page 141: View Subnet Manager Details
The Internal InfiniBand Topology appears. View the connections between InfiniBand nodes and Ethernet or Fibre Channel Gateways. View the descriptions and node GUIDs of the InfiniBand nodes and any gateways. View Subnet Manager Details In addition to initializing and maintaining the InfiniBand fabric, the subnet manager also communicates with subnet agents to track changes to the InfiniBand topology as they occur. -
Page 142: View Advanced Node Information
View the kind of node that is being managed in the Type field. The value is channelAdapter, switch, route , or error. The error value indicates an unknown type. View the number of physical ports available on the node in the NumPorts field. -
Page 143: View Basic Port Information
View the supported base management datagram (MAD) version in the BaseVersion field. This field indicates that the channel adapter, switch, or router supports up to and including this version. View the supported base management datagram (MAD) class format in the ClassVersion field. - Page 144 The InfiniBand Subnet Details window appears. The Node tab is selected by default. Click the Ports tab. View the Subnet Prefix of the node to which the port belongs. The SubnetPrefix field identifies the InfiniBand subnet to which the node belongs. View the Global Unique Identifier (GUID) of the node to which the port belongs.
-
Page 145: View Advanced Port Information
10. Determine whether or not the nodes can actually communicate, and view the state transition that has occurred in the State field. A Transition is a port change from down to initialize, from initialize to down, from armed to down, or from active to down as a result of link state machine logic. - Page 146 View the initial value of the lease-period timer in seconds in the MKeyLeasePeriod field. Refer to section 14.2.4, Management Key, InfiniBand® Architecture, Vol. 1, Release 1.0, for more information. View the enabled link width in the LinkWidthEnabled field. The value is an integer that indicates the enabled link-width sets for this port.
- Page 147 35. View the maximum propagation delay allowed for this port to reach any other port in the subnet in the SubnetTimeout field. This value also affects the maximum rate at with traps can be sent from this port. Delay is affected by switch configuration. This parameter, along with RespTime, may be...
- Page 148 used by requestors to determine the interval to wait for a response to a request before taking other action. Duration is calculated as (4.096 ms * 2^SubnetTimeout ). 36. View the maximum time allowed between the port reception of a subnet management packet and the transmission of the associated response in the RespTime field.
-
Page 149: 13: Monitoring And Reporting Through The Gui
Monitoring and Reporting Through the GUI This chapter gives an overview of the following: • “About Analyzing Network Data” on page 137 • “About Tabular Formats” on page 138 • “About Graph Formats” on page 138 • “Creating a Data Analysis Table” on page 140 •... -
Page 150: Data Captured
Data Captured The data displayed depends upon the selected cards and ports. General interface activity, such as the number of octets, packets, and transmission errors, is provided for all port types. About Tabular Formats Network data may be viewed in a tabular or graph format. The tabular format is ideal for numbers crunching when you want to know precise activity statistics. - Page 151 • • Area Figure 13-2: In-Octets Statistics for a Single InfiniBand Port The appropriate graph icons become enabled when a set of graph data is selected. You can click a graph type and modifier to indicate how to display that data. Table 13-1: Graph Types and Descriptions Graph Type Associated...
-
Page 152: Creating A Data Analysis Table
Visual Modifiers Modifiers may be used singularly, or in conjunction with other modifiers, to change the orientation and scale of the graph displayed. With the use of display modifiers, the data displayed can be: • Horizontal/Vertical • Stacked/side-by-side • Standard/Logarithmic Table 13-2: Visual Modifiers and Descriptions Graph Type Associated... -
Page 153: Export A Data Table
If multiple ports (or cards) are selected, then select the type of counter data you want to display from the counter scroll-list on the bottom of the graph window. Select the statistical data to be monitored. Hold down the <Cntrl> key and click multiple fields to select specific counters Left-click and drag the mouse to define a selection rectangle Export a Data Table Once you have created a data table, you can export the table to a saved file for future reference. -
Page 154: Print A Data Table
Click the Export table icon. Navigate to the location where you want to store the data table. Enter a file name for the data table. Save the file. Print a Data Table Once you have created a data table, you can print the table for reference. Create a data table. -
Page 155: Creating A Data Analysis Graph
Click the print icon from the data table. Creating a Data Analysis Graph Statistical data is displayed in a table by default. However, you can use the data in the tabular format to create a variety of graphs. Create a table of network data. Refer to “Creating a Data Analysis Table”... - Page 156 Once you have selected the data to be monitored, the Graph icons become active. Select the icon that represents the type of graph you want to create. Refer to “Types of Graphs” on page 138 The graph appears in a new window.
-
Page 157: Modify A Graph
Modify a Graph All graphs (except for pie graphs) have the option of using visual modifiers to alter them. For example, a graph can be displayed horizontally or vertically. Click the icon that represents the visual modifier that you want to apply to the graph. Refer to “Visual Modifiers”... -
Page 158: Print A Graph
Print a Graph Once you have created a data graph, you can print the graph for reference. Create a data table. Refer to Click the print icon above the graph. About SNMP Traps Events Sent to Trap Receivers Event messages are generated whenever a significant event occurs. Significant events sent to trap receivers include: •... -
Page 159: Configuring Snmp Settings
• InfiniBand Multicast group is deleted • InfiniBand Partition is added • InfiniBand Partition is deleted • InfiniBand Multicast member is added • InfiniBand Multicast member is deleted • InfiniBand Node is added to a partition • InfiniBand Node is deleted from a partition •... -
Page 160: Editing A Current Snmp Trap Receiver
The Trap Receivers, Insert Trap... window appears. Enter the IP address of the desired Trap Receiver in the Address field. Enter the SNMP community string in the Community field. Check the Receive Events box if you want to enable the Trap Receiver to receive events. You can also add the Trap Receiver, then enable it later. -
Page 161: 14: Monitoring Through The Cli
Monitoring Through the CLI This chapter gives an overview of the following: • “About InfiniBand Events” on page 149 • “About Tracing” on page 149 • “About SNMP Traps” on page 151 • “Configuring SNMP Settings” on page 152 About InfiniBand Events Event messages are generated whenever a significant event occurs in the system, and cannot be turned-off or configured by the user. -
Page 162: Types Of Traces
The trace feature has different levels of control to allow you to dynamically select the amount and/or type of tracing information to be generated. Types of Traces The following types of traces are available: • Application • Module Application "Application" is used to specify the application for which tracing should be configured. The amount of applications listed may change. -
Page 163: About Snmp Traps
Flow "Flow" is used to specify a particular part of the code. The way that flows are specified varies depending on the module. • 0x1 : This flow shows all the Subnet Management (SM) discovery, assignment routing and routing processing. •... -
Page 164: Configuring Snmp Settings
Store the contact information for your device by entering the snmp-server contact command, as well as the ASCII string of contact information. Topspin-90(config)# snmp-server contact "Local HP support Representative" Store location information about your device by entering the snmp-server location command, as well as the ASCII text string of location information. - Page 165 acceptable p_key values ...56 access changing access levels ...52 community strings ...51 access levels ...47 adding new users ...49 admin user account commands ...50 Authorized Trap Receivers ...30 autoconnect, Element Manager auto-negotiate set IB interface with CLI ...40 set IB interface with GUI ...34 base version ...131...
- Page 166 master poll retries ...68 default link down ...134 default partitions ...54 default passwords ...46 deleting a user account ...50 image files ...103 system images ...103 destination LID ...78 device ID ...131 device ID field ...131 device vendor ID ...131 diag code ...133 disable autoconnection with element manager...
- Page 167 IPoIB ...2 LID ...132, destination ...78 source ...78 link speed active ...134 link speed enabled ...134 link speed supported ...134 link state machine ...133 link width enabled ...134 link width supported ...134 LinkSpeedSupported ...134 ...134 local identifier ...132 local identifier mask control ...134 local port number ...131...
- Page 168 PKey ...74, poll interval synchronization ...69 poll-interval ...86 port transition ...133 PortGUID field ...131 privileged-execute mode ...20 protocol version ...120 Q_Key violation ...135 Q_Keys ...135 QKey ...74 queue pairs ...3 rate ...74, RDMA transfer size ...121 read/write access ...47 rebooting about ...37, system ...102...
- Page 169 ...46 switch element display route ...91 switch elements sm routing ...6 synchronization cold sync ...5, poll interval ...69 transactional ...5 syslog ...114 system hostname ...12 TClass ...74, telnet login ...11 terminal parameters setting ...22 tftp image upgrade summary ...95 TFTP server ...99...