After our EqualLogic PS6000 decided to go into a NetBSD Kernel Panic (crash) while performing a firmware update from version 5.1.1 to 5.1.2 using the Group Manager GUI, resulting in a firmware mismatch on our controllers, As advised by EqualLogic Tech support I performed another update to FW 5.1.2 using the CLI, with success.
Using the CLI has its advantages, altough the process is a little more involved, ie copying the firmware to the array member using FTP then SHH to the member to execute the update, you do get to see what happens during a firmware update in more detail.
Here is the process of performing a Firmware update using the CLI.
I will be using winscp to copy the firmware TGZ file to the group member, winscp can be downloaded here, and PuTTY a free Telnet/SSH client which can be found here.
Firstly download the firmware from EqualLogic Support https://support.equallogic.com
Lets update the firmware.
- FTP to the Management interface of the EqualLogic Array, Upload the TGZ file.
- Now that the TGZ file has been uploaded to the Array use PuTTY to SSH to the EqualLogic Management IP, login into the member with the group admin account. Once you have logged in execute the "update" command without the quotes.
Below you can see an extract from my PuTTY log file, and the process that happens:-
EQL01> update
11:36:44 Updating from kit file "kit_V5.1.2-R197668_718896520.tgz"
This command will update the firmware on the array's flash card memory
using the update kit file that was copied to the array.
If you choose to proceed, you will be shown the current firmware version
and the version to which you will update. You will then be given the
choice to proceed again.
Do you want to proceed (y/n) [y]: y
11:36:47 Verifying kit integrity.
11:37:08 Unpacking kit file.
11:37:14 Kit file verification complete, no problems.
11:37:31 Active and secondary controllers detected.
11:37:37
11:37:37 When the update procedure completes, the update procedure will
11:37:38 automatically send e-mail to your PS Series support provider and to
11:37:38 the e-mail address you have configured for your support contact:
11:37:38
11:37:38 its@**********
11:37:38
11:37:38 E-mail notification requires connectivity from the array to the
11:37:38 Internet through the SMTP server at address 10.11.5.40. If
11:37:38 the SMTP server is not available or does not have access to the
11:37:38 Internet, you can forward the message from the support contact
11:37:39 email address to update-notice@dell.com.
11:37:45
11:37:46 Currently running version is: Storage Array Firmware V5.1.1 (R189834) (H2)
11:37:46 You will be updating to version: Storage Array Firmware V5.1.2 (R197668)
11:37:46
The firmware update will not take effect until you restart the array.
Do you want to proceed (y/n) [n]: y
11:38:12 Preparing to update active controller.
11:38:13 Updating flash memory on active controller.
11:38:13 This will take several minutes.
11:40:50 Update of flash memory on active controller completed.
11:40:57 Preparing to update secondary controller.
11:40:59 Updating flash memory on secondary controller.
11:40:59 This will take several minutes.
11:44:35 Setting up for Email notification of update once new firmware has been loaded
11:44:36
11:44:36 Update completed successfully.
11:44:36
11:44:36 The firmware update will take effect when the array is restarted. To
11:44:36 restart the array, enter the restart command at the CLI prompt.
Now Restart the EqualLogic.
EQL01> restart
There is new firmware in the update area.
As part of applying the new firmware, the active and secondary control
modules will switch roles. Therefore, the current active control module
will become the secondary after the firmware is applied.
Would you like to load the firmware now? (yes/no) [no] yes
11:47:23 Verifying new firmware integrity.
11:48:30
11:48:30 PLEASE NOTE:
11:48:30 The restart process may take up to approximately 10 minutes.
11:48:30 During the restart process, do not restart or power down the array.
11:48:30
11:48:31 Start update of flash memory on secondary controller.
11:48:35 Setting cache to write through
11:48:47 Update of flash memory on secondary controller completed.
11:48:48 Restarting secondary controller.
11:49:08 Waiting for secondary controller to restart...
11:49:14 Waiting for secondary controller to restart...
11:49:19 Waiting for secondary controller to restart...
11:49:24 Waiting for secondary controller to restart...
11:49:30 Waiting for secondary controller to restart...
11:49:35 Waiting for secondary controller to restart...
11:49:40 Waiting for secondary controller to restart...
11:49:45 Waiting for secondary controller to restart...
11:49:51 Waiting for secondary controller to restart...
11:49:56 Waiting for secondary controller to restart...
11:50:01 Waiting for secondary controller to restart...
11:50:07 Secondary controller successfully restarted. Start secondary control module synchronization.
11:50:07 Waiting for secondary control module synchronization...
11:50:12 Waiting for secondary control module synchronization...
11:50:17 Waiting for secondary control module synchronization...
11:50:23 Waiting for secondary control module synchronization...
11:50:28 Waiting for secondary control module synchronization...
11:50:33 Waiting for secondary control module synchronization...
11:50:39 Waiting for secondary control module synchronization...
11:50:44 Waiting for secondary control module database synchronization...
11:50:49 Waiting for secondary control module database synchronization...
11:50:55 Waiting for secondary control module database synchronization...
11:51:00 Waiting for secondary control module database synchronization...
11:51:06 iSCSI PR PPool synchronization ..
11:51:06 Waiting for iSCSI PR PPool synchronization...
11:51:08 Secondary controller successfully updated. Transition current active controller to secondary controller.
11:51:20 Restarting active controller to complete the update.
At this point I lost connectivity to the EqualLogic, probably because it was failing over, dropping a couple of pings volumes remaining available though. I started SANHQ both controllers now on version 5.1.2 and correct failover occured, also recieved the notification E-Mail stating that the Firmware update was successfull.
"Update completion notification
Update started: 2012.01.06 11:37:39 GMT
Group name: DATASTORE01
Group address: 192.x.x.x
Group description: Primary Datastore Member list:
Name Serial No. Version
EQL02 ************** 5.1.1
EQL01 ************** 5.1.1
Member being upgraded: EQL01
Member description:
Member location: "Primary Site"
Version before update: Storage Array Firmware V5.1.1 (R189834) (H2)
Version after update: Storage Array Firmware V5.1.2 (R197668)
Compat. level before update was: 11.0..11.0
Allowable group compat. level was: 10..12
Post-update compatibility level: 11"
Done!