Have you ever looked in OpenManage Essentials and seen the above when looking at a device? I recently had this experience when checking on a number of older servers that we were not receiving alerts properly for.
Checking on the iDRAC and server it appeared that the management agents were running and correctly configured so that the OME server could contact the device, but attempts to discover and inventory were still failing. What was going on?
The Dell Troubleshooting Tool is an excellent utility by Dell to interrogate devices using variety of protocols. Querying the iDRAC using WSMAN soon found the problem:
Many of thew older servers had internal SSL certificates installed on them, which had subsequently expired. As most of the servers had been decommissioned, renewing the certificates had been overlooked.
Getting rid of the expired certificate is not as straight forward as it should be, with no ability in the iDRAC6 web interface to delete the certificate. This was resolved by accessing the IDRAC via SSH and using the racadm command:
After the iDRAC interface rebooted to apply changes, OME was then able to discover and inventory the iDRAC interface.
The Dell troubleshooting tool proved to be a very useful tool in the infrastructure admin’s toolbox for dealing with non-obvious management protocol issues.
It’s fair to say that I have not done any scripting for HP iLO since the release of their PowerShell scripting toolkit. I simply didn’t have a need. However in the past week I received a request to update my SSL Signing script for iLO to use the HP toolkit, and so I have.
As the HP cmdlets only support iLO3 and above, this script is also only compatible with iLO 3 and above.
The mechanics of the script depart from my previous scripts, in that they require a list of iLO interfaces in a text file, rather than doing DNS queries – this list should be a trivial thing to compile using Find-iLO HP cmdlet. It also no longer makes attempts to correct issues that may cause the certificate to not install correctly, it will simply note the problem and then list the skipped hosts in an exceptions report as the script completes. This greatly simplifies the script.
The iLO PS Library has been updated to version 1.1.2. This release has minor bugfixes and a new function for parsing RIBCL output to obtain the CSR that is created. Latest version of the script can be downloaded of the iLO PS Page.
I have completely re-written the iLO SSL Signing Script in the last week to make use of the iLO PS Library. This has seen a reduction in script code, but more importantly with the assistance of Joe I’ve fixed a couple of significant bugs in the original script – The script will now work properly for iLO3 devices for example. Script can be downloaded from the PS Scripts Page.
On a similar note, the iLO bulk update script for mass configuration has also been updated to use the iLO Library. The result being that the output from the reports is in an object format that is much easier to manipulate in Powershell. Once again, you can obtain on the PS Script Page.
I’ve been talking a lot about HP iLO devices and SSL recently, and hopefully this will bring some closure to this.
One of the early requests I had when I started working where I do was to “get rid of the certificate warnings” when signing into iLOs. Up until a few months back this was not possible due to iLO’s only understanding SSL Certificates for short hosts as opposed to FQDNs.
When the firmware was updated to support FQDN SSL Certificates, I was now able to start looking into getting all the certificates signed by our internal Microsoft Certificate Services CA. However there was no way I was going to manually configure 200+ interfaces.
So I wrote a script, and it is now available for all.
Great find from Oscar Perez over on the HP ITRC Forums. This tag is not documented in the current HP Scripting guides and I wish I knew about it before I logged in by hand to 200ish iLO devices to manually update the setting.
What the tag does is tell the iLO to general a CSR for an SSL Certificate using a FQDN rather than a short name. If you identify iLO devices in DNS with a subdomain (e.g. server.ilo.mydomain.com) you need to do this.
For the last couple of weeks I have been working on (re)building the HPSIM environment at work, which has included an upgrade to the latest version (6.2) and “setting it up properly”. My opinion of “setting up properly” included signing the SSL certificate with our internal CA.
One of the more perplexing issues I had was that the Remote Support Advanced Pack was not working and I had no information on why. Trawling the web eventually found a post on the HP ITRC that you need to import the CA certificate into the DESTA service for it to trust HPSIM:
desta cert -trustfile <file>.cer
Restarted the DESTA service and all was happy again
This little anecdote is about iLO Firmware and SSL
Firmware 2.01 for iLO2 may be downloaded from here.
One of my biggest gripes with HP iLO devices was that their self-signed certificates were next to useless is enterprise environments where a lot of people create DNS A records for the iLO so it is easy to remember (i.e. server.ilo.domain.com). The CSRs generated by iLO were hostname only, and attempting to install a FQDN Certificate resulted in it being rejected because the Certificate CN did not match the hostname.
iLO2 Firmware 2.0 released in July finally introduced FQDN SSL Support, however it suffered some bugs in CSR generation, and unfortunately still did not quite work as intended.
Finally firmware 2.01 was released in late August and this does appear to work. I have successfully created a FQDN SSL Certificate and applied to the iLO.
I strongly recommend this update for anyone with iLO2 interfaces who want to sign SSL Certificates using thier own enterprise CA.
Sadly, testing with iLO 3 firmware 1.10 has not been successful with the iLO rejecting the generated certificate. I am following this up with HP now.