CALL US

888.221.5905

CARISIRT: Yet Another BMC Vulnerability (And some added extras)

After considering the matter for the past 6 months while continuing to work with Supermicro on the issues, I have decided to release the following to everyone. On 11/7/2013, after reading a couple articles on the problems in IPMI by Rapid7’s HD Moore (linked at the end), I discovered that Supermicro had created the password file PSBlock in plain text and left it open to the world on port 49152.

If you take a look at the /nv directory, you will find the file IPMIdevicedesc.xml file; a file which was already known to be downloaded via the aforementioned port. You can quite literally download the BMC password file from any UPnP enabled Supermicro motherboard running IPMI on a public interface (reference link at the bottom of this article). This is not the only file that is vulnerable to this. All the contents of the /nv/ directory are accessible via browser including the server.pem file, the wsman admin password and the netconfig files. When I attempted to reach out to Supermicro , the standard response received was that the UPnP issue had already been patched with the newest IPMI BIOS version. However, flashing a system is not always a possibility.

After my previous attempts to gain forward momentum with this issue had failed, and after getting the advice to release from several other security professionals, I reached out to one John Matherly (Shodan) and discussed with him what I had found. Being the awesome person that he is, he provided data on every host that was responding to a web request on port 49152 and the response to “GET /PSBlock”. I was blown away by the results (below):

Total Hosts responding to web requests on port 49152: 9,867,259

Vulnerable Systems: 31,964

(Now keep in mind that not everything responding on port 49152 is a Supermicro product. As it turns out, many products use the embedded UPNP software by default, but let’s get through Supermicro first)

This means at the point of this writing, there are 31,964 systems that have their passwords available on the open market. It gets a bit scarier when you review some of the password statistics. Out of those passwords, 3296 are the default combination. Since I’m not comfortable providing too much password information, I will just say that there exists a subset of this data that either contains or just was “password”.

Besides flashing, there is another (albeit unsupported) temporary fix. Most of the systems affected by this particular issue also have their “sh” shell accessible from the SMASH command line. If you login to the SMASH via ssh and run the command “shell sh”, you can drop into a functional SH shell. From there you can actually kill all “upnp” processes and their related children, which provides a functional fix. That is of course until the system is completely disconnected from power and reconnected, during which the IPMI module will reboot. This is what I have done for our own systems that were unable to be permanently fixed at this time. After continual monitoring, I am satisfied with the results and there has not been any noticeable impact on functionality.

So what are the rest of those IPs? Why are 9.8 million devices using port 49152, which is one port above the registered port range (1024-49151)? There are another 9,835,313 that have port 49152 open and are returning something from an HTTP GET request. It appears that many systems that employ an embedded Linux-based solution (i.e., home routers, remote management devices, and IP cameras) have varying iterations of the UPnP feature set installed. This causes them to broadcast their kernel and often times the hardware architecture. Take for instance this string from all vulnerable Supermicro BMC (Baseboard Management Controller) products: Linux/2.6.17.WB_WPCM450.1.3. The kernel version is 2.6.17, which was compiled for the Nuvoton’s WPCM450 chip. If you combine this knowledge with a search database for online devices, such as John Matherly’s interface located at shodanhq.com, embedded host identification becomes a breeze.

Another very disturbing discovery was that a lot of systems are running older versions of the Linux kernel. Approximately 23,380 of the total hosts are running the 2.4.31.x kernel, another 112,883 are running the 2.4.30.x kernel, and 710,046 systems are running the 2.4.19.x kernel. The largest number of systems responding to an HTTP GET request were systems running under the banner of AT&T U-Verse with a total of 6,448,716. However, they do not broadcast any information, and they respond with the HTTP code “200 OK”.

So what do we do? Is it our fault as consumers/businesses for trusting our vendors, or is it our vendors’ fault for ‘being human’? Short answer, none of the above, but the longer answer is a bit more complicated. Sure, a password should never be in plain text, and nor should any feature set be hardcoded into an embedded product if its features serve no functional purpose, but we do not live in a perfect world. Things have a way of slipping through the cracks, even by the most diligent of companies. Supermicro , to its credit, no longer employs the WPCM450 chips. With the release of their newer X10 series motherboards, they have replaced the aging WPCM450 BMC with the ASPEED AST2400 BMC product with a newer kernel version.

It is time to call for stronger security of embedded platforms. With the advent of services like shodanhq.com, and research operations such as Rapid7’s Project Sonar, devices can no longer dwell amongst the anonymity of the nearly 4.3 billion IPv4 addresses. Recent findings on the above platforms have proven everything is visible. With the advent of IPv6 and the ‘Internet of Things’, we as both customers and vendors need to ensure the security of our networks and connected devices.

To protect ourselves, I have found that the best solution is to keep informed and stay involved. If you find a vulnerability, reach out to the respective vendor first. If the vendor is unresponsive or does not share your urgency, there are organizations such as the US Department of Homeland Security’s Computer Emergency Readiness Team (US-CERT) and the MITRE Corporation who will assist. If they determine the issue does not meet their criteria for assistance, try subscribing to a security-minded mailing list and see if somebody there will assist you. As for the devices you have around your home or workplace, an interesting adventure is to search them and append the word “vulnerability”. You’ll be amazed by the results. As for staying informed, I personally suggest the use of packetstormsecurity.org and isc.sans.edu, as well as subscribing to the os-security mailing list. And finally, keep your device firmware up-to-date. Security is no longer something left to the professionals; it needs to be a part of our daily internet lives.

Finally, I want to thank HD Moore (Rapid7), John Matherly(Shodan), Kurt Seifried(Red Hat), and Dan Farmer (fish2.com or trouble.org) for their support in this and making this possible. If you want more information regarding the history of IPMI vulnerabilities as whole, I strongly suggest you read the material located at Dan Farmer’s IPMI research page, found in the references section below. On the Supermicro side, the Sr. Product Manager Arun Kalluri has been an incredible resource in gathering information on what Supermicro has done in light of this issue, and has provided a link to other known issues in the BMC product (which is in the articles reference). I have contacted MITRE’s cve-assign group, who has also worked with Supermicro on this matter. To date, no CVE has been assigned. The team at Metasploit will be releasing a module for the PSBlock and wsman password retrieval shortly, so be on the lookout!

I would love to hear your thoughts on this, so please leave a comment or shoot us an email! If you have any questions, concerns, or if you would like more information, please contact our security team at sirt@cari.net and we will get back to you shortly.

Regards,

Zachary Wikholm
Senior Security Engineer
Security Incident Response Team (CARISIRT)
CARI.net
sirt@cari.net

Articles & Reference Material:

https://community.rapid7.com/community/metasploit/blog/2013/07/02/a-penetration-testers-guide-to-ipmi

https://community.rapid7.com/community/metasploit/blog/2013/11/06/supermicro-ipmi-firmware-vulnerabilities

http://www.supermicro.com/FAQ/index.aspx?&se=16536&k=y

http://www.supermicro.com/products/nfo/files/IPMI/CVE_Update.pdf

http:/fish2.com/ipmi

53 Responses to “CARISIRT: Yet Another BMC Vulnerability (And some added extras)”

  1. Stefan Voigt

    wow… thats crazy… more details on shell sh please cause it doesnt work on any of my servers !

    -> shell sh
    shell command not support now.

    thx!

    Reply
    • Zach W

      Thanks for the read! The shell sh only works on specific versions. Is port 49152 open on those systems as well?
      You can also email us at sirt@cari.net if you would like more information on mitigation techniques.

      Reply
    • Abnormal

      No I didn’t I’m afraid. I tried on Centos 5.x and Debian 5. I might have tried the vsireon of Ubuntu that was current at the time too. The issue seemed to be that the IPMI card didn’t present a network interface, which is counter to all of the other IPMI cards I’ve used, which were on Dell and Supermicro servers.As you can see in the article, I called HP and they didn’t understand what I was on about.

      Reply
  2. Stefan Voigt

    PS: sirt@cari.net bounces. and I don\’t see any other way how to get in touch with you.

    Reply
  3. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Daily Tech News

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  4. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Tech Tips

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  5. ADnjus | At least 32,000 servers broadcast admin passwords in the clear, advisory warns

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  6. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Kronosim

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  7. Randy Bush

    vulnerability confirmed, if you like to write xml :)

    this is disgusting

    Reply
  8. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Tech-RSS.com

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  9. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Binary Reveux

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
    • Sinan

      Adam,I have a ML115 G1. The integrated IPMI/BMC conrtoller (not the optional LO card) is actually functional. I was able to get a login prompt and browse what I believe is the IPMI structure . I’m not very familiar with IPMI commands yet The first thing you need to do is check in the BIOS and make sure that the IPMI/BMC is enabled and assigned to the serial port. You will then be able to get to the IPMI/BMC CLI from the serial port. Once you have some sort of terminal connected to the serial port, you use ESC ( (without the quotes) to assign the serial port to the IPMI/BMC conrtoller and ESC Q to give the serial port back to the system.Once you have done ESC (, hit enter a few times and you should get a login prompt. I use admin as the user name and password. You will then get a /./-> prompt. From there you use the show command to list targets , properties and verbs possible for the targets.I was able to cd into map1 then nic1 and see a bunch of properties that you can assign values to (ie: network address/mask/gateway/dhcp enable, etc.I believe this may be what you are looking for.For my use, I just want to be able to reboot this server remotely when the OS becomes unresponsive. So I have connected the ML115 serial port into a serial console server (DECServer 700) which is accessible from the network. Then it’s just a matter of telneting into the DECServer, connect a session to the serial port of the ML115 and issue IPMI commands from there to reset the server. Now I just need to find out what that IPMI command is!Sylvain

      Reply
  10. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Gizmo Envy

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  11. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Techbait Tech News

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  12. Alert issued over plain text passwords in some Super Micro motherboards | What's On Scotland

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  13. Alert issued over plain text passwords in some Super Micro motherboards – Jumbosky Money

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  14. Alert issued over plain text passwords in some Super Micro motherboards | POPFIX - Celebrity, Tech, Sports News

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  15. Alert issued over plain text passwords in some Super Micro motherboards | News around The World

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  16. Yebaa.com | Alert issued over plain text passwords in some Super Micro motherboards

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  17. Alert issued over plain text passwords in some Super Micro motherboards | Wizeguys

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  18. Alert issued over plain text passwords in some Super Micro motherboards

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  19. Alert issued over plain text passwords in some Super Micro motherboards | News all the time

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  20. Connect IMS - Integrated Marketing Solutions || Alert issued over plain text passwords in some Super Micro motherboards

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  21. 32,000 motherboards spit passwords in CLEARTEXT! | Techbait Tech News

    […] “This means at the point of this writing, there are 31,964 systems that have their passwords available on the open market, Wikholm wrote on web host Carinet’s security incident response team blog. […]

    Reply
  22. Alert issued over plain text passwords in some Super Micro motherboards | Protect Your PC | Tips, Advice, and support. Protect Your PC | Tips, Advice, and support.

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  23. Alert issued over plain text passwords in some Super Micro motherboards – Health and Fitness

    […] controller (BMC) in the WPCM450 line of chips incorporated into motherboards made by Super Micro, wrote Zachary Wikholm, senior security engineer for Cari.net, a server and cloud computing […]

    Reply
  24. 32,000 motherboards spit passwords in CLEARTEXT! | ste williams

    […] “This means at the point of this writing, there are 31,964 systems that have their passwords available on the open market, Wikholm wrote on web host Carinet’s security incident response team blog. […]

    Reply
  25. More than 32000 servers expose admin passwords in the clear | Security Affairs

    […] that the problem is well known and a series of patches has been already released to fix the critical vulnerability, as explained by experts at CARI.net […]

    Reply
  26. At least 32,000 servers broadcast admin passwords in the clear, advisory warns |

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  27. Plaintext Supermicro IPMI Credentials ExposedDigital Era | Digital Era

    […] response team for San Diego-based cloud-based hosting provider CARI.net yesterday disclosed that a file storing passwords in plain text is open over port 49152. Close to 32,000 vulnerable systems responded to a GET/PSBlock query on the Shodan search engine […]

    Reply
  28. Another IPMI Mishap Leaves Thousands Of Servers Vulnerable To Trivial Hacks | The Security Ledger

    […] researcher Zachary Wikholm over at Cari.net has published evidence of what he says is a head-slapping vulnerability affecting devices that use IPMI … (BMCs) made by the firm […]

    Reply
  29. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | Change your style, keep your budget

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  30. BMC vulnerability exposes admin password

    […] into the motherboards. Security Researcher at CARInet Security Incident Response Team, discovered that Baseboard Management Controller (BMC) of Supermicro motherboards contain a binary file that […]

    Reply
  31. At least 32,000 servers broadcast admin passwords in the clear, advisory warns | World Updates

    […] passwords despite the release of an update that patches the critical vulnerability, an advisory published Thursday […]

    Reply
  32. Another BMC Vulnerability | Mike the Crypto Goat

    […] good folks over at CARI SIRT have discovered a vulnerability in the BMC controller shipped with many Supermicro servers. The […]

    Reply
  33. Уязвимость в BMC-контроллере Supermicro позволяет получить доступ к паролям управляющего интерфейса | AllUNIX.ru — Всероссийский портал о UNIX-

    […] Сontroller) чипе, используемом в материнских платах Supermicro, выявлена уязвимость, позволяющая злоумышленнику получить […]

    Reply
  34. Plaintext Supermicro IPMI Credentials Exposed | RobertJGraham.com

    […] response team for San Diego-based cloud-based hosting provider CARI.net yesterday disclosed that a file storing passwords in plain text is open over port 49152. Close to 32,000 vulnerable systems responded to a GET/PSBlock query on the Shodan search engine […]

    Reply
  35. Update: Another IPMI Mishap? Researcher Claims Supermicro Devices Vulnerable | The Security Ledger

    […] Specifically: researcher Zachary Wikholm over at Cari.net has published evidence of what he says is a head-slapping vulnerability affecting devices that use IPMI Base Management Controllers (BMCs) made …. […]

    Reply
  36. Chris

    There doesn’t appear to be any fix published by Supermicro for the affected IPMI cards on their AMD motherboards, e.g. H8SGL. It becomes increasingly clear they are a very dodgy vendor.

    But worse… as I browse round the filesystem on these IPMI cards, I am increasingly appalled. They’re a complete, shocking mess. No wonder they’re riddled with security holes given the evident poor-calibre of the developers who cobbled them together.

    Reply
    • Zach W

      Hey Chris,

      Thanks for the comment! I’ll take a look at this right now. And yes, the file system is incredibly scary. I don’t think they’ve changed the structure since the original on-board BMCs. The kernel version matches that time period. It’s extremely unnerving, realizing that they are recycling the flawed structure. Who knows if the new version of the firmware is actually truly patched, or just “duct tape” fixed.

      Zach W.

      Reply
  37. Supermicro IPMI BMCs plaintext passwords exposed | Threatpost | The first stop for security news

    […] response team for San Diego-based cloud-based hosting provider CARI.net yesterday disclosed that a file storing passwords in plain text is open over port 49152. Close to 32,000 vulnerable systems responded to a GET/PSBlock query on the Shodan search engine […]

    Reply
  38. David

    Have a Supermicro H8DG6-F that’s vulnerable.

    Digging around on SM’s web-site the only IPMI
    update that can be found is the currently-installed
    SMT 2.50 version from 2012.

    No problem effecting the work-around, but
    it looks like Supermicro has failed to beat
    a fix out of ATEN. Has anyone had any luck
    with obtaining a solution directly from
    ATEN, the BMC manufacturer? In general
    Supermicro focuses on assembling the most
    suitable components for particular servers
    and does not waste much time and effort
    slapping OEM re-branding on the
    components they integrate. I find this
    a refreshing contrast to the likes of HP,
    who’s servers are inferior in my experience.

    Reply
    • Zach W

      Hey David,

      Thanks for the comment! You are the second person to mentioned this. I”m going to contact Supermicro at this immediately. Shoot us an email about this as well to sirt@cari.net and I will get that ball rolling immediately.

      Zach W.

      Reply
  39. Markus

    The X9SRi-F is vulnerable as well, the latest version published is 2.14.
    So if there is a fixed firmware, it is not available via the website

    Reply
  40. Radu G

    Apparently there is no IPMI firmware update for X7SPA-HF either. This is concerning.

    Reply
  41. Arun

    Please check Supermicro\’s FAQ @ http://www.supermicro.com/support/faqs/faq.cfm?faq=18897 Call technical support and request appropriate patches.

    Reply
  42. dg8ngn

    Hi,

    unfortunately my board can only be upgraded to version 2.67. However if you can get shell access you might want to adjust the firewall permanently:

    iptables -A INPUT -p tcp -m tcp –dport 49152 -j DROP
    iptables-save > /nv/ipctrl/rultbl.sav

    Best regards,
    Jann

    Reply

Leave a Reply

WE ARE HERE FOR YOU

WRITE TO US

SEND US AN EMAIL

SUPPORT

LIVE CHAT

CARI.net
8929 Complex Drive
San Diego, CA 92123
© Copyright 2014 CARI.net

SUBSCRIBE

Latest headlines, offers and new services delivered to you daily

WE ARE SOCIAL

You can find us in the different social networks.