Thursday, July 13, 2017

Reverse Engineering Hardware of Embedded Devices: From China to the World

This article covers some basic hardware reverse engineering techniques on PCB-level, which are applicable to any electronic embedded device to showcase how to analyze a previously unknown (to the researcher or public white-hat community) hardware device. SEC Consult operates a dedicated Hardware Security Lab as part of its SEC Consult Vulnerability Lab. The presented material is a glimpse into ongoing research at the SEC Consult Hardware Security Lab and pentests that involve hardware hacking techniques.

Nowadays, we are living in a world dominated by embedded systems. Everyone can be spied on through various channels. Routers, IP-cameras, phones, and other embedded devices are affected by security vulnerabilities and are therefore easily hack-able. Recent outbreaks of Mirai and other IoT-based malware reinforce this observation. To audit the firmware of such a device in-depth - either out of curiosity or for customers as part of increasingly mature security programs - we have to disassemble it and find debug interfaces. Only a debuggable and running system can reveal all of its secrets. This process is invasive and usually leads to damaged devices – hence, more than one device is mandatory for in-depth analysis!

How to proceed with known devices

A quick workaround for avoiding a costly analysis of embedded devices are alternative firmwares. Large companies can easily run acceptance tests of a vendor-built firmware by security consultants to minimize dangers of such systems before they buy a bunch of IoT devices. But there are only few alternatives for users and hobbyists. They can install alternative firmware from third parties like OpenWRT. This firmware does not always have a better performance than the original one (because of the proprietary chip-sets) but it is often a more secure alternative.
When we are charged with a security audit of a stock firmware from a known device, the hardware analysis part is often reduced to a online search in some relevant forums (mostly the OpenWRT Wiki).

Hacking hardware – identifying debug interfaces for undocumented devices

It is mostly impossible to flash a completely unknown device with generic open firmware. Therefore, we have to dissect it in order to examine the used firmware and test it for security vulnerabilities. However, reverse engineering of hardware, especially with proprietary chip-sets, does not sound really easy, but some basics can be done by most hobbyists. A scanning electron microscope (SEM) is in many cases not necessary to identify first vulnerabilities. In our example, we present a Broadcom System on Chip (SoC), which is used as a core element for the wireless repeater Belkin F9 K1106as. This ancient device is not our particular point of interest but its chip-set is used in many other devices, which are supported by the community. By getting the pinout of the used Broadcom SoC, we receive the power to reverse engineer all devices, which are based on this particular SoC. After opening the device, we can see the PCB. Our first goal is to get serial access to the device in order to gain a shell or at least access to the bootloader.

Belkin F9 K1106as PCB and its different modules

A simple way to identify the modules under the metal shields is to remove it.

Opening an ESD-shield with an electronic-pliers

To reverse the pinout of the debug interfaces hidden on the PCB, we do not need to remove the metal in all cases. The exposed UART connector helps us for initial debugging since we got a root shell on this device directly by connecting to it with an FT2232H board using 115200 Baud 8n1 as terminal parameters.
UART can be found relatively easy in comparison to other interfaces. There are only two pins, transmit and receive. When a quick level switch in strobes is performed on a trace/pin, which has the voltage level of a power supply while it is idle then this is an indicator for a possible serial communication. Serial communication does not always mean UART, but that helps to limit the possibilities. Grouped pin (3-5 pins) are very often a UART connector, which is used during development.

A popular pinout is ( GND | RX | TX | VCC ) or ( GND | TX | RX | VCC ).

But where is JTAG? This standard enables a developer or a skilled hacker to control the CPU like a puppet and debug the SoC during runtime, program and read the whole flash memory and perform self-tests. This question can be answered by using a brute force tool for JTAG. SEC Consult developed its own tools (hardware boards & software) with additional functionality for that purpose but there are some projects available online, which can also be used for that. The TP10X pins are guessed to be JTAG because there are usually 4-5 pins for that debug port.

Brute-forcing the JTAG pinout

Hardware analysis tool "SEC Xtractor" developed (HW & SW) by SEC Consult Vulnerability Lab

After bruteforcing we got the pinout! The UART connector was also outlined for the sake of completeness.

JTAG pinout

As JTAG adapter, the cheap FT2232H mini module was used. To make it even more easy, an additional adapter-board with labeled headers is used.

Connecting to JTAG via a FT2232H mini module

A quick connection attempt with OpenOCD gives us the following result:

OpenOCD output

The chip seems to have an instruction register width of 32 and the Chip-ID of 0x2535717f. All we knew before, was that the Broadcom SoC was labeled with some letters (BCM5358UB0KFBG). Now we have much more information – but only for this specific device. The JTAG port can now be used to control the chip and gain access to the system on the lowest level.

Hacking hardware – grab the firmware

To complete the information pool, which can be collected for this SoC, a firmware dump can be done as a last step. A Macronix serial flash can be located on the back side of the PCB. These serial flashes can be read with flashrom and an FT2232H board in few minutes. A quick look on its datasheet reveals the pinout.

Macronix SPI-flash pinout

After unsoldering the chip, we placed it into an adapter and started flashrom:

Reading the SPI memory with flashrom

The dump contained the whole program memory and the permanently stored information (NVRAM). Writing this flash with flashrom and soldering back the chip is another possibility to "flash" the device with own firmware. By calling "binwalk -A dump.bin" we receive many "MIPSEL" (MIPS little endian) instructions, which lead to the assumption that this Broadcom SoC is based on an ordinary MIPSEL32 CPU. A serial flash in a SOP package is one of the flashes, which are the easiest to dump. More challenging memory chips are NAND and NOR flashes, because of the interfaces, the tiny packages and the pin count.

A first analysis of this memory-dump by the IoT Inspector reveals some initial security vulnerabilities and a bunch of interesting information about the used software inside the device. Since we analyzed an old device, some of the vulnerabilities refer back to 2007:

Excerpt from IoT Inspector report

Hacking hardware – reverse the SoC pinout

As we mentioned before, getting the pinout of the SoC would enable us to reverse engineer any device using a BCM5358UB0KFBG. At this point we are forced to remove the BGA with hot air from the PCB in the most cases (chip-off). This can be done with a SMT rework station.

Hot air rework station

Belkin PCB without Broadcom SoC

Following the traces, reading the datasheets of the other chips, doing some measurement between the pins and ground/power (GND/VCC) and some logical deduction skills lead us to the following partial pinout for the most important pins of the BCM5358UB0KFBG:
Partially reversed pinout of BCM5358U0KFBG

It was obvious that the whole BCM535x chip family have a similar pinout after a little search online. The web site collects information about computer hardware, and of course Broadcom SoCs.
There we found the following entry:

A quick look on pictures of other routers with these related chips shows that the pinout seems to be similar to the other types (BCM5357 / BCM5357B0 / BCM5357C0 / BCM5358 / BCM5358U). During the journey from one Broadcom base router to the other, striking resemblances between some devices were observed.

The wikidevi website contains much information about devices but besides that, the manufacturer is also specified very often. China seems to be the only country where all routers come from.

Hardware hacking – supply chains from China

All these Chinese electronics are often produced in the same factories. The chips, the peripherals and the routers. As a consequence, many products are from the same type of quality. The same engineering companies are also producing for competitive vendors. Just compare the two internal photos of an ASUS RT-N53 router and Belkin F9 K1106as wireless repeater. The labels are suspiciously equal.

ASUS PCB, Source:
Belkin PCB

Coincidence? Very unlikely, because the labeling is per default chosen automatically by the ELCAD (electronic computer aided design) program. Nobody changes the labels when he isn't forced to do that. Under the metal of the Belkin repeater picture are more similarities, that means the same schematic was reused for ASUS and maybe for many other devices from other vendors. Therefore, the same ODM (Original Design Manufacturer) was charged with the manufacturing and the branding to Belkin was done later.
This is not an unusual way to manufacture embedded devices, especially for companies from the US. The firmware is also often based on standardized or self-developed SDKs of the manufacturer in Asia.
The increased risk of backdoors is the logical consequence - as we saw in the past:

This research was done by Thomas Weber on behalf of SEC Consult Vulnerability Lab. SEC Consult is always searching for talented security professionals to work in our team. More information can be found at: