MindShaRE: Hardware Reversing with the TP-Link TL-WR841N Router
September 03, 2019 | Vincent LeeMindShaRE is our periodic look at various reverse engineering tips and tricks. The goal is to keep things small and discuss some everyday aspects of reversing. You can view previous entries in this series here.
In early 2019, we received a bug submission from a new researcher affecting the TP-Link TL-WR841N Router. While this vulnerability is still in disclosure phase, we would like to share lessons learned when we were vetting this submission.
TL-WR841N is an inexpensive ($18 USD) and very popular router on Amazon.com. It was one of the top 10 best-selling routers on Amazon.com at the time of the submission. Being a popular router, TP-Link has released multiple iterations on this device over a span of several years.
OpenWRT support for old iterations of this router could be one of the many reasons this router is so popular. OpenWRT is a Linux distro for embedded devices. Many how-to guides and tutorials covering how to set up hardware debugging interfaces for old iterations of this device exist on the Internet. However, the number of tutorials on newer iterations has tapered because OpenWRT has dropped support for later revisions of this device.
In this blog post, we will be using the TL-WR841Nv14 router and outline the steps required to set up remote debugging on this device with BusyBox and gdbserver.
Figure 1 - The TP-Link TL-WR841Nv14 Router
This router uses IEEE 802.11n technology and provides up to 300 Mbps throughput. It has two non-detachable antennas. The case of this router is held down by two screws from the bottom of the case only. There’s no screw hiding under the label.
Figure 2 - Bottom of router
In addition to the screws, the case is securely fastened by 7 additional clips. We spent quite a while undoing the clips. The clips are quite robust and resist forceful entry.
Figure 3 - Internal view of the router
With the case open, we can see all the major components: a MEDIATEK MT7628NN system on a chip (SoC) with a MIPS24Kc processor, a ZENTEL A3S56D40GTP-50 256MB SDRAM, and the GigaDevice 25Q32CSIG SPI Flash Memory.
Figure 4 - Close up view of the MEDIATEK MT7628NN SoC
Figure 5 - Close up view of the Zentel SDRAM
Figure 6 - Close up view of the GigaDevice SPI Flash Memory
The board is held in place by 4 registration marks, without using any screws. The lower surface of the PCB contains only soldering pads for through-hole components. Surface mount components are populated exclusively on the top surface. These are all design details that indicate the PCB is highly optimized for cost reduction.
Figure 7 - Lower surface of PCB
Near the front of the board there is a UART debugging interface at J1 near the SPI flash. The circuit designer kindly labeled the RX and TX pins. A quick measurement on the Vcc pin with a multimeter suggests that it is a 3.3V interface. We soldered on some pin headers for convenience.
Figure 8 - Close up view of the UART debugging interfaces
Getting shell
In our lab, we only have a 5V USB to Serial TTL cable. Luckily, we also have a BSS138 based bi-directional logic level shifter. The logic level shifter allows our 5V cable to communicate with the 3.3V UART interface by shifting the 5V signals from the USB to 3.3V signals that the router requires. Let’s connect the RX, TX and Ground pins of our 5V cable to the 3.3V UART interface of the PCB through the logic level shifter. We intentionally left the Vcc pin unconnected to avoid damaging the target board.
Figure 9 - Block diagram of the serial communication set up
Using PuTTY, we connected to the serial interface with the baud rate of 115200.
Figure 10 - Output from the UART
Success! We now have output from the UART of the router. But it seems that the router is not responsive to keyboard input. What is going on? Let’s figure it out! Here are the steps I took to troubleshoot the setup:
1 - Verifying the serial cable
It is possible that the TX connection of the cable is broken and not sending the signal to the router. To verify the cable, I shorted the RX and the TX pins of the cable with a jumper wire, and used PuTTY to see my own keystrokes appearing on the screen. Doing this helped us verify that we have a working USB to Serial cable.
2 - Verify the logic level shifter works
When working with hardware, anything is possible and nothing is guaranteed. It is possible that the logic level shifter has a weak solder joint and is not transmitting the TX signal to the router properly. We can verify the logic level shifter’s functionality by observing the logic level shifts with an oscilloscope. We set up CH1 to probe the TX signal from the cable, and CH2 to probe the output of the logic level shifter. For this experiment we connected the level shifter output to the oscilloscope’s CH2 only, and disconnected it from the router.
Figure 11 - Block diagram of the debugging set up in step 2
We then sent the space character using the PuTTY connection to create a signal and got the following output:
Figure 12 - Initial oscilloscope output
From the above diagram, we can see that the waveform of CH2 matches CH1 and has the expected level-shifted signal. We should also note the rounding of the rising edge of the CH2 signal, which might become a problem later on. Additionally, the digital oscilloscope correctly decoded the UART signal and showed the space character (ASCII 0x20) being transmitted. This step helped us verify that we have a working logic level shifter.
By this time, we knew that the cable, the logic level shifter and the router all worked as intended, but somehow, they didn’t work when connected together. This was the appropriate time to doubt your self-worth, and post the following meme to Twitter:
3 – Debugging the whole setup
After completing the above two steps, we could tell that all the components were working separately but not in conjunction. This showed that there’s a systematic error within the setup. We reconnected all the parts following the schematic in Figure 9 and continued to investigate with the oscilloscope. This time, we set up CH1 to probe the TX signal from the cable, and CH2 to probe the signal at the RX pin of the router. Sending a space character in the PuTTY terminal yielded the following output from the oscilloscope:
Figure 13 - Second oscilloscope reading
CH2 is still shifting levels, except logical HIGH is now only 1.52V! We should investigate further…
4 – Probing router
If we take a closer look at circuitry around the J1 UART header, three SMT (surface-mount) resistors near the pin header can be found, namely, R18, R87 and R89. Using the continuity test feature on the multimeter, we determined that R18 is a 1kΩ pull down resistor connected to the RX pin. This is our culprit!
Not wanting to de-solder the minuscule SMT component, we accepted the risk of damaging the cheap router and directly plugged the 5V RX signal from the serial cable to the RX pin of the router. And voilà! A fully functional, bi-direction serial terminal appeared.
Figure 14 - Bi-directional serial terminal
Here’s a look at the completed hardware setup (minus the oscilloscope).
Figure 15 - Complete hardware setup
Poking around
As we poked around in the shell, we saw that the firmware is Linux-based and is running a very old Linux kernel.
Figure 16 - Showing the Linux version
To conserve space, many of the Linux utilities have been removed from the device. The shell on the device is very limited, and the on-device BusyBox utility is also very stripped-down.
Figure 17 - Available commands on the router shell
Uploading BusyBox and gdbserver
BusyBox is a single, size-optimized Linux utility package. Due to its small footprint, it is very popular for embedded Linux devices. The shell we obtained from the serial interface has limited functionality as well as a stripped-down version of BusyBox. We can make our lives much easier by uploading our own fully-loaded BusyBox via the existing TFTP client on the device. You may find a precompiled little-endian MIPS BusyBox binary on the official website.
Using the mount command, we can see that /var is the only available partition to store our BusyBox binary. Since /var is a ramfs type filesystem, the file we write to it will be gone when the router reboots.
Figure 18 - Determining the writeable locations on the device
We can transfer the BusyBox and gdbserver binaries onto the router using the following commands:
Figure 19 - Installing BusyBox and gdbserver
After adding execution permission to the binaries, we can execute our version of BusyBox and have access to more Linux utilities.
Figure 20 - Available commands with full BusyBox installed
Setting up GDB
Let’s say we want to debug the dropbear SSH server on the router. Before we launch the GDB server, we should extract the target binary and save it to the machine that is going to run the GDB client. This will help in the step of loading debugging symbols into the GDB client.
After finding out the PID of the dropbear SSH server, run the following command on the router to start a GDB server:
gdbserver localhost:23947 --attach <PID>
On our GDB client machine, we’ll run the following command:
gdb-multiarch -x dbgscript
Figure 21 - Output from GDB Session
Contents of our gdbscript follows:
And there you have it! We have now successfully connected to the gdbserver. With some luck and hard work, you should be able to find some bugs and make something like this happen:
Figure 22 - Output from GDB Session Crash
Conclusion
Many researchers hesitate to delve into hardware vulnerability research. One of the many reasons is that things may not work as expected despite following the steps outlined in the tutorial exactly. When this happens, first verify each component works individually. Then, from a known working component, verify connections by measuring continuity, voltages, and signals. Work your way out from the known working component and hopefully you will soon discover the culprit. Finally, having access to an oscilloscope is instrumental in troubleshooting hardware. Getting a good one will serve you for many years to come.
I hope to see your hardware-related submissions in the future. Until then, you can find me on Twitter @TrendyTofu, and follow the team for the latest in exploit techniques and security patches.