Tutorial- Unpacking cable modem firmware

So, what about Your ISP cable modem? Some providers shares a cm firmware, some not and this is the most often situation. There is a couple of ways, to get this piece of binaries. When You have it, You may think what is the next step. A few basic tools (hexdump, strings, dd, lzmadec,  etc.) provided with linux usually make the job. Now, go back to ours binary. First we must know, if it is a zip or lzma package maybe? File command show us, what it knows about it.

$ file cable_modem_firmware.bin 
cable_modem_firmware.bin: data

Nothing. 

Of course, file tool doesn't reveal intersting news at this step, as usually. Ther is no magic bytes recognized on the begining of .bin file.

Next tool is "strings". In example mentioned below, it show us at least 12-chars strings: 

$ strings -12 cable_modem_firmware.bin 
cable_modem_firmware.bin

4wX8<X$2ee`E-

(8i\R#F"\b-#O

t^paU>ETvn{^o

S<W/[d6h)!@|;q

p!&G,C< >       ,r:

h;BVFqX*99YJT

Hro+'xP@LRD1#

/.3jDeFloQ1     s

pDvaz|IR/_[;q-

nRG<5@W9U#?F,>Q

Again, nothing interesting. There is only a couple of no user-friendly strings. This can be encrypted, or compressed file.

Let's try to find LZMA magic numbers. We will change a presentation of bin file to hex (with -C flag, to present an ASCII chars) and direct it to grep. Grep will make filter to show only interesting rows. A "5D 00 00" magic number may indicates a lzma compression.

$ hexdump -C  cable_modem_firmware.bin | egrep -i '5d 00 00 00'

00000050  00 00 00 00 e7 ba 00 00  1f d8 01 85 5d 00 00 00  |............]...|

Thats it! This can be a begining of archive at 0x5C. 0x5C because each byte have their own position in hex notation (0-9 and A-F):

           0  1  2  3  4  5  6  7   8  9  A  B  C  D  E  F

00000050  00 00 00 00 e7 ba 00 00  1f d8 01 85 5d 00 00 00  |............]...|

Let's try to cut a bin file from this position, to other file, because this is the moment when we "have something" :)

$ dd if=cable_modem_firmware.bin of=lzma-file-maybe.lzma bs=1 skip=$((0x5c))
4501058 bytes (4,5 MB), 19,6582 s, 229 kB/s
So we used some parameters for dd tool:
if - input file

of- output file

bs- read 1 and write 1 byte immediately

skip- skip here (to this byte) and start cutting.

Now, we have new file which we can inspect.
$ file lzma-file-maybe.lzma 
lzma-file-maybe.lzma: data

Ok, so the file isn't to be lzma? It is. We must know something about this type of compression.
Let's get some legitimate LZMA file. It can be anything compressed:

$ file lzma-example.lzma
lzma-example.lzma: LZMA compressed data, non-streamed, size 2236416

We see, this file have an information about size, of uncompressed data. Let's look at hex presentation directed to head tool. It show us only 3 first rows:

$ hexdump -C lzma-example.lzma |head -3

00000000  5d 00 00 80 00 00 20 22  00 00 00 00 00 00 00 6f  |]..... ".......o|

00000010  fd ff ff 80 37 3e 18 2f  d7 d6 1e d7 b5 e7 b9 4d  |....7>./.......M|

00000020  1e 73 7f 95 12 a4 bc 86  49 33 cf e3 3d d6 e2 db  |.s......I3..=...|

First 5 bytes have are a magic bytes, and information about compression level 5D000080 and dictionary size.

Next 8 bytes tell us about uncompressed size in litlle endian.

00 20 22 00 00 00 00 00

And we need to read each byte from end, so the 12th becomes first:

00 00 00 00 00 22 20 00, let's check it. We will use an echo to print result as intput base in hex and  directed to bc (binary calc), to convert it to decimal:

$ echo "ibase=16;0000000000222000"|bc

2236416

Notice, that:

$ file lzma-example.lzma

lzma-example.lzma: LZMA compressed data, non-streamed, size 2236416

Is the same value! This indicates that some file-size information are available, some not. This is propably the reason, why FILE command didn't recognize this binary as a LZMA archive.

Go back to our firmware and take a look whats wrong with it.

$ hexdump -C lzma-file-maybe.lzma | head -3

00000000  5d 00 00 00 01 00 20 20  0e 00 0d 3a 28 ab ef 31  |].....  ...:(..1|

00000010  23 33 44 83 db 18 9b 57  12 d9 ed 76 9b d2 8d 4c  |#3D....W...v...L|

00000020  ad 5b 7f 7a 0f 11 d2 c8  a8 77 99 48 98 fb 58 74  |.[.z.....w.H..Xt|

Ok, drop first 5 bytes and look at next 8th:
00 20 20  0e 00 0d 3a 28
backwards:
28 3a 0d 00 0e 20 20 00
and convert it to decimal:
$ echo $((0x283a0d000e202000))

2898643604054482944

Hmmm, this looks a litlle to much for a file system size. There could be a few possibilities:

-real size in bytes was replaced by this bytes

-information about real size was droped.

________________________________________
Let's make an exercise, because true is out there.

1. Download any file, it can be a photo for example.

2. Compress it with lzma 

3. Concatenate with some other non-lzma file. 

4. use file command on it, and next try to decompress it.

$ file photo-01.jpg

export: JPEG image data, EXIF standard, progressive, precision 8, 695x447, frames 3

$ lzma photo-01.jpg

$ cat photo-01.jpg.lzma lzma-file-maybe.lzma >output-file.lzma
$ file output-file.lzma

output-file: LZMA compressed data, streamed
$ lzmadec -d output-file.lzma >export
lzmadec: output-file.lzma: File is corrupt

but:

$ file export

export: JPEG image data, EXIF standard, progressive, precision 8, 695x447, frames 3

Hey, it works! 

Looks like LZMADEC is not worries about incorrect data and it's trying to uncompress files.

So lets try to write some information about uncompressed data size. We'll take 64MB. This is 67108864 bytes:

$ echo $((64*1024*1024))

67108864

In hex this is:

$ echo "obase=16; 67108864"|bc 

4000000

Make it more clearly to read (spaces), because there is to many zeros for me ;)

4 00 00 00

Append zeros at begining to have exact 8 bytes. 0 at begining means this same like in decimal notation.

00 00 00 00 04 00 00 00

check:

$ echo $((0x0000000004000000))

67108864

It's Ok!

Let's asume, that our file have uncompressed size droped. Append this hex result to lzma file, but remember to read bytes from the end. You can use Your favorite hex editor.

It should looks like this:

$ hexdump -C lzma-file-maybe.lzma | head -2

00000000  5d 00 00 00 01 00 00 00  04 00 00 00 00 00 20 20  |].............  |

00000010  0e 00 0d 3a 28 ab ef 31  23 33 44 83 db 18 9b 57  |...:(..1#3D....W|

test:

$ file lzma-file-maybe.lzma 

lzma-file-maybe.lzma: LZMA compressed data, non-streamed, size 67108864

:)

extracting moment:

$ lzmadec -d lzma-file-maybe.lzma > extracted.out

lzmadec: lzma-file-maybe.lzma: Unexpected end of input

We must correct something. A file is extracted and possible to read. But the file size we used is still inorrect. Let's check, how many data lzmadec unpacked:

$ ls -l extracted.out 

-rw-rw-r-- 1 sudokillall sudokillall 23439860 cze 20 15:49 extracted.out

Edited header size to hex in litlle endian and appened to header looks like:

$ hexdump -C lzma-file-maybe.lzma | head -3

00000000  5d 00 00 00 01 f4 a9 65  01 00 00 00 00 00 20 20  |]......e......  |

00000010  0e 00 0d 3a 28 ab ef 31  23 33 44 83 db 18 9b 57  |...:(..1#3D....W|

00000020  12 d9 ed 76 9b d2 8d 4c  ad 5b 7f 7a 0f 11 d2 c8  |...v...L.[.z....|

$ 7z t lzma-file-maybe.lzma 

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18

p7zip Version 9.20 (locale=pl_PL.UTF-8,Utf16=on,HugeFiles=on,4 CPUs)

Processing archive: lzma-file-maybe.lzma

Testing     lzma-file-maybe

Everything is Ok

Size:       23439860

Compressed: 4500974

And finally decompressing without errors with -k flag to keep our original file on disk:

$ lzma -dk lzma-file-maybe.lzma

Quick searching shows, that it is eCos running device

$ strings lzma-file-maybe | grep "chip"

Broadcom BCM%04x Rev. %02x chipnum=0x%x, 802.11 Wireless  NI

Now You can try to find hidden webs maybe. everything is clear at this moment.

$ strings lzma-file-maybe | grep ".asp"

//MoCA.asp

// Quick_setup.asp(Setup,Quick_setup)

// AccessRes.asp (Internet Access Policy), PortrangeTriggering.asp

// WSecurity.asp  (Wireless Security)

// WMACFilter.asp (Wireless MAC Filter)

// AdvancedWSettings.asp (Advanced Wireless)

// Administration.asp

// AppGaming.asp  (Port Range Forwarding)

// Diagnostics.asp

// DMZ.asp

// EditList.asp  (Internet Access PCs List)

// Factorydefaults.asp

// FirmwareUpgrade.asp  ( copy to FirmwareUpgrade.asp )

// Log.asp

// QoS.asp

// Setup.asp

// Setup_DDNS.asp

//RgMediaServer.asp

// Setup_routing.asp

// SinglePortForwarding.asp

// wireless.asp

// WL_FilterTable.asp (Wireless MAC Filter)

// Status.asp

// Quick_Setup.asp

// Setup_DDNS.asp

// Setup_routing.asp

// FixedCPEIpAssignment.asp

// Adiministrator.asp

// vpn_main.asp

// log_data.asp

// RgMacFiltering.asp

// RgParentalBasic.asp

// RgTodFilter.asp

// RgUserSetup.asp

// RgParentalEL.asp

// DHCPClientTable.asp

// Docsis_status.asp

// ChannelsSelection.asp

// Docsis_speed.asp

// Docsis_signal.asp

// SingleForwarding.asp

// AppGaming.asp

// PortRangeTriggering.asp

// DMZ.asp

// ChannelsSelection.asp

// RgIpFiltering.asp

// EMTA_Log.asp

// MTA_Parameters.asp

// MTA_Status.asp

// MTA_EvtLog.asp

// MTA_Batt.asp

// EMTA_Log_line_diag_1.asp

// LineDiag_2.asp

// LanSetup.asp

// FixedCPEIpAssignment.asp

// Administration.asp

// Factorydefaults.asp

// SIP.asp

// Docsis_log.asp

// Docsis_status.asp

// Setup_routing.asp

//Quick_setup.asp

// Docsis_status.asp - Config file

// MTA_Batt.asp

// Log.asp

// Administration.asp

// Rg_UserSetup.asp

// LanSetup.asp & Docsis_status - Time String

// Setup_DDNS.asp

// Rg_UserLogin.asp

// RgVpnL2tpPptp.asp

//L2tp.asp

// RgVlanSetup.asp

// Wireless.asp

// Docsis_system.asp

// LanSetup.asp

// Setup_DDNS.asp

// QoS.asp

// LanSetup.asp

// Docsis_system.asp

// Voice_state.asp

// Administration.asp

// Iptv.asp

// ChannelsSelection.asp

// RgVpnL2tpPptp.asp

// Quick_setup_coh.asp

//Docsis_signal.asp

//Wireless.asp

//WGuestNetwork.asp

//wlanScanPopup.asp

// Docsis_status.asp - Codewords table

// MTA_CallLog.asp

// LanIPv6Setup.asp

Thats all.