Make Analysis Great Again (or never type the same thing twice)

TL;DR: If you ever write the same thing twice, especially in a console, you should automate it. MAGA automates and standardizes a lot of the command-line tools offered in the SANS FOR408 (now SANS FOR500) course.

The Problem

Not everyone loves the command line and many feel very insecure or nervous about using it. However, a lot of great tools are command-line only which creates a problem of either having to get really comfortable with the command line (the correct answer) or find a way to type as little as possible (which also minimizes mistakes). As a general rule of thumb, if you write something twice, especially if it contains half a dozen options of varying length, you should automate it.

A Solution

While taking SANS FOR408 last year, I found a lot of my fellow students fell to one degree or another into the first camp. In order to help fellow students on the final project and to standardize my own approach through labs and the final, I wrote “Make Analysis Great Again” (MAGA) a simple batch script to automate most of the initial interaction with the solid command-line tools offered on the Windows SIFT Workstation.

The goal wasn’t to have a one-click answer, but a uniform starting point that removed a LOT of typing (where mistakes could be made by messing up output formats, for example). After hitting go on this script, students could move on to other tasks and then return for the output to feed into the rest of the lab or final. It let students worry about the actual analysis, instead of fretting over messing up the time stamp format between three different runs. I still do all of my programming from the command line and am a firm believer that forensic examiners need to know it, but for the sake of getting the most out of FOR408, it was more important to me that they know what happened after hitting go.

Based on the 408.16.2 SIFT Workstation, MAGA kicks off the following tools:

MAGA also copies a lot of the key files to be examined to one folder and prompts to launch Woanware’s USB Device Forensics tool at the end to standardize an initial analytic process. All of the output goes into an output directory structure that, based on forensic examiner input, creates a folder hierarchy to keep all the users ripped from a particular machine grouped together with a similar structure. These folders are what contain the output files from all the great command line tools, as well as the copied input files to let the forensic examiner dive right in to their analysis.


I had intended to write something about this script sooner, I just forgot to (I also forgot to submit my pull request for some additional RegRipper functionality until writing this). This is quite likely out of date now that SANS FOR500 is a thing, but I’d love to hear from someone in that course if this works on that SIFT Workstation as well. If not, it should be able to easily be adapted for that workstation and the tools loaded onto it, work I’d be happy to do!

NB: I want to stress that this tool is NOT intended to replace learning how the underlying tools work, or what the different command line options do. It is intended to be a first start that will run a good bit of the tool suite in a standard way that will allow analysts to immediately start comparing apples to apples and be able to use more tailored commands to progress their work in whatever direction the analysis takes them.

Mining Hidden Gems WIth SQLite Miner

Read More

TL;DR: Data hidden in SQLite may not be human-readable and SQLite Miner will help you find the hidden gems inside those databases. You can use either the latest stable release or deal directly with the master repository.

The Problem

Mobile applications can store whatever they want in databases in formats that may or may not be obvious to the naked eye. For example, a column may contain compressed data that obscures the plaintext from the forensic examiner’s view, such as Apple iCloud Notes using GZIP for the Note data. Alternately, the file format itself may not be obvious to everyone, such as a JPEG picture without EXIF which starts with the obscure 0xFF 0xD8 0xFF 0xD8. To effectively consider everything that may be in the database of an unknown application, the forensics examiner needs to identify all of these common file formats and do something to make the information readable.

Possible Solutions

For well known applications, relying on your forensic tool of choice may be the right way to go. The downside to this is if the tool does not fully support that database, you may not even know if there is missing information, or if that missing information is garbage, encrypted, or what.

Screenshot of how Oxygen Forensic shows a SQLite file that has compressed information within it.

Another possible solution if the tool you prefer is not parsing the database would be to export the SQLite file and use hexdump to simply eyeball the entire file. While this would allow for rudimentary searches for known magic numbers, it wouldn’t put those entries in the context of their columns to know how to interpret the information. Nor will you want to do this for a file of any significant size.

Another solution could be to export the SQLite file out and open it in some CLI or graphical browser. Then the examiner can eyeball the data and run some specific queries to try to identify these files in the context of their columns. One problem with this approach is that those queries require the forensics examiner to understand the structure of the database (i.e. know which columns to search). Another problem is the sheer number of queries that need to be run to chew through a good number of the file types that could have data stored in the SQLite database.

All of these approaches get exhausting, even if you limit your searches to known test file types you put into the application. This may lead to a forensics examiner trusting their tools or just accepting the existing push-button approach, to the detriment of their investigation.

A Better Solution: SQLite Miner

Continue reading “Mining Hidden Gems WIth SQLite Miner”

There’s Gold In Them There Blobs!

Read More

TL;DR: Apple iCLoud Notes are GZIP’d when stored and this script will decompress them for you.


To set the stage for this post and overall blog, I recently took the SANS advanced smartphone forensics course, FOR585. It was a great opportunity and this flip-phone using Luddite took away a ton of good information. One takeaway was a desire to contribute more to the forensic community and publishing code that I use myself which would benefit others, hence this blog. Another takeaway was that iCloud Notes needs to be better understood, hence this post.

Apple Notes

Apple Notes itself appears to be decently parsed by the major tools so I won’t spend much time on it. The relevant SQLite database (notes.sqlite) stores data fairly simply. Even without any tools beyond hexdump, you can clearly find test notes saved in the database. The below image shows a note containing the text “I moved this note.”

iCloud Notes

iCloud Notes, on the other hand, is not parsed nicely (yet) by anything but Cellebrite’s Physical Analyzer. This is likely because the messages are not stored in plain text. For example, the same note above, when moved to iCloud Notes, looks like this:

Continue reading “There’s Gold In Them There Blobs!”