Analysis of large binaries and games in Ghidra-SRE

Ghidra is a free and open source reverse engineering suite. It is flexible with scripting and plugins and can be used for almost any architecture. If one does not exist you can add it yourself, if there’s a bug you can fix it yourself, which is the beauty unlike competitors such as IDA Pro. (Did I mention it was free as well???) I have been checking out Ghidra since the initial public release and for “real work” situations it was less than ideal. I kept watching process on the bug tracker, and noticed that 10.x had gotten many improvements and bug-fixes over the 9.x release and thought to give it a second try.

The initial results were not spectacular, running into the same Swing timeout errors (which can be solved by looking here), and overall taking 24+ hours to analyze a 300MB executable with symbols just to finally crash about about a day and half later, leaving me back at square one. This is the experience of many people that I have talked to who are into games reverse engineering where binary sizes can balloon very fast, even without symbols. I will go over the steps for PC executables that are large without symbols, but the process should be about the same for other platforms (PSX, PS2, PS3, PS4, PS5) provided you have the loaders/scripts and follow the instructions for them.

Prerequisites

Before beginning we will need a few things, first you will need to have Ghidra downloaded and extracted, at the time of writing this is version 10.0.1. Since most games are written in C++ and may have some form of RTTI (Run-Time Type Information) having a plugin to handle this is ESSENTIAL. Luckily there is, made by someone who is one of the most amazing developers that contributes to Ghidra that I have met astrelsky! His work in this area is some of the greatest I’ve seen come out of the OSS community. We will be using his plugin for the C++ Class Analyzer which handles a plethora of C++ specific things. You can download the C++ Class Analyzer on the releases section of GitHub. At the time of writing 10.0.1 is not “supported”, but it is just a very minor tweak to get it to work and will probably be updated in the future. Open the download zip file (at time of writing, ghidra_10.0_PUBLIC_20210623_Ghidra-Cpp-Class-Analyzer.zip) in your favorite archival tool (I use 7-Zip) and find the file extension.properties in the folder Ghidra-Cpp-Class-Analyzer and double click to edit it.

extension.properties file in the archive

You will need to edit the version information to say version=10.0.1 instead of the default version=10.0 save the file, and 7-zip usually will ask if you want to save to the archive, select yes.

Updated version information

Launch Ghidra and click on File -> Install Extensions click the + and select the Ghidra-Cpp-Class-Analyzer zip file.

It should be added to the list and check the checkbox to enable it and restart Ghidra.

Once Ghidra is restarted, create a new project, give it a name (I use GhidraTutorial) for this and drag an drop in your binaries. If you are loading PS1/2/3/4 ensure that the right architecture is selected. For PC binaries it usually detects correctly on the first try. In this case I am using the game Horizon Zero Dawn GOG release on PC. Allow the import to complete, but do not launch the CodeBrowser in Ghidra quite yet.

Import prompt for Horizon Zero Dawn

You will need to close Ghidra at this point because the GUI for CodeBrowser when analyzing causes nothing but issues. This is where the normal usage of Ghidra will come to a pause. We will be using the Ghidra Headless Analyzer which is a CLI tool for Analyzing or Importing files into a Ghidra database. Open up a new Terminal window (I use Powershell on Windows, Bash on Ubuntu/Linux) and navigate to the Ghidra support directory. In my case this is PS C:\Users\godiwik\Documents_tools\Ghidra\ghidra_10.0.1_PUBLIC\support .

Since we have already imported the executable with the correct architecture we will just need to analyze using the -process flag. The command we will use is .\analyzeHeadless <directory to the Ghidra database> <DatabaseName> -process "*" -recursive which should handle analyzing anything we have already imported. If this does not work for you, try adding some to the wildcard such as Horizon*, or if you have it in a folder, supply the folder name with a wildcard behind.

The example that I am using is .\analyzeHeadless ../../ GhidraTutorial -process "/*" -recursive since my Ghidra database (GhidraTutorial.gpr/GhidraTutorial.rep) is one directory above the Ghidra root directory. If no errors appear, just go take a bathroom break, get some coffee, play with your favorite pet, feed your kids whatever you have to do because this will take awhile…

In process of analyzing… Waiting for completion

After everything is all completed, you can re-open Ghidra again. And then double click on the executable to open up the CodeBrowser.

Headless Analyzer Completed in ~30m

After this point everything should be analyzed, and the C++ Analyzer should also have been ran during the headless analyzer. If you aren’t seeing good results, then click Analysis->All Open->Deselect All->Windows (or GCC) C++ Class Analyzer (prototype), increase the Decompiler timeout from 30 seconds to 120 or 240, and click Locate Constructors then click Analyze. Wait for that to complete and everything should be ready for you to explore!

Additional C++ Class Analyzer options

All Done!

I hope that this was able to help someone, I know I was frustrated with attempting to use the GUI for large binaries, this is a workaround that is pretty awesome and you could even automate this for automatic analysis.

I’d like to thank OSM, Seremo, Shiro, krystalgamer_, Z80 in no particular order for taking interest and or helping proofread.

kiwicon – Modern Warfare Remastered Console Source

Just something I threw together in 5 minutes. Do note that a bunch of dvars have been hashed, and no longer are able to be changed by their original commands. Some work, some don’t. There is no output from the game, but you can input console commands.

Some commands that work:

  • spdevmap
  • cg_fov
  • cg_fovScale
  • jump_height
  • g_gravity
  • g_speed
  • fx_enable

 

kiwicon for Call of Duty: Modern Warfare Remastered SINGLEPLAYER Source Only (version 1.0.0.1 ONLY!!!!!)

Rime: November Progress Report – Preparing and Automation

It’s been a wild few months, and I would like to thank all of the testers of Rime that have been reporting bugs, suggestions and usability problems.

Rime: Operation Metro WIP

One main issue that I have with Rime is the lack of automation. It had gotten an auto-updater quite awhile back, but it was very primitive and needs an overhaul so I can push updates faster to the testers and eventually the entire community. With this documentation is key for such a complicated and large project and before hand-generating the documentation or even using a tool like DoxyGen wasn’t exactly up-to-par. I’ve spent the last few days creating DokuGen which takes the C# assemblies and XML documentation and creates a DokuWiki format output so you can just drop it in your dokuwiki/data/pages/ directory and have it reflect instantly online!

Continue reading “Rime: November Progress Report – Preparing and Automation”

Tutorial: PlayStation 4 development on Windows

If you are like me, you hate developing on Linux. Nothing beats Microsoft’s Visual Studio, nothing. Recently with the Windows 10 Anniversary update you have WSL or Windows Subsystem for Linux, as excited as I was for this to come out Visual Studio is a “slow” adopter of this technology and I currently spent many hours poking and tweaking things until I got them right. So  here it is, a full tutorial on how to get PlayStation 4 development going on Windows.

Note: This is not using any of Sony’s official software development kit, and is free for anyone to use.

Continue reading “Tutorial: PlayStation 4 development on Windows”

Rime: June Progress Report

3 Months Later, much progress has been made. Here is another dump of the git repository change-log. I won’t be able to continue development on Rime until after July due to myself switching apartments again (I move every year =[ ). With that being said, I have set up a Continuous Integration server for Rime so current alpha testers can get bleeding edge builds if needed. There’s only one problem, deploying the builds 😛 I will be writing an open source deployment tool that I will upload on my github when it is completed.

With that being said, I have a software recommendation, If you haven’t heard PVS-Studio is a hell of a tool catching many programmer errors and mistakes. Rime didn’t suffer from many errors, only 1 high level issue, ~40 medium (mostly serialization issues), and 200 low priority ones (caused by floating point arithmetic) Read more to see full change-log.

Continue reading “Rime: June Progress Report”

Rime: March Update Report

Good news, Rime is becoming more and more functional every day that it gets worked on (busy with school primarily). The last month has been both productive and a ton of work has gone into the project. Here are some of the changes from the git history.

Implementations:

  • Implemented “-cleanup” option for the auto-updater in order to clean up old files on next launch.

Additions:

  • Added more editing abilities to the PartitionBrowserControl.
  • Added logging of version on startup (the UI is currently broken and doesn’t show it)
  • Added “Dashes” member to GUID in order to get the dashes format as well.
  • Added checking of instances on CtrRef ToString in order to show “*null*” if the partition or instance is null. (may change to partition only)
  • Also made it where if you double click on an CtrRef it will automatically open up a new editor with that instance loaded.

Bug Fixes:

  • Fixed bug with the PluginManager in where multiple instances would be removed due to old-style name checks. This was resolved by properly removing that one instance.
  • Fixed crash in the OutputPlugin due to the parameters being null.
  • Fixed threading related crash on updating the UI when a new update was made available.
  • Fixed crash in the logger when parameters were null.

Changes and Corrections:

  • Corrected List DataTemplate, by changing the member type from TreeView to ItemsControl.
  • Refactored update system.
  • Major refactoring to allow plugins to load properly. Also disabled the Theme in order to properly see what is going on in the ebx editor.

Thanks to all of the testers we have, and special shoutouts to Powback for assisting with the documentation and preparation for the public build.

For usage with VeniceEXT
For usage with VeniceEXT

Rime: January Progress Report

Time flies, I really thought I wrote the last entry last month, but apparently its been quite awhile. Here’s a bit of a status update…

I’ve been swamped with work and fixing issues and researching more cool things for Venice Unleashed. So work on Rime has been slow, but after reading the last blog post, there has been a ton of work behind the scenes that have been done with Rime.

  • Converted quite a few parts to use the MVVM for raw speed and performance
  • Added chunk reading via the VFS
  • Implemented a “new” ebx reader written by NoFaTe almost 3 years ago (+- a year)
  • Corrected issues with said EBX reader
  • Implemented ebx VFS and cross-ebx reading
  • Ebx reading and parsing is all async and threaded now and done in the background without affecting initial map loading times. (Maxed out my laptops i5 for ~9s before finishing)
  • Updated all projects to Visual C++ 14.0 and .NET 4.6.1 (fixes all UI issues using C# 6.0)
  • The bundle browser is now sorted alphabetically
  • Updated bundle browser plugin to use the VFS instead of manually parsing bundles itself.

And many many more changes to increase performance etc.

Rime: September Progress Report

Hello there, I know it’s been awhile since I’ve updated this and a few people thought the project was dead (I don’t know why, but they did lol).  Here’s a quick update of what has happened over the last few months. Most of the work has been done by the other member of Venice Unleashed NoFaTe!

Fixes:

  • Fixed a crash that would occur due to plugins without UI’s being added as UI elements.
  • Fixed various threading issues and collection changed issues due to the asynchronous nature of Rime.
  • Fixed a ton of issues dealing with the UI and how they interface with plugins.

Updates:

  • Updated WebLink security (cross communication to modders link to prepare for the community)
  • Updated the BinaryReader/Writer with more efficient versions.

Refactoring:

  • Removal of a lot of deprecated, unused, commented out code.
  • Deprecated all previous (semi-functional) mounting code (due to the new readers/virtual file system).
  • Moved a TON of shared code (used between Lib/Rime itself) to the RimeCommon and restructured it.
  • Moved a TON of Frostbite specific utilities and classes to RimeLib.
  • Refactored the plugin system again (Will have to update the RimeExamplePlugin github soon).
  • Refactored the handling for Frostbite Guid class.

Features:

  • Added the ability to auto-start certain plugins on launch
  • Added all available plugins to be launched from a menu if you are unsure how to launch them (only ones with UI).
  • Added auto-detection of game builds and information.
  • Added BuildInfoProxy for x86 and x64 versions of Frostbite to read out the engine’s build info natively.
  • Added detection for Frostbite2014.4 (Star Wars Battlefront Alpha Trial).
  • Implemented a new Virtual File System.
  • Loading of patch files and patched superbundles completed.

Changes:

  • Added Rime to a CI solution to prepare for testing builds for those who have already been selected for testing.

 

H2UnlockPatcher: Halo 2 Beta Unlocker

Here’s a little side project that I didn’t have very much time to fully complete. It is a unlocked Halo 2 Beta! Currently when you sign in, just select “do not sign in” and then from there you can use System Link, Split Screen, Xbox Live (arranged game).

Just a note, the System Link is missing some code to allow xbox’s to see each other, I am slowly working on a fix for this. I spent a few weeks a few months ago working on my own xbe (Xbox Executable) patcher and patch creation system, which allowed me to build patched executable really quickly. In this time I recoded the entire main menu in assembly to “unlock” everything, I even had force loading which is disabled in this patcher due to unforeseen crashes.

Features List:

  • Removal of the EULA screen
  • Removal of all watermarking
  • Removal of all main menu limitations (if code is still there)
  • Re-linking of System Link, Xbox Live, and System Link screens.
  • Controller input hook (not active)

Pictures:

Unlocked Main Menu
Unlocked Main Menu

Halo 2 Beta Ingame

You will need the executable matching SHA1: C42E58FA8EEDCD4F46707C2FD0D8CEFCC61A6410

This will not be redistributed.

Download (Requires .NET Framework):

H2UnlockPatcher

peace,
-kiwidog