Category: Modding

Hey,

There have been a few things that have been going on in the last few months. I just wanted to give everyone an update on the current status of Rime and the Venice Unleashed interop. First things first, plugin’s are hard, if you have checked the previous post on plugins and open source, I am making use of a plugin system for all external and internal plugins. That way you, the hobby modder can look at what we have to offer, and the functionality of it and be able to write your own code to interact with the Frostbite engine. That’s all fine and dandy, but I am a security-oriented coder. I have been working on for the last week a system to overlay the plugin interface’s to allow the code to be sandboxed, only allowing read/write access to the folder where the plugin resides. This will lessen the chance of someone writing a malicious plugin to steal sensitive information, or execute code that could potentially be harmful.

The way that I was going to get around these issues, is force the sandbox to run as lower-level privileges than Rime itself. (Even though Rime does not require to be ran as administrator, some people will anyway). That a malicious plugin writer won’t be able to gain privileges via Rime. And all of this is where the problem comes in, I have been debugging and researching the best approach to this in C#, and there just isn’t one (publicly at least). So I am making the decision to scrap the secure plugin system for the closed/open alpha of Rime until we get a functional system for the beta. With that being said, I highly recommend that any plugin creator open source their plugins so people can compile it from scratch. If you download a pre-built executable, use caution and only download from trustworthy people.

Fixes:

• Fixed incorrect TextureHeader structure
• Fixed DXT1-3 native C# processing.
• Fixed EBX reading code
• Entire application structure refactoring for prep for release (removing unused/old code etc, that’s why we have source control ain’t it?)
• Abstracted multiple classes to prepare for multi-game Frostbite support.

Broke:

• Internal communication of all parts of Rime. (Yeah… I fudged up bad)
• Plugins in their entirety (which means all the built-in editors for Rime have been disabled as well, Rime’s internals uses the same plugin system)
• DirectX 11 Mesh Renderer (Instead of using laggy WPF for everything)

Additions:

• Battlefield 4 Resource Types
• Proper loading of all game files and resources. (Just like how it gets done in-engine, no longer needs to decompile the entire map/game to mod resources)
• Retrieving encryption information, and real time modding via Venice Unleashed.

Hello all,

Just an update to give some progress on Rime, what’s determined to be the worlds greatest Frostbite tool other than DICE’s internal FrostEd. (endrant.)

There has been some confusion on when, or where Rime will be released. Currently the answers are both, “Soon”. But as a step forward to speed up development certain parts of Rime will be Open Source. This is the first open source project that I will be hosting and hopefully people will be contributing. Now back to that word “certain”… As it stands now, Rime itself will NOT be open sourced, that way there can be tight control over builds, coding style, and management. What is currently open source as of a few minutes ago is the Plugin System, and eventually the RimeLib will be open sourced.

The plugin system will allow anyone to write plugins to interact with new content, or just write better editors than the built in ones if you are so inclined. But the plugins will be called upon each time a certain style of content is requested to be opened. All of Rime’s internal editors, content loaders are all just built-in plugins, so the system has been tested so far as working. With the exception of IPlugin (which is included in the repository) you will not need ANY other source from Rime to get started writing awesome plugins. You have full reign over C# at this moment with plugins so you can load external files etc, just as you would any other C# application.

Now, what is this RimeLib? Well RimeLib is currently the native C# library that was designed to handle all of Frostbite’s various classes and formats. Pretty much RimeLib does all of the hard work for you, so you don’t have to worry about alignment, hex editors, or any of that. For example, loading a mesh into a nice C# class with formatting is only 1 line of code with RimeLib. The library is still in early development phases and has not undergone a huge re-write since it’s first inception in IceEditor Alpha, this will all need to change along with proper documentation for all of the classes and a separation between release and debug builds (release will only have FINALIZED and 100% confirmed structures, methods, classes; while debug has everything including experimental code). You will be able to find RimeLib’s source code sooner than later (Hopefully a few releases into Rime’s Closed Alpha testing).

What are you waiting for? Go check out the awesomeness of Open Source!

This project hasn’t been touched in quite awhile. I last left off with figuring out mesh data and with help from dani we got it all done.

This is a very rough preview, that we can load models and load them into a model viewer all using engine files.

Whats to come is proper lighting, faces, texturing etc automatically from the new backend that Rime has been coded on. Instead of forcing data into a million different formats, why not live edit?

tl;dr; This video was showing Frostbite 2 model data being loaded natively and being displayed using WPF.

Due to the developments of Rime and Venice Unleashed, I have decided to release this after a few days of work. It does nothing but validate the signature with the DICE public key (not provided). It will spit out the output if the signature is valid or not. Source code included but not recommended to use. Just use it for educational purposes. This may seem useless to the 99.9% of you guys who are waiting for Venice Unleashed and Rime, which is true but it helps people reversing the file format and Frostbite(tm) engine.

Venice Crypt Testing Project with Source

Well after a 2 second question with Sir. NoFaTe, I started adding a very interesting part to Rime. This is mainly in-case due to the announcement of Venice Unleashed’s Client-as-Server methods. As most server admins know, moving around 30GB of data for a server install is just too painful. This is due to the engine ignoring a TON of file formats when it is running as a dedicated server. Therefore one thing leads to another and there will be a standalone version of Rime just for re-building all of the maps in a minimal state. aka we go in, throw out all of the garbage that 1. Gets Ignored, or 2. Not used anymore and then run the game again. I haven’t started any testing but it should reduce the file sizes drastically.

Ignored File Types: (Thanks NoFaTe)

• SwfMovie
• MeshSet
• MovieTexture
• RenderTexture
• ITexture
• DxTexture
• SwfTexture
• RenderTexture (again?)
• OccluderMesh
• EnlightenDatabase
• EnlightenShaderDatabase
• EnlightenSystem
• EnlightenProbeSet
• StaticEnlightenDatabase
• VisualTerrian
• TerrainDecals
• ImpulseResource
• (Certain) Sound Files

Well with Battlefield 4 by one of my most favorite studio’s Digital Illusions CE (better known as DICE) releasing within 24 hours has released.. I think I should share some information here, just to be nice to everyone involved.

Before we begin, I’d like to say that this will sound very similar to Halo’s format (Blam Engine) way of explaining things. From content like images and sounds being called raw, metadata being called meta, and Ebx, and other “describing” formats being called tags. From my observations they have seemed to take a more centralized approach to storing all of their information. They have moved all of the raw data, and certain central ebx files into the cascat format. The cascat format had been used in Battlefield 3 base, but it seems like it was neglected after release. Cas stands for Content Addressable Storage and how it pretty much looks is, you have a catalog file which contains offsets, file sizes, and a SHA1 hash of all of the data inside. (And by all of the data, I mean each file that is within a cas file has a SHA1) The way that I have reversed it, it appears that you contact for certain things a CAS handler class which will then look up the file, raw, whatever that is in it based on SHA1. In Battlefield 3, these SHA’s were not in any kind of order and that caused the game to take forever to load on original maps even if you had an SSD because it was re-computing SHA1’s or looking them up in a unorganized manner. This problem most likely still exists in Battlefield4, unless the cas building tool sorted the hashes before hand. I have not actually looked into if hashes have been sorted.

In the BF4 Alpha Trial, Frankelstner noticed there was a different ebx header for the ebx data. Ebx data is Binary XML in a nutshell. I took a look into the matter and a quick IDA review showed that some of the information in the header had changed sizes, by that I mean it went from a 32bit integer, to a 16bit integer in certain locations. Quick modification to previous source code got us pass that. Even better yet, upon analyzing the dumped Alpha Trial executable, I noticed that certain things that I could rely on for finding the ebx process function had changed, the most notable is the string “Bad EBX Partition Header” or something similar to that. So that got me a bit interested. After a few hours up comparing functions to the previous ebx reading code I was able to find and update most of the functions. DICE still uses a giant switch statement (Thanks Hex-Rays 🙂 ) as a partition reader, which whoever wrote that should be spanked (not really, but its frustrating to reverse). But there was something different about it all.

In Venice (Battlefield 3) and Medal of Honor Warfighter and previous Frostbite 2 based games, the ebx reading code (binary xml) is called StreamingPartition blank, where blank would be Reader, Array, etc. In Warsaw (Battlefield 4) its just called EbxPartition blank, where blank means the same as before. There are minor changes throughout but it caused me to separate some of the code into 2 different sections within “IceEditor”. There is a reason I put IceEditor in quotes for now. Along with ebx code changing certain things in the header, having a 64-bit alignment also became an issue. Its not a major issue because well screw alignment.

tl;dr; IceEditor is now called Project Rime and will probably be a open source project near its completion. Since once all of the base framework is finished it won’t be that hard to properly add support for new games using the Frostbite engine. There will also be backward compatible support eventually if someone wants to step up for the project. (BC2, BC1 support) There has been many many hours lost in reformatting without backing up the project, source control servers deciding to go poop and losing all data. lada lada.

Here we go again! This is pretty much an update on what I have been doing over the last few months. Long story short, “real life” sucks. There is nothing but a bunch of work and school keeping me busy. I currently don’t have internet, television, or phone in my current apartment due to it being temporary for the next few weeks. Then I’m moving into a more “permanent” residence.

VeniceFX Progress:

• All players now sync correctly (less/no jumpiness via spectating at a distance)
• All sounds work properly when in free-cam mode
• Spectator mode re-implemented
• Xbox 360 controller support should work…(untested, no controller)
• Fixed bug preventing spectating a player
• Commands can be saved in a text document in BF3’s root directory for tweaking VeniceFX colors/plugins/commands
• Fixed click breaking everything
• Removal of the need of the Alt-Tab “fix”
• Ray-casting added to “jump” to a player
• First person spectating is experimental and may not work/crash
• Updated to work with the latest release.
• v4 uses 100% engine functions (using 100% engine functions means the code base shrank a great deal)
• Spectator text is now pretty 🙂

VeniceUnleashed:

• Soon “TM”
• Closer to completion than not
• Still being worked on
• Sorry for not more information

IceEditor:

• Backend code base has been re-designed to be closer to official DICE
• Will probably be re-done one more time to support BC2 and BF4
• Light BF4 support added
• More ITexture/Data->Raw formats added (Now supports DXT1,2,3,5, ARGB8)
• Decompiling original map support added
• Decompiling some patch’s support added
• Delta patching experimental code first success.
• Decompiling a map broke once more
• UI re-design. (Thanks to rukqoa)
• SHADOW implementation

Any other questions or concerns, just leave a comment below.

As many of you may have seen, I started working with DirectX 11 and Battlefield 3 to create some modified textures etc. Primarily for movie/machinima makers. The project has changed from TeamCanineConfig to venicefx for the updated framework and using quite a few engine features in Frostbite 2. Here are a few videos that show off the features so far.

EDIT: The newest video of VeniceFX is up, it will use everything through console commands. Thanks DICE for re-implementing that 😉

Continue reading “VeniceFX – Progress Update 2, 3 & 4”