Last time I talked about the implementation of MallocTracker and the integration with Unreal Engine 4. While I did a good effort to try to explain the insides of MallocTracker, one thing that was clearly lacking is how to interpret the output generated by MallocTracker. In this post I will talk about MallocTracker Viewer and looking at the output generated.

Text output.

When allocations are not tagged and scopes are not defined then the output generated by MallocTracker is not that meaningful but some context is still provided. Let’s look at three allocations as text:

0x0000000063b40000,Main Thread,Unknown,127471600,GlobalScope,UnnamedAllocation
0x000000008bda0000,Main Thread,Unknown,123401431,GlobalScope,UnnamedAllocation
0x00000000ba8d83d0,Main Thread,Unknown,48,GlobalScope,UnnamedTArray

As you can see there is one line per allocation, and under each allocation there is a field. The format for each line is:

Address, Thread name, Group name, Bytes, Scope Stack, Name

So even with the allocations completely untagged there still the thread context that might be useful. But let’s see what we get when we use the UObject MallocTracker scope support as well as adding a scope in FArchiveAsync::Precache() and tagging the single allocation done in that function with the filename of the loaded file:

0x0000000063b40000,Main Thread,UObject,127471600,GlobalScope|HierarchicalInstancedStaticMeshComponent|HierarchicalInstancedStaticMeshComponent,../../../../dev/UnrealEngine/PZ4.8/../../../Unreal/KiteDemo/Content/Maps/GoldenPath/Height/Height_x2_y2.umap
0x000000008bda0000,Main Thread,Systems,123401431,GlobalScope|FArchiveAsyncPrecache,../../../../dev/UnrealEngine/PZ4.8/../../../Unreal/KiteDemo/Content/KiteDemo/Environments/Landscape/Materials/M_Grass_Landscape_01_INST_HOLE.uasset
0x00000000ba8d83d0,Main Thread,Unknown,48,GlobalScope|Material|M_CustomDepthBillboard,UnnamedTArray

With the proper tagging the data is far more meaningful. We now know that a 121.56 MiB allocation was done for the HierarchicalInstancedStaticMeshComponent UObject that was loading the Height_x2_y2.umap file from disk. Same quality of output is available in the other three allocations. While it does seem that the output is very useful, it is pretty much impossible to deal with huge number of allocations in an actual game. The Kite demo after loading everything has over 600k number of allocations that go from 16 bytes to 121.56 MiB happening in 18 different threads.

One sensible thing to do is to import that data into a spreadsheet such as Excel which can already deal with the CSV output generated by MallocTracker. That is very useful when trying to look for specific allocations such as looking for the biggest allocations and such. But in terms of navigation it is still hard to deal with it since there any many duplicated that could be collapsed (meaning allocations with same name, size, group and thread but different addresses). So it is necessary to make something custom. Enter MallocTrackerView.


MallocTrackerView is a tool I’m writing in C# to be able to visualize the data generated by MallocTracker better than I could do with Excel or a text editor. This is my first piece of code written from scratch in C#, in the past I only had to do relatively minor changes to C# tools such as an optimization for Unreal Engine’s MemoryProfiler2. Initially my idea was to write it in Python since I had already used it while doing some research for Electronic Arts and I thought that it was a good fit. The reason I decided against it is that Unreal Engine 4 doesn’t use it at all and it seems the only presence of it is on third party libraries. Using C++ was also a possibility but integrating a proper UI library would be too much. So considering that there are already a bunch of tools written in C# within the engine’s depot I decided to do it in C#. The first thing I had to consider once I had chosen C# is that I would need support for a TreeView control that had support for columns since the idea was to have three columns:

  • Name. This would be the thread name, scope name or allocation name depending on where you are in the hierarchy.
  • Size. Accumulated size of the allocations under that node.
  • Count. Accumulated number of allocations done under that node. Since there can be multiple allocations with the same name under the same thread and scope stack, it is possible that a leaf node has a count higher than one.

Since the default C# TreeView control didn’t properly support that and I really wasn’t into writing my own solution (after all one of the reasons to use C# is to focus on the actual tool rather than infrastructure), I did a rather quick search to see how people solved the need for a similar control. As usual Stack Overflow pointed me to the solution which was to use ObjectListView. After doing a bit more research it seemed like people were happy with it so I decided to use it.

MallocTrackerView right now is very much alpha but in its current state it is fairly useful because it covers the most relevant use case that isn’t covered by a text editor or Excel which is to look at the data in terms of scopes. Let’s go over it by opening a MallocTracker dump done in the Kite demo:
That’s how the UI looks like after opening a MallocTracker dump. The root nodes in the treeview are the different threads. As you go deeper into the tree the scopes are shown and the allocations as well, here is an example:
Here we are looking at the allocations done in the ScotsPineMedium_Billboard_Mat material UObject from the main thread. This is where we can see the usefulness of having such system available, we now know the specific allocations related to that UObject, and correctly accounting for the actual memory allocated rather than relying on hand-written GetResourceSize and serialization numbers. By comparison, this is the info you get from the obj list command related to ScotsPineMedium_Billboard_Mat:

Object NumKBytes MaxKBytes ResKBytes ExclusiveResKBytes
Material /Game/KiteDemo/Environments/
3K 3K 62505K 0K

Even better is to filter the allocations by scope name where we can see related allocations. Here is the output when looking at the allocations that contain the scope name “ScotsPineMedium”:
And you can dig even deeper by combining filters where I filter by allocation group allocations with a certain name:
I think this shows clearly what can be done when having MallocTracker enabled even if only three allocations were properly tagged.

Bad news.

I made a pull request to integrate MallocTracker to Unreal Engine 4 and it was rejected by Epic. I think it was worth going over their reasons for not integrating because someone else might have the same doubts. Let’s go over their concerns as shown in the response to the pull request:

  • It’s too much support burden to maintain those tags. I think this concern comes from the lack of understanding on how the system works. As a programmer you don’t have to tag the allocations. It is better to tag allocations, but if you don’t you still have the scopes to provide context to the allocations as you have seen on the previous examples. If you decided to tag an allocation (and you can use MallocTrackerView to determine which allocations to tag) then you don’t have to worry about that allocation ever again. The tagging is a change in a single line of code (be it an FMemory call, a container constructor, etc) you don’t need to maintain. What is a burden is maintaining all the UObject’s GetResourceSize() functions so I beg to differ. It is also worth noting that this kind of tagging isn’t something that isn’t sure it can scale, I have seen it used in full AAA games throughout the code without this concern being relevant at all.
  • It most likely won’t be used by anyone at higher level. This makes the assumption that non-engine people currently have a good grasp on the memory usage of their game code and assets. While it is true that the MemReport command does offer some insights, it is still not enough and it is certainly harder to visualize than using MallocTracker with MallocTrackerView.
  • At a lower level it usually doesn’t provide enough information. The only tool provided by the engine that does a better job is MallocProfiler with the MemoryProfiler2 tool. But the issues with it are that it is basically unusable in any memory-constrained environment (particularly consoles), the performance is completely unacceptable to the point that the game becomes unplayable, and many of the allocations have the exact same callstack even though they refer to completely different UObjects. Instead MallocTracker runs which pretty much the same performance as having it disabled, it has a considerably lower memory overhead (at least an order of magnitude smaller, for example it needs 35.3MiB to store 614,145 allocations), and it does provided proper information even in terms of allocations done to create UObjects from Blueprints.

In the end it is fine to have this pull request rejected, after all Epic can’t accept whatever is sent their way, but I happen to think that having this in would be rather useful for everybody. But to Epic’s credit, given the open nature of the engine anybody can ask me for the changes and get to use this even if Epic themselves won’t integrate it. That is much better that you can probably do using Unity or some other closed engine.

Overview video.

Adding memory tracking features to Unreal Engine 4.

It has been a while since I last talked about memory allocation and tracking. I have had the time to implement the tracking on Unreal Engine 4 and I think that I go over it. I’m going to make the assumption that you have read the previous blog post I made about this: “Limitations of memory tracking features in Unreal Engine 4” and “Memory allocation and tracking“.

Unreal Engine 4 memory management API.

Basic allocation methods.

There are three basic ways to allocate or free memory within the Unreal Engine 4:

  • Use of GMalloc pointer. This a way to get access to the global allocator. Which allocator is set to be used depends on GCreateMalloc().
  • FMemory functions. These are static functions such as Malloc(), Realloc(), and Free(). They also use GMalloc for the allocations but before doing that it checks if GMalloc is defined before every allocation, reallocation or free. If GMalloc is nullptr then GCreateMalloc() is called.
  • Global new and delete operators. By default they are only defined in the modules in ModuleBoilerplate.h which means that many calls to new and delete were not being handled within the Unreal Engine 4 memory system. The overloaded operators actually call the FMemory functions.

There are cases where memory isn’t deallocated and freed through those mechanisms which shouldn’t happen if possible. To catch those cases I submitted a pull request, which is already integrated and set for release on version 4.9, that catches those allocations via the c runtime library call _CrtSetAllocHook(). One such example is that the engine’s integration of zlib didn’t do allocations through the engine’s facilities, using _CrtSetAllocHook() I found that out and made a pull request with the fix which is set for release on 4.9.

The basic API for both, the direct calls to GMalloc and the FMemory functions, is this:

virtual void* Malloc( SIZE_T Count, uint32 Alignment = DEFAULT_ALIGNMENT ) = 0;
virtual void* Realloc( void* Original, SIZE_T Count, uint32 Alignment = DEFAULT_ALIGNMENT ) = 0;
virtual void Free( void* Original ) = 0;

These are the places that would need to be modified if we want to add any new piece of data per allocation.

Integrating with the engine

Similar to the approach I took for the stomp allocator, I made a new class called FMallocTracker that derives from FMalloc which allows me to hook it up to the Unreal memory allocation system. Since a valid allocator must be passed when creating the FMallocTracker instance all the actual allocations will be done with that allocator. The FMallocTracker is only there to keep tracking information, nothing else. But that is not enough, we actually need to get the tracking data we want in all the way to the allocator. So the first step is to modify the allocator’s functions when we have the memory tracking feature enabled:

virtual void* Malloc( SIZE_T Count, uint32 Alignment, const uint32 GroupId, const char * const Name ) = 0;
virtual void* Realloc( void* Original, SIZE_T Count, uint32 Alignment, const uint32 GroupId, const char * const Name ) = 0;
virtual void* Malloc( SIZE_T Count, uint32 Alignment = DEFAULT_ALIGNMENT ) = 0;
virtual void* Realloc( void* Original, SIZE_T Count, uint32 Alignment = DEFAULT_ALIGNMENT ) = 0;

The new parameters are:

  • Name. Name of the allocation. This name can be whatever you want but the recommendation is that it be a literal that is easy to search for. I will show the means to provide more context later in this post.
  • Group. This is the id for the group that is responsible for this allocation. These are the groups that I have defined but it something that you should definitely tune for your needs.

This change implies changes to all the allocators to have it be transparent within the engine, but once that work is done then you can tag the allocations without worrying about the underlying implementation. The benefit of tagging allocations isn’t just for tracking purposes but it is also relevant for code documentation. Having worked on large codebases as a consultant with this kind of tagging done is a great benefit when ramping up, to deal with interactions among different groups, and fix memory related crashes.

The next step is to integrate with the new and delete operators. As I already mentioned, in the engine they were defined in the ModuleBoilerplate.h, to provide better coverage I first move it to MemoryBase.h. The next step was to define our new operator overloads to pass in the name and group.

OPERATOR_NEW_MSVC_PRAGMA FORCEINLINE void* operator new  (size_t Size, const uint32 Alignment, const uint32 GroupId, const char * const Name)	OPERATOR_NEW_NOTHROW_SPEC{ return FMemory::Malloc(Size, Alignment, GroupId, Name); }
OPERATOR_NEW_MSVC_PRAGMA FORCEINLINE void* operator new[](size_t Size, const uint32 Alignment, const uint32 GroupId, const char * const Name)	OPERATOR_NEW_NOTHROW_SPEC{ return FMemory::Malloc(Size, Alignment, GroupId, Name); }
OPERATOR_NEW_MSVC_PRAGMA FORCEINLINE void* operator new  (size_t Size, const uint32 Alignment, const uint32 GroupId, const char * const Name, const std::nothrow_t&)	OPERATOR_NEW_NOTHROW_SPEC	{ return FMemory::Malloc(Size, Alignment, GroupId, Name); }
OPERATOR_NEW_MSVC_PRAGMA FORCEINLINE void* operator new[](size_t Size, const uint32 Alignment, const uint32 GroupId, const char * const Name, const std::nothrow_t&)	OPERATOR_NEW_NOTHROW_SPEC	{ return FMemory::Malloc(Size, Alignment, GroupId, Name); }

To avoid having to fill up the code checking for USE_MALLOC_TRACKER it is nice to provide some defines to create those allocations as if USE_MALLOC_TRACKER was set but without incurring in unnecessary cost when it isn’t set. The intention is that this should be something with very little to no performance cost. So here is the basic definition:

	#define PZ_NEW(GroupId, Name) new(DEFAULT_ALIGNMENT, (Name), (GroupId))
	#define PZ_NEW_ALIGNED(Alignment, GroupId, Name) new((Alignment), (Name), (GroupId))
	#define PZ_NEW_ARRAY(GroupId, Name, Type, Num)  reinterpret_cast<##Type*>(FMemory::Malloc((Num) * sizeof(##Type), DEFAULT_ALIGNMENT, (Name), (GroupId)))
	#define PZ_NEW_ARRAY_ALIGNED(Alignment, GroupId, Name, Type, Num)  reinterpret_cast<##Type*>(FMemory::Malloc((Num) * sizeof(##Type), (Alignment), (Name), (GroupId)))
	#define PZ_NEW(GroupId, Name) new(DEFAULT_ALIGNMENT)
	#define PZ_NEW_ALIGNED(Alignment, GroupId, Name) new((Alignment))
	#define PZ_NEW_ARRAY(GroupId, Name, Type, Num)  reinterpret_cast<##Type*>(FMemory::Malloc((Num) * sizeof(##Type), DEFAULT_ALIGNMENT))
	#define PZ_NEW_ARRAY_ALIGNED(Alignment, GroupId, Name, Type, Num)  reinterpret_cast<##Type*>(FMemory::Malloc((Num) * sizeof(##Type), (Alignment)))

Here are a couple of examples of how the tagged allocations look compared to the non-tagged allocation in terms of code:

Tracking allocations done by containers.

One of the issues that does come up when naming allocations in a simple way to recognize each time is dealing with containers. There is hardly ever a single instance of anything within the engine and when making a game, be it position of particles or number of players, so a lot of containers are used within the engine. When making an allocation within a container it wouldn’t be too useful to have a generic names. Let’s look at this example from FMeshParticleVertexFactory::DataType:

/** The streams to read the texture coordinates from. */
TArray<FVertexStreamComponent,TFixedAllocator<MAX_TEXCOORDS> > TextureCoordinates;

A generic name for allocations done within the allocator assigned for that container would be something like “TFixedAllocator::ResizeAllocation”. It doesn’t say much. Instead, a better name for all allocations related to that container would be something like “FMeshParticleVertexFactory::DataType::TextureCoordinates”. In order to do this we need to be able to assign names and groups to the containers in such way that whenever an allocation is done within that container, the name and group of the container is fetched to tag those allocations. In order to do that we will need to make changes to the containers and to the allocators that can be used with those containers. That would involve adding pointer and a 32-bit unsigned integer per container when building with USE_MALLOC_TRACKER enabled, and the changing the necessary constructors to add that optional information. One of the constructors for a TArray would look like this:

TArray(const uint32 GroupId = GROUP_UNKNOWN, const char * const Name = "UnnamedTArray")
	: ArrayNum(0)
	, ArrayMax(0)
	, ArrayName(Name)
	, ArrayGroupId(GroupId)

With those changes in place we have the necessary information to send to the allocators to be able to tag those allocations. The next step is to look at those allocators and make the necessary changes to pass that information to the underlying allocator being used. Those container allocators in general use the FMemory method of allocating memory, and the FContainerAllocatorInterface defines the ResizeAllocation function that actually does the allocation of memory. Similarly to the previous changes, we need to add the name and group for the allocation.

	void ResizeAllocation(int32 PreviousNumElements, int32 NumElements, SIZE_T NumBytesPerElement, const uint32 GroupId, const char * const Name);
	void ResizeAllocation(int32 PreviousNumElements, int32 NumElements, SIZE_T NumBytesPerElement);

Again, since we don’t want to fill up the engine’s code with ifdefs, we again rely on a define to simplify that:

#define PZ_CONTAINER_RESIZE_ALLOCATION(ContainerPtr, PreviousNumElements, NumElements, NumBytesPerElement, GroupId, Name) (ContainerPtr)->ResizeAllocation((PreviousNumElements), (NumElements), (NumBytesPerElement), (GroupId), (Name))
#define PZ_CONTAINER_RESIZE_ALLOCATION(ContainerPtr, PreviousNumElements, NumElements, NumBytesPerElement, GroupId, Name) (ContainerPtr)->ResizeAllocation((PreviousNumElements), (NumElements), (NumBytesPerElement))

With that in place then we can pass the ArrayName and ArrayGroup to container allocator.

One thing that is also necessary is to change the name or group of a container after construction because that it what’s necessary to be able to name allocations done by containers of containers. One of such example is this where we need to set the name or group after a FindOrAdd in any of these TMap containers:

/** Map of object to their outers, used to avoid an object iterator to find such things. **/
TMap<UObjectBase*, TSet<UObjectBase*> > ObjectOuterMap;
TMap<UClass*, TSet<UObjectBase*> > ClassToObjectListMap;
TMap<UClass*, TSet<UClass*> > ClassToChildListMap;

Once that’s done then all the allocations done by the containers will be tagged properly as they change. So now we just need to set the name for the container. Going back to the FMeshParticleVertexFactory::DataType::TextureCoordinates example, we can now set the name and group for the allocation:

	: TextureCoordinates(GROUP_RENDERING, "FMeshParticleVertexFactory::DataType::TextureCoordinates")
	, bInitialized(false)

Defining scopes.

As part of the “Memory allocation and tracking” post I mentioned the need to define scopes for the allocation in order to provide context. The scopes are not the same as callstacks (which is something already provided by MallocProfiler). Many allocations happen within the same callstack but referring to completely different UObjects. That even more prevalent with the use of Blueprints. Due to that it is very useful to have scopes that would allow tracking or memory usage even within Blueprints.

To leverage the code already present in the engine I took the approach of reusing the FScopeCycleCounterUObject struct which is used to define scopes related to UObjects for the stats system. The engine already has placed those scopes where it’s necessary, and you can still place your own allocation-tracking-specific scopes by using the FMallocTrackerScope class. Also to improve visibility two scopes are created automatically on each FScopeCycleCounterUObject, a scope with the name of the class of the UObject, and a scope with the name of the UObject. That makes it easier to collapse the data per class name when we eventually create a tool to visualize the data. It get a better sense of the complexity let’s look at a single scope coming from the Elemental demo:
If we analyze the allocations under that scope we see the following:

Address Thread Name Group Bytes Name
0x0000000023156420 Main Thread UObject 96 InterpGroupInst
0x00000000231cf000 Main Thread Unknown 64 UnnamedTSet
0x0000000023168480 Main Thread UObject 80 InterpTrackInstMove
0x0000000028ee8480 Main Thread Unknown 64 UnnamedTSet
0x0000000022bc2420 Main Thread Unknown 32 UnnamedTArray
0x00000000231563c0 Main Thread UObject 96 InterpGroupInst
0x00000000231cefc0 Main Thread Unknown 64 UnnamedTSet
0x0000000023168430 Main Thread UObject 80 InterpTrackInstMove
0x00000000231cef80 Main Thread Unknown 64 UnnamedTSet
0x0000000022bc2400 Main Thread Unknown 32 UnnamedTArray
0x0000000023156360 Main Thread UObject 96 InterpGroupInst
0x00000000231cef40 Main Thread Unknown 64 UnnamedTSet
0x00000000231683e0 Main Thread UObject 80 InterpTrackInstMove
0x0000000028ee8380 Main Thread Unknown 64 UnnamedTSet
0x0000000022bc23e0 Main Thread Unknown 32 UnnamedTArray
0x00000000231cef00 Main Thread UObject 64 InterpTrackInstAnimControl
0x00000000231ceec0 Main Thread UObject 64 InterpTrackInstVisibility

Those are just 17 allocations on the Play function in a Blueprint. The actual number of allocations that there are when I made the capture on the Elemental demo was 584454. The number of unique scopes is pretty high as well, 4175. And with that we are just talking about the 607MiBs allocated at the time of the capture even though the peak number of bytes allocated was 603MiBs. This goes to show the need for this kind of memory tracking.

MallocTracker implementation

As I mentioned previously, MallocTracker was implemented in a similar fashion as the stomp allocator I made previously. The MallocTracker was made to be lightweight and follow the performance requirements mentioned in “Memory allocation and tracking“.

The implementation is fast enough to be enabled by default without causing too much of an impact in performance, and with a fairly low overhead in terms of memory. For example the Elements demo showed an overhead for tracking of ~30MiBs and under 2 ms on the CPU on a debug build, even less on optimized builds. As usual there is a compromise between memory overhead and performance, those numbers are based on the approach I decided to take. There are other approaches to take which will either favor performance or memory overhead but I think I struck a reasonable balance.

To start analyzing the implementation let’s look at a concrete example. This is what happens when FMemory::Malloc() gets called:

  1. AllocDiagramFMemory::Malloc() gets called requesting a certain number of bytes with a given name and group assigned to that allocation.
  2. FMemory::Malloc() calls FMallocTracker::Malloc() with the same arguments assuming that GMalloc points to an FMallocTracker instance.
  3. FMallocTracker::Malloc() allocates the actual memory by using the allocator that was passed in during FMallocTracker creation, in this case FMallocBinned.
  4. FMallocTracker::Malloc() modifies atomically some global allocation stats such as peak number of bytes allocated, peak number of allocations, etcetera.
  5. FMallocTracker::Malloc() gets the assigned PerThreadData instance for the current thread.
  6. FMallocTracker::Malloc() calls PerThreadData::AddAllocation to store the allocation’s data on this thread’s containers.
  7. FMallocTracker::Malloc() returns the pointer the underlying allocator returned in step 3.

Global statistics.

Very few global stats are included. They are there to give you a very quick overview but nothing else. The global stats collected are:

  • Allocated Bytes. Number of bytes allocated when the data was dumped.
  • Number of allocations. Number of allocations when the data was dumped. A high number of allocations will usually cause high memory fragmentation.
  • Peak number of allocated bytes. The highest number of bytes allocated since MallocTracker was enabled.
  • Peak number of allocations. The highest number of living allocations since MallocTracker was enabled.
  • Overhead bytes. Number of bytes that are MallocTracker internal overhead.

All those stats are atomically updated since there are being accessed by all threads doing allocations.

Data per thread.

In order to improve performance and avoid contention of resources as multiple threads allocates and frees memory, most of the work is done on a per-thread basis. All allocations and scope stacks are stored per thread. All allocations have a relevant scope stack defined and the uppermost scope is the GlobalScope. The same scope names usually show up in multiple scope stacks. As such in order to minimize memory overhead all scope names for that thread are stored uniquely and referenced on the scope stacks. And since scope stacks are show up in multiple allocations then we store scope stacks uniquely. Let’s look at a concrete example, scope names in blue and allocations in orange:
To store that data we would have three distinct arrays which aren’t shared among the different threads:

  • Unique scope names. This stores unique scope names used in this threads. At least the GlobalScope is assured to be there. This will store new scope names as they are pushed into the stack.
  • Unique scope stacks. This stores unique stacks by creating a dynamic array of fixed-size arrays where indices to the relevant scope names are store.
  • Allocations. Data for each allocation. This includes the address of the allocation, size in bytes, group and name assigned to the allocation, and the index to the unique scope stack.

If we refer to the previous graph we see that we have five allocations. Here is the data to store those five allocations:

Reallocating and freeing memory.

Reallocations and frees make things a bit more complicated due to the fact that it is rather common within the Unreal Engine to see instances of allocations happening in one thread and then being reallocated or freed on a different thread. That means that we can’t assume that we will find the relevant allocation on the per-thread data of the calling thread. Since that is the case it also means that we need to introduce some locking. In other to reduce the contention for a global lock instead each per-thread data class has its own lock. Both in reallocation and freeing the per-thread data of the current calling thread is checked for the existing allocation. If it isn’t found there then it looks for that allocation on the other per-thread data lock them in the process one at a time. This ensure that contention is reduced and the rest can keep themselves busy as long as possible.

Dealing with names.

In order to make the MallocTracker fast enough and acceptable in terms of memory consumption I had to put in place a restriction on any kind of naming, be it the name for the allocations or scopes. The restriction is that the lifetime of the memory where those names are stored must be the same or longer than the actual allocation or scope. The reason is that making any copy of that data greatly impacts performance and memory consumption, so only pointers are stored. While that may seem like a complex restriction to live with, I find it to be perfectly fine since you should know the lifetime of your own allocations. If you don’t know about the lifetime of your allocations in order to know for how long to keep those names alive then you have bigger problems to deal with.

Another particular implementation with respect to allocation and scope names is the need to deal with ANSI and wide character names. To make this more transparent all those pointers are assumed to be ANSI unless the 63rd bit in the pointer is set in which case the pointer is assumed to point to a wide character name. FMallocTracker provides a way to get a pointer with that bit set for wide char names, and to set if necessary for FNames which can be either wide or ANSI. At the time of output to file the names are handled properly and dumped to file.


Unfortunately I won’t be able to convey the true usefulness of this type of system until I make a tool to visualize the data properly, but you can take me work that it is really useful. Finding fragmentation issues and dealing with rampant memory usage is so much easier with this data. This is considerably better than what is already provided in the engine in terms of performance, memory consumption, and data quality. The next step would be to actually fully transition the engine to use the tagged allocation but that’s something that can be done as needed. It certainly doesn’t make sense to spend too much time just tagging allocations where many of them are not conflicting. Instead it is better to just tag the big allocations to get more insight into the specific issues. But even if you find tagging allocations too boring, you may still get useful data. Here is the data of the biggest allocations captured on the Elemental demo.

Sample data and source code.

To make more sense out of this I also provide sample data. The sample data comes out of running a modified version of the Elemental demo on a test build. You can download the data from here and view it with any text editor that support big files.
You can also view the code by looking at the pull request I made for Epic Games to see if they will take this update. The pull request is available here.

Video overview.

Memory stomp allocator for Unreal Engine 4.

As I have been working on the memory tracking feature one of the issues I had to deal with is memory stomps. They are hard to track even with the debug allocator provided by Unreal Engine 4. I will go over what are the cases to handle and the actual implementation.

Symptoms of a memory stomp.

The symptoms of a memory stomp could be clearly evident as an explicit error about a corrupted heap, or as complex as unexpected behavior without any crash at all which is why they are so hard to catch. This is exacerbated by the fact that any performance-aware allocator won’t actually request pages to the OS on every allocation but rather request multiple pages at once and assign addresses as necessary. Only when they run out of available pages will they request new pages to the OS. In that situation the OS won’t be able to do anything to let us know that we messed up (by for example, throwing an access violation exception). Instead execution continues as usual but behavior isn’t what is expected since we could be effectively be operating with memory that is unrelated to what we want to read or write. Just as an example, I had to deal with the case where some CPU particles were behaving in a way that they would end up at origin and color change every now and then. After looking for a logic error, I was able to determine that the issue had nothing to do with the code changing the color or transform but rather a memory overrun on unrelated code which ended up in the same pool in the binned allocator. Another example is when pointers get overwritten. If you don’t see that the pointer itself was written somewhere else, rather than having the data that is being pointed to corrupt, you may waste some time. Depending on the nature of the code accessing the data pointer it may or may not crash. Still, the symptom would be similar to that of overwritten data.

Types of memory stomps.

A memory stomp could be defined as doing different type of operations with memory that are invalid. All these invalid operations are hard to catch due to their nature. They are:

  • Memory overrun. Reading or writing off the end of an allocation.
static const size_t NumBytes = 1U << 6U;
uint8_t * const TestBytes = new uint8_t[NumBytes];
// Memory overrun:
static const size_t SomeVariable = 7U;
TestBytes[1U << SomeVariable] = 1U;
delete [] TestBytes;
  • Memory underrun. Reading or writing off the beginning of an allocation.
static const size_t NumBytes = 1U << 6U;
uint8_t * const TestBytes = new uint8_t[NumBytes];
// Memory underrun:
TestBytes[-128] = 1U;
delete [] TestBytes;
  • Operation after free. Reading or writing an allocation that was already free.
static const size_t NumBytes = 1U << 6U;
uint8_t * const TestBytes = new uint8_t[NumBytes];
delete [] TestBytes;
// Operation after free:
TestBytes[0] = 1U;

One confusion that does raise up is if dereferencing a null pointer could be considered a memory stomp. Given that the behavior when dereferencing null is undefined, it wouldn’t be possible to say that it is effectively a memory stomp. So to keep things simple, I rely on the target-platform OS to deal with that case.

How it works.

The stomp allocator work by using the memory protection features in the different operating systems. They allow us to tag different memory pages (which are usually 4KiB each) with different access modes. The access modes are OS-dependent but they all offer four basic protection modes: execute, read, write, and no-access. Based on that the stomp allocator allocates at least two pages, one page where the actual allocation lives, and an extra page that will be used to detect the memory stomp. The allocation requested is pushed to the end of the last valid page in such way that any read or write beyond that point would cause a segmentation fault since you would be trying to access a protected page. Since people say that a picture is worth a thousand words, here is a reference diagram: MallocStompDiag

Diagram not to scale. The memory allocation information + the sentinel only accounts for 0.39% of the two pages shown.

As it is visible, there is a waste of space due to the fact that we have to deal with full pages. That is the price we have to pay. But the waste is limited to a full page plus the waste in the space of the first valid page. So the stomp allocator isn’t something that you would have enabled by default, but it is a tool to help you find those hard to catch issues. Another nice benefit is that this approach should work fine on many platforms. As a pull request to Epic I’m providing the implementation for Windows, Xbox One, Linux and Mac. But it can be implemented on PlayStation 4 (details under NDA) and perhaps even on mobile platforms as long as they provide functionality similar to what’s provided with mprotect or VirtualProtect. Another aspect for safety is the use of a sentinel to detect an underrun. The sentinel is the last member of the AllocationData which is shown as “Memory allocation information” which is necessary to be able to deallocate the full allocation on free, and to see how much information to copy on Realloc. When an allocation is freed then the sentinel is checked to see if it is the value expected (which in my code it is 0xdeadbeef or 0xdeadbeefdeadbeef depending if it is a 32-bit or 64-bit build). If the sentinel doesn’t match the expected value then there was an underrun detected and the debugger will be stopped. But that will only work for very specific cases where the sentinel is overwritten. So changing the layout with a different mode would help with this issue. Here is the diagram: MallocStompDiagUnder

Diagram not to scale.

This basically flips where the no-access page is set. This allows finding underruns that happen as long as it writes before the “Memory allocation information” shown in the diagram. That piece of data is small (the size depends on the architecture used to compile). The sentinel is still present to deal with small underrun errors. The underrun errors that manage to go below that point will actually hit the no-access page which will trigger the response we want from the OS. The last aspect is that any attempt to read or write from memory already free will fail as it happens. A performance-aware allocator (such as the binned allocator) would add those pages to a free list and return them when someone else request memory without actually making a request to the OS for new pages. That is one of the reasons why they are fast. The stomp allocator will actually decommit the freed pages which makes any read or write operation in those pages invalid.

Memory allocation and tracking.

After that optimization interlude, let’s go back to memory allocation and tracking. As I mentioned last time I would like to go over the different requirements of the memory allocation and tracking features necessary for shipping a AAA game without having a horrible time doing memory optimizations.

Solution Requirements

Most AAA games have a lot of resource that go in and out of system and video memory constantly. This is more exacerbated in sandbox games, and less so in stadium games (such as NBA 2K15). In any case, the solution needs to be fast and provide proper data related to all allocations happening. A fast solution that doesn’t provided proper tracking facilities won’t do, and a solution with proper tracking facilities but it is slow won’t do either. Both aspects are important in equal measure. At the same time, ALL allocations must go through it which implies that client code nor third party libraries won’t allocate memory on their own and global new and delete operators should be overridden.

Tracking Information

The solution must provide relevant allocation tracking information. The information must be as general as how much memory is used overall, as specific as what was the address returned for a specific allocation, and everything in between. Any allocation must have relevant tracking information that is defined when it was allocated and can be used as a reference for programmers to detect issues.

General Information

The general information provided should be extremely simple. It should provide the following items:

  • Allocated bytes.
  • Number of allocations.
  • Peak allocated bytes.
  • Peak number of allocations.

Allocations Grouping

Just like there are different groups/teams focused on different areas of a game, allocations should also be grouped. Some of the groups are rendering, gameplay, UI, audio, etc. The different groups have different memory allocation patterns and requirements. As such, it is a good idea to group allocations based on that criteria because it provides the following benefits:

  • Optimal allocation setup. Not all groups have the same allocation needs so it is better to allow allocation setup per group. That means that for example, not all groups need mutexed allocators, not all groups may use the same small block allocator, etc.
  • Budget tracking and enforcement. It is essential to have each group have their own amount of ram that they can keep track of, and systems programmers can assign in agreement with the different groups. Basically this will guarantee that they all have the their fair share and that everybody keeps memory usage under control.
  • Easier to detect corruption issues. Since all allocations have a group assigned with a proper allocation setup, it isn’t that hard to solve corruption issues or issues during allocation. The grouping provide good initial context.
  • Better performance. Since mutexes are not required in all the groups or allocators that cost can be avoided. On groups that require mutexes there is also less contention since there isn’t a single mutexed allocator (such as the global allocator) to lock. It is also possible to make trade-offs between absolute performance, and peak memory usage when it comes to defining how memory is allocated.

Allocation Naming

All allocations should be “named” in order to have proper ways to recognize the different allocations. What the name implies is up to whoever requests the memory, and you may enforce naming conventions, but that tag should be available in order to track memory allocations. For the sake of performance, these tags should only be available on non-shipping builds.

Allocation Scoping

The solution must allow the stacking of scopes per thread to provide more context to allocations. This provides better context than simply using the callstack of the allocation, and it is far cheaper that grabbing the callstack. On Unreal an example might be where are scope is created during UObject creation so all allocations happening for that UObject are nested under that scope. All non-scoped allocation would still belong to a global scope. Here is an example of a scope stack with the allocation as a leaf node and its relevant data:

Main Thread												Pointer				Bytes	Group
	Global Scope
							FPhysXAllocator::allocate	0x000000000b093fe0	131720	Physics

Allocation flagging

Allocations may provide optional flags that depending on the allocator used it will have a certain meaning. Some flags examples are:

  • Lifetime flags. Provides hints as to how long this allocation is expected to live. This allows the allocator to be more clever when allocating memory to reduce memory fragmentation.
  • Clear allocation flag. Let the allocator know that you want the memory allocated cleared to zero before returning.


The solution must provide acceptable performance even in non-shipping builds with the tracking features enabled. And by acceptable that means that the frame time should never go above 50ms per frame with tracking features enabled. When it goes above that threshold people start trying to avoid using the tracking features and that’s the slippery slope you will have to recover from at the worst possible time, shipping time. Also performance hit and general overhead on shipping builds should be slim to none.

Allocations Grouping

To implement the best possible allocation scheme without exposing the complexities to the client code, it make sense to define multiple allocators per group. Those allocators would be called in order, which ever allocator succeeds at allocating the memory returns the allocation. So for example under a normal group three allocators could be added:

  • Static Small Block Allocator (SSBA). This could be a static small block allocator that doesn’t grow and accepts allocations up to 256 bytes.
  • Dynamic Small Block Allocator (DSBA). This could be a dynamic small block allocator that grows as necessary and accepts allocations up to 1024 bytes.
  • Standard Allocator (SA). Standard OS allocator that allocates any size allocation.

So a request for 1032 bytes would try the SSBA first, the DSBA, and finally it will request the memory to the SA. It is also perfectly fine to just have one allocator if the allocator you use provides the functionality you need. For example, you may use jemalloc which already handles different sized allocations by itself with the proper locking mechanisms.


With all this data we ought to be able to create the proper facilities for memory allocation and tracking. Next time we will delve into the API that we would eventually implement. If you think I forgot any other item please don’t hesitate to comment.