UPBGE - What is Depsgraph? And How to Optimize for Depsgraph?

  Blender Dumbass
January 04, 2025

👁 45

https://lm.madiator.cloud/ : 👁 1
https://blenderdumbass.org/ : 👁 2
https://www.google.com/ : 👁 1

#DanisRace #MoriasRace #Game #Gamedev #UPBGE #blender3d #animation #GTAClone #programming #python #project #performance #depsgraph

License:
Creative Commons Attribution Share-Alike

Audio Version

If you ever activated the Framerate and Profile option in your UPBGE game, you have probably seen that one of the most demanding things there is called "Depsgraph".




You see things like "Physics", "Logic" and even "Rasterizer" and you immediately understand what you need to do to optimize you game. But "Depsgraph"?... It looks like a mysterious thing that nobody knows nothing about. Yet is it one the most problematic things there is in your game. And you are going mad just trying to figure it out.

This happened to me when I started doing Dani's Race, my Free / Libre GTA clone game. Which, by the way, you can help me release, by signing a petition, which is really really important, but I digress...

What is Depsgraph?

Depsgraph or Dependency Graph is a genius solution for Blender, which ended up being a curse for UPBGE. As you may know UPBGE is simply the new Blender, but with the game engine preserved.




Back in the Ye Olde days of Blender, you had to release your cursor when changing things, for those things you've changed to be updated in the scene. The modern: dragging your mouse on the color-picker and seeing it update in real time, was unheard of. Well... Until Blender Developers introduced the Dependency Graph, that is.




You know Blender. The color is not just a value on its own. It is a value in a specific node, that is in a specific material, that is in a specific mesh data, linked to a specific object. That now also could be in various collections. So updating color is updating all those things along the way to make the color show up.

The old system would update objects, when the mouse is released. But objects might have multitudes of those data points. They might have many materials, each with many nodes. And updating all of it was unnecessary. So the new depsgraph was born as a kind of optimization method to limit those updates to only whats needed. And that made it possible for the first time to make real-time updates to materials and other stuff, eventually paving the way to insane things like Geometry Nodes.

How Depsgraph works in the game engine?

Having the full Depsgraph implementation in a game engine is absolutely insane. But it makes UPBGE one of the coolest engines out there. You can use the damn Geometry Nodes in your game! This is some next level stuff. But it comes with a few performance penalties.

Blender, when you are working with it as an editor, only runs depsgraph updates when you are actually changing things. When you are dragging sliders. When you are inputting values. The game engine, though, runs Depsgraph on every single frame of the game. Just to make sure that everything will work.

Another potentially damning thing that Depsgraph has, is that it is running on the same thread as the rendering of the frame. And most of the time it takes roughly the same time. This makes your GPU wait half of the time until your CPU will finish calculating depsgraph. Which halves the FPS. There are proposals to separate it into another thread, but the developers seem to be tired or something. So nothing really moves at the moment.

How to optimize for Depsgraph?

First and most important thing when it comes to depsgraph is the amount of objects in the scene. No matter how trivial a change will be, it will still need to loop over the entirety of those objects to check whether they need an update or not. So making less objects will make everything better.




Modifiers especially Geometry Nodes are very heavy on depsgraph. Modifiers change the geometry of an object based on its original geometry. And some even take other objects and other object's geometries as inputs. Making depsgraph loop over much more data to figure out what it needs to update. And because thing are in constant motion in a game, it will try updating modifiers more often. Resulting in bad performance.

Similarly to modifiers, Blender's Constraints are also problematic. If you can implement the same constraint with logic or even code, it will be much better to do so, because using the constraints from within Blender results in more Depsgraph complexity, since they are a part of the same data-tree that the depsgraph is looping over.

When it comes to logic and code, you have to avoid certain things too.

For example, moving an object by simply inputting the position of the object like this:

Object.position = (0,0,0)

... will result in higher depsgraph load, for some reason, than using an applyMovement() function. So I made a simple function like this:

import mathutils
def Move(object, position, times=1):

    # Moving object using applyMovement
    object.applyMovement( ( mathutils.Vector(position) - object.position ) * times )
    

... to move objects more efficiently. This function basically takes the original coordinates and the target coordinates. And calculates the offset, to then use it in the applyMovement() function.

But moving an object is barely a noticeable thing compared to adding and deleting objects. There is something stupidly wrong with the way adding and deleting is implemented, that I had to completely bypass the whole thing in a weird way that is both clever and terrifying.

The game I'm developing has this module called Reuse, which abstracts the adding and deleting, but without actually adding or deleting anything.

For example, here is the delete function.

def Delete(object, inactive=False):
        
    # To make this callable from logic bricks the next thing is needed.
    try:
        object = object.owner
    except:
        pass
    
    # Making a list of those objects in reuse dictionary.
    # Technically if Create() was use to create this object it's
    # not needed, but incase other objects will be stored like this.
    
    if object.name not in reuse:
        reuse[object.name] = []
    
    # Sometimes just recording that the object is available is enough:
    if not inactive:
        
        # Instead of deleting we are going to store it for later
        
        object.worldLinearVelocity = [0,0,0]
        object.worldAngularVelocity = [0,0,0]
        object.scaling = [1,1,1]
        object.suspendPhysics()
        Move(object, (0,0,-1000))
        object.visible = False
        
        # For some objects
        if "Motion" in object.actuators:
            object.actuators["Motion"].dLoc = [0,0,0]
        
    
    # Storing the object for later use
    if object not in reuse[object.name]:
        reuse[object.name].append(object)
    
    
    # Making sure it will not self distract again after it is reused
    object["self-destruct"] = 2
    if object in selfDestruct:
        selfDestruct.remove(object)

As you can see there is no object.endObject() present anywhere. The object is not actually being removed from the scene. But instead as you can see it is being moved to coordinates of 0, 0, -1000, it is being made invisible and the physics of the object are being suspended.

This is the closest thing I can do to deleting it, without actually deleting it, because deleting it will result in a huge spike in depsgraph. And the same goes for adding something in.

There is a similar function I have to "Create" an object:

def Create(object, selfDestructFrames=0, selfDestructInactive=False, visible=True, declarenew=False, frompoint=None):
      
    # Making a list of those objects in reuse dictionary.
    if object not in reuse:
        reuse[object] = []
        
    # If the list is empty ( we need more objects ) we make a new one.
    if not reuse[object]:
        if not frompoint: frompoint = object
        object = scene.addObject(object, frompoint, 0, False)
        new = True
            
        if object.name not in amounts:
            amounts[object.name] = 0
        amounts[object.name] += 1
        
    else:
        object = reuse[object].pop(0)
        object.restorePhysics()
        object.worldLinearVelocity = [0,0,0]
        object.worldAngularVelocity = [0,0,0]
        new = False

    # If self descructing
    if selfDestructFrames:
        SelfDestruct(object, selfDestructFrames, selfDestructInactive)
        
    object.setVisible( visible )
    if declarenew:
        return object, new
    else:
        return object

This one does contain the scene.addObject() function, but only for those objects that were not yet deleted before. But if there is a deleted object that is available to be used, it will re-enable its physics, make it visible again, and use it.

In the first frame of the game, in the Init() function about which I talked not so long ago, in a different article there is a whole thing dedicated to precalculating of those objects. Basically I know that roughly speaking 50 Light Poles might be visible as some point in the game at once. And all of them will need to be spawned into position when the character arrives to the place where they spawn.




So I can tell the game to add the Light Pole object and then immediately delete it, 50 times. Giving me a cached, deactivated 50 Light Poles waiting to be moved into place at a moment's notice. If I don't do that on the first frame, there will be a significant drop in frame rate whenever the player enters a place with many light-poles. Because at that very moment it will need to add 50 new ones. Each effecting depsgraph in some serious way.

But that is not all of it...

Additionally to trying to minimize calls that I know directly effect depsgraph, I also try to minimize calls that do anything potentially a bit too heavy on the engine. And this I do using a handy function I call GoodFPS().

impact = {"test":None,
          "lastTestTime":0,
          "data":{},
          "frame":0,
          "timer":1,
          "estimate":None}

def CalculateImpact(previous, current):

    result = {}
    for i in previous:
        p = previous[i][0]
        c = current [i][0]
        result[i] = c - p

    return result

def GoodFPS(func="", factor=1.0, boolean=True, traceback=False ):

    # This will be used in an if statement before executing
    # some, potentially intense work. And if the performance
    # is dropped below a certain amount. It will stop executing
    # that command.

    settings       = bge.logic.globalDict.get("settings", {})
    frame          = round(bge.logic.getRealTime(), 2)
    currentMetrics = bge.logic.getProfileInfo()
    targetFPS      = settings.get("fps", 30)

    if not boolean:
        return bge.logic.getAverageFrameRate() / targetFPS 
    
    # We need to let the game initialize for a few seconds.
    # So we limit all executions in that time.
    if frame < 5: return 0

    # Controlling the amount of how aggressively this function is executed.
    optset = settings.get("opt", 1.0) ** 0.1
    choice = numpy.random.choice([False, True], p=[optset, 1-optset])
    if choice:
        # Restraining over-bombardment.
        if func in impact["data"]:
            if impact["data"][func].get("lastframe",0) + impact["timer"] > frame:
                return  0.0
        return 1.0

    # Calculating impact timer ( which will time how often one function can be
    # exectuted ).
    impact["timer"] = 10 - (10 * ( bge.logic.getAverageFrameRate() / targetFPS ))
    
    # Recalculating if FPS goes too low.
    # TODO: figure out a way to actually do that reasonably.
    #if bge.logic.getAverageFrameRate() < targetFPS * 0.5:
    #    impact["data"] = {}

    # We want to record FPS impacts of various executed things.
    
    # 1  |  On first 2 frames it should tell everyone to do nothing.
    # 2  |  On second frame it should record profile. And execute.
    # 3  |  Then on third comparing the profiles should give us impact.

    if not impact["test"] and func not in impact["data"] and \
       impact["lastTestTime"] + impact["timer"] < frame:
        
        impact["test"] = {
            "func"   :func,
            "stage"  :0,
            "frame"  :frame,
            "profile":None
        }

        # This is needed to not bombard the player with slow tests
        # immediately.
        impact["lastTestTime"] = frame

    if impact["test"]:

        test = impact["test"]
        
        if test["stage"] == 0 and test["frame"] < frame:
            test["frame"] = frame
            test["stage"] = 1

        if test["stage"] == 1:
            test["frame"] = frame
            test["profile"] = currentMetrics.copy()
            test["stage"] = 2
            return 1.0

        if test["stage"] == 2 and test["frame"] < frame:
            impactIs = CalculateImpact(test["profile"], currentMetrics)
            impact["test"] = None
            impact["data"][func] = impactIs
        
        return 0.0

    elif func in impact["data"]:

        # If we have data about how agressive the changes are,
        # we can use that data to estimate next frames FPS with it.

        # If the function was recently activate, skip it.
        if impact["data"][func].get("lastframe",0) + impact["timer"] > frame:
            return  0.0
        
        # Clearing out the estimates on new frames
        if impact["frame"] != frame:
            impact["frame"] = frame
            impact["estimate"] = None

        # Making a fresh estimate
        if not impact["estimate"]:
            impact["estimate"] = {}
            for i in currentMetrics:
                impact["estimate"][i] = currentMetrics[i][0]

        # Creating a temporary estimate only for this function
        tempEstimate = impact["estimate"].copy()

        # Calculating it's function impact
        totalMS = 0
        for i in tempEstimate:
            tempEstimate[i] += impact["data"][func][i]
            totalMS += tempEstimate[i]

        # Deciding whether we have speed to spare on this calculation
        targetMS = 1 / ( ( targetFPS )  / 1000 ) 
        if totalMS > targetMS:
            return 0.0
        else:
            impact["estimate"] = tempEstimate
            impact["data"][func]["lastframe"] = frame
            return 1.0
        

As you can see it is not a small thing. But the way it works is rather simple. For example you want to spawn a car, but it is not critical. It is a background NPC car or something. And it could spawn now or in a few seconds from now and it will not effect much of anything. So instead of:

Vehicle.Spawn("NeonSpeedsterBox", (0,0,0), (0,0,0))

... you do this:

if Opt.GoodFPS("Spawning an NPC Car"):
    Vehicle.Spawn("NeonSpeedsterBox", (0,0,0), (0,0,0))

First time that the GoodFPS() function will encounter the "Spawning an NPC Car" call, it will suspend everything else, returning False everywhere for a few frames, while returning True ones for this call. This will activate it and the GoodFPS() function will be able to record the impact this call has on the profile. Not just on the overall FPS, but on every individual category: Logic, Animation, Physics, Depsgraph and so on, separately.

Later when it has the database of all possible calls that the game can give it, it can calculate based on the impacts it knows, how many of them can run until the FPS drops below a certain threshold. Anything effecting the FPS below that threshold will not be activated. Resulting in a more or less stable overall performance of the game.




Basically instead of trying to spawn an NPC car on every frame. It will only try doing it when doing it will not effect the performance too much. Of course I did add a setting in the launcher where you can set how aggressive this function is at trying to preserve the target FPS. And for me I feel comfortable at about 90%. Which lets the FPS occasionally drop below the target, but not significantly, while making sure that there is enough stuff in the world to interact with.

Conclusion

A lot of people telling me that I'm stupid for choosing UPBGE as my game engine for Dani's Race, and I agree with them. It would be so much simpler to make the game in an engine that is automatically pre-optimized for things. But there is a certain challenge, a certain satisfaction at figuring out something that is a bit harder to figure out. There is a certain level of achievement in taming the Depsgraph and keeping it under control, making the game work.

Happy Hacking!!!

Subscribe RSS

Author
Website

Share on Mastodon

Share

Post