The Machinery - Part 4

2020-07-21

Our Machinery definitely turns the page on certain conventions. And not in a way that seems troublesome or pointless. The goal really appears to be about strong communication and making sure things describe intent. One that I didn't expect was having engines in addition to entity systems.

There is a clear and interesting distinction between them and my preconceived notions for what an "entity system" would be, is actually what is called an "engine" in The Machinery.

I don't think it's a problem to call the functions that operate, or transform data on components engines. In fact I rather like it personally. And yes, it's still in the phase where I have a total crush on The Machinery and haven't entered the pit of familiarity and all that.

ANYWAY...

Engines! They are the things that run around and apply themselves to your scenes based on component filters, filters which inform The Machinery as to what component data should be provided to the engine when it's called.

Entity systems on the other hand are simply called upon to perform some kinds of work (like a generic 'tick' or 'update') and aren't given any specific component data. For the rest of this post Imma ignore them.

Registering an engine

Engines operate in "simulation" mode (runtime) and so they are registered as an implementation of the tm_entity_simulation_i interface:

tm_add_or_remove_implementation(reg, load, TM_ENTITY_SIMULATION_INTERFACE_NAME, component__register_engine);

Similar to the interface for creating a component, the tm_engine_simulation_i interface is a single function pointer with which we register our component__register_engine function as an implementation.

Our implementation takes a pointer to a entity context, gets handles to the required components, initializes our tm_engine_i interface and registers it.

I'm not yet certain what the relationship between an entity context and an entity itself is yet. Is this engine registered for every entity in our scenes with the correct combination of components?

The entity engine interface can define filters which describe the set of entities it should be applied to. This can take the form of specifying an inclusive list of component types (only entities with all of the listed components), and exclusive list (only entitye with none of the listed components), or a more comprehensive "filter" function which can be used to apply more complicated expressions as needed.

Our example component specifies both the component list and the filter function.

---

The entity api provides a function to lookup components by their hash, which we use to get a handle to the 3 components we're interested in:

const uint32_t sprite_component = tm_entity_api->lookup_component(ctx, TM_TT_TYPE_HASH__SPRITE_COMPONENT);
const uint32_t transform_component = tm_entity_api->lookup_component(ctx, TM_TT_TYPE_HASH__TRANSFORM_COMPONENT);
const uint32_t link_component = tm_entity_api->lookup_component(ctx, TM_TT_TYPE_HASH__LINK_COMPONENT);

This is part of the reason I'm currently confused by the distinction between an entity context and and entity. Are we getting handles to specific component instances or some archetypical index for the component for every entity?

Taking a look into entity.h I see that lookup_component gives us an ID for a component (where 0 means "no component"). This ID is actually used in other functions (such as add_component, get_component, and remove_component). In these other cases you're using the component ID in combination with an entity handle and the entity context.

---

With our three component IDs determined we can initialize our engine interface and register it:

const tm_engine_i sprite_component_engine = {
    .name = "Sprite Component",
    .num_components = 3,
    .components = { sprite_component, transform_component, link_component },
    .writes = { false, true, true },
    .update = engine_update__sprite_component,
    .filter = engine_filter__sprite_component,
    .inst = (tm_engine_o *)ctx,
};
tm_entity_api->register_engine(ctx, &sprite_component_engine);

We give the engine a name, list the required components and declare whether we intend to write or read data on the component. In our case we're intending to update the transform or link component, but only read from our the sprite component.

Our update and filter functions are associated with the engine as is the entity context.

Filter functions

These functions are useful when a simple description of a set isn't possible using the components and excludes fields of the tm_engine_i interface.

In the case of this engine the filter function is used because of the relationship between the link component and the transform component. Our filter function says that any entity with a sprite component and either a link component or transform component are valid for this engine.

We need to specify this, otherwise we would only be operating on entities without a parent or entities with a parent, instead of both. This is because entities with a link component can only have their transform modified through the link component.

Filter functions work by way of component masks, although I'd assume you're free to use other means depending on the case. Our sample filter is defined as:

static bool engine_filter__sprite_component(
    tm_engine_o *inst,
    const uint32_t *components,
    uint32_t num_components,
    const tm_component_mask_t *mask)
{
    return tm_entity_mask_has_component(mask, components[0]) &&
        (tm_entity_mask_has_component(mask, components[1]) ||
            tm_entity_mask_has_component(mask, components[2]));
}

Clarification time

Before I hop into the update method, it's time to sit down with some docs and understand more concretely all the new terms and build up a better mental model.

I don't always avoid docs before trying to understand things, but when there is sample code it's just easier for me to start there first and build up questions which lead me through the documentation with more of a goal.

---

entity.h has a neat and concise description of the four main features of the entity system in The Machinery:

• An entity is an ID which uniquely identifies a game object.
• A component is data that can be associated with an entity.
• An engine is an update function that is applied to all entities with certain components.
• A system is an update function that is applied to the entire entity context.

It's explained that entities live in something called a simulation context and that entity IDs are only unique for a specific context. Applications are able to have multiple contexts running if need be. Each one being it's own isolated world.

Seems fair to imagine for now that we'd only be dealing with a single context here since we're setting up a small scene in the editor and running it.

An interesting part is called The Blackboard. This is the system designed for providing constant data to engines without having to explicitly provide it to each instance. The example used in the docs was a global such as "delta time". Each blackboard value is a struct with an ID and a union that's either a double or a void*. The IDs defined in the entity system headers are all hashes of strings such as:

// Blackboard item representing the total elapsed time in the simulation.
#define TM_ENTITY_BB__TIME TM_STATIC_HASH("tm_time", 0x6a30b071f871aa9dULL)

Blackboard values are provided to engines as a pair of start and end pointers. It seems a bit strange to have to search through the blackboard values like this in order to find a value you're interested in. But that strangeness is perhaps the result of having more often used other languages (including Rust) where you'd probably just be handed a map of some kind. Is always running a hash function to find something, perhaps multiple things, worth the cost compared to just iterating over these items and comparing integers? I think that would just depend on how many items you might be looking for, and how many items you're putting into the blackboard.

To illustrate how this works, here is what we do early in the update function to get the elapsed simulation time:

double t=0;
for (const tm_entity_blackboard_value_t *bb = data->blackboard_start; bb != data->blackboard_end; ++bb)
{
    if (bb->id == TM_ENTITY_BB__TIME) {
        t = bb->double_value;
    }
}

Applying the engine

We're now able to take a look at the biggest function in the file. Where something actually happens and is applied to the scene.

This sample component applies a sinf modulation to the y coordinate of an entitys transform or link component. It uses the frequency and amplitude fields of our component and caches the incoming y coordinate on the y0 field to be used in the next update.

Let's quickly look at the signature of the update function:

void engine_update__sprite_component(
    tm_engine_o *inst,
    tm_engine_update_set_t *data);

The tm_engine_o* is the pointer to our entity context (which we provided when registering this engine), and the tm_engine_update_set_t* is what holds pointers to our blackboard values and our entity data to operate on.

For such a simple transformation of data, a whole bunch of stuff is going on in this component so I'll start with a rough outline:

• Create a temp allocator
• Initialize pointers for our lists of entities (one for link components, and one for transform components)
• Get a pointer to the link_component manager
• Get the current simulation time via the blackboard. (see above)
• For each tm_engine_update_array_t in the update set:
  • Get the pointer to the list of each component type
  • For each entity in this update array:
    • Update the sprite component y0 field.
    • Update the y coordinate on either the link or the transform component
    • Store the entity ID in either the link or transform
• For each entity that had a link component
  • Use the link component api transform function to update it's transform
• Use the entity api notify function to signal changes for the entities with just a transform component
• Destory the temp allocator

---

With this outline in mind, let's look more closely at a couple of things.

Temp allocator

The allocator is used by the carray utility code which implements a stretchy buffer. There are two macros which init and shutdown the allocator and they should bookend whatever scope will require it:

TM_INIT_TEMP_ALLOCATOR(ta);
/* ... all the code ... */
TM_SHUTDOWN_TEMP_ALLOCATOR(ta);

If you initialize a temp allocator but forget to use a matching shutdown, what happens? Memory leak? Yep... BUT a very nice thing about this macro is that it uses a simple trick to provide a warning for you when you forget to shut it down:

..\..\plugins\sprite_component\sprite_component.c(74,5): warning : unused variable 'ta_TM_SHUTDOWN_TEMP_ALLOCATOR_is_missing' [-Wunused-variable] [C:\projects\slowgames\CosmicTrash\build\sprite_component\sprite_compon
ent.vcxproj]
c:\projects\slowgames\OurMachinery\headers\foundation/temp_allocator.h(98,14): message : expanded from macro 'TM_INIT_TEMP_ALLOCATOR' [C:\projects\slowgames\CosmicTrash\build\sprite_component\sprite_component.vcxproj]
<scratch space>(11,1): message : expanded from here [C:\projects\slowgames\CosmicTrash\build\sprite_component\sprite_component.vcxproj]
  sprite_component.vcxproj -> c:\projects\slowgames\OurMachinery\bin\plugins\tm_sprite_component.dll

-----------------------------
tmbuild completed in: 2.088 s

Working with stretchy buffers

Throughout this update we're collecting entity IDs into two lists. We first initialize them as null pointers:

tm_entity_t *mod_link = 0;
tm_entity_t *mod_transform = 0;

With the previously initialized temp allocator, we can now push values on to these:

if (link) {
    /* snip */
    tm_carray_temp_push(mod_link, a->entities[i], ta);
}
else {
    /* snip */
    tm_carray_temp_push(mod_transform, a->entities[i], ta);
}

These stretchy buffers track their count and capacity in a header behind the address of the first item. Behind meaning, when allocating the buffer, the initial bytes are the header, then the pointer to the head of the array is the next address.

You can iterate over them using the usual C pointer shenanigans:

for (tm_entity_t *e = mod_link; e != tm_carray_end(mod_link); ++e) {
    /* snip */
}

All the operations you might need or expect when working with carrays are available. One interesting thing about them though is that you use them by including an "*.inl" file rather than a header. Most of the code seems to be macros.

Our example uses the temp allocator and associated carray macros, but there are other options, including the use of static memory.

Using the link_manager

One of the first steps in the function is to get a pointer to the link_manager:

const uint32_t link_component = tm_entity_api->lookup_component(ctx, TM_TT_TYPE_HASH__LINK_COMPONENT);
void *link_manager = tm_entity_api->component(ctx, link_component)->manager;

This is used at the end of the function in order to make use of the tm_link_component_api. We must use this to correctly update any linked transforms:

for (tm_entity_t *e = mod_link; e != tm_carray_end(mod_link); ++e) {
    tm_link_component_api->transform(link_manager, *e);
}

Iterating over the update data

The tm_engine_update_set_t struct has a field arrays which holds one or more tm_engine_update_array_t structs. These update array structs hold the lists of entities and component data.

The component data appears to be an array of pointers to arrays of component data actually. The component data lists are in the same order you specify in the tm_engine_i implementation used to register the update function.

When we get down to business with our component update work, we first need to iterate over these update arrays:

for (tm_engine_update_array_t *a = data->arrays; a < data->arrays + data->num_arrays; ++a) {
    /* snip */
}

Now for each of these we get our lists of component data:

struct tm_sprite_component_t *sprite_component = a->components[0];
tm_transform_component_t *transform = a->components[1];
tm_link_component_t *link = a->components[2];

Finally we can iterate over each entity:

for (uint32_t i = 0; i < a->n; ++i) {
    /* snip */
}

Just in case it isn't obvious, in this loop i is the index of the entity data, and not the entity ID. The actual entity IDs are stored in the entities field.

Updating the entity y coordinates using sinf

For the actual "work" to be done we're finally ready to modify some transforms! Yeee-haaw!

if (!sprite_component[i].y0) {
    sprite_component[i].y0 = transform[i].tm.pos.y;
}
const float y = sprite_component[i].y0 + sprite_component[i].amplitude * sinf((float)t * sprite_component[i].frequency);

if (link) {
    link[i].local_transform.pos.y = y;
    tm_carray_temp_push(mod_link, a->entities[i], ta);
}
else {
    transform[i].tm.pos.y = y;
    ++transform[i].version;
    tm_carray_temp_push(mod_transform, a->entities[i], ta);
}

Wrapping up!

Adding our component to one of the sample entities shows that it does indeed move in the Y-axis as expected. Updating the frequency and amplitude takes effect in real-time.

Here is a short screen cap. (How does one make gifs these days?)

This was also a rather large post. But thanks for taking a look while I refuse to read docs before playing around and making assumptions.

Having come to this point I'm barely scratching the surface of what is possible. There are a lot of directions I'd like to go from here. The next set of posts will be a bit more focused on learning how to do something specific I think. Armed now as I am with the rudimentary understanding of The Machinery, I'd like to take the next steps converting this example component into the 3D sprite component I have in mind. This probably means that I'd need to start by exploring the asset import workflow and learning more about what the creation graphs can do.