It took longer than anticipated, but the prototype described in the previous post has been coded and documented! There were a few changes and additions along the way, but that just goes to show how nothing is ever really final, until you sit down and produce it. Most notably was the addition of a new class called TagClass, which is used to verify if an observed object should be memorized, and how the memory should be flagged.

Before I outline how the program ended up working, you may download the Unity Package file here:

• Download the Unity Package file

While a lot of the functions in this program are fairly simple, there are quite a few of them. Because of this, I think it is a good idea to quickly summarize what happens when an object is observed in a stepwise manner.

When an object is being observed
  If line of sight is clear
    Submit the object for verification and eventual storage

When an object is being verified
  If the object's tag is on the list
    Submit the object for storage

When an object is being submitted for storage
  If no memory of the object exists
    Create a new memory
  Else if a memory of the object already exists
    Create new observation in existing memory

When a new memory is created
  A temporary instance of a MemoryClass is initialized
  An observation is added to the temporary memory
  The temporary memory is copied to the list of stored memories

When an observation is created
  If existing observations are too close (redundant)
    Delete existing observation(s)
  If adding new observation exceeds observation-per-memory limit
    Delete oldest existing observation
  An observation containing the current location of the object is added to the memory

Once again, each object observed can only have a single memory associated with them. However, each memory can contain multiple observations of the same object.

Additional features include the constant increase in decayTime/age on all observations, forgetting observations that have aged beyond a certain limit, as well as verifying and deleting memories whose referenced objects no longer exist, or have no more observations left.

Setup:

There’s a few things outside of the code which are important to be aware of, if you want to work with the solution yourself. Starting with the scene setup (scripts will be referenced by name, see the Code section further down for explanations to each of these):

The black lines are for the objects which have been configured as observable in this demo. CubeTag1 and CubeTag2 have been tagged with “Tag1” and “Tag2” respectively. These objects do not contain any scripts, only the tags are necessary for enabling observation.

The green lines are for the field-of-view objects, which physically describe the observer’s field-of-view. In this demo, there’s a cone and a sphere. Any object crossing into a field-of-view object will undergo the steps listed in the pseudocode above. These objects contain the ViewConeScript as a component.

The red lines indicate physical containers, which other things have been childed to. The SightHandler object contains a script called SightHandlerScript, which helps managing the field-of-view objects. I could also have attached that script directly to the Observer object and childed the field-of-view objects to that instead, but I chose to keep them separate. The Observer contains the MemoryScript, which does everything related to handling memories and observations.

If you want to test and see how it works, when running the program, clock on the Observer object and take note of the inspector properties on the MemoryScript attached:

The Redundancy Range indicates at which distance the observations of the same object, will begin to overwrite each other.

The Observation Limit indicates how many observations of the same object are allowed to exist. If this is exceeded, the oldest observation is overwritten by the newest one.

The Decay Threshold indicates how old an observation can become, before it is deleted from the memory. As the current configuration, the decay timer increases by 1 every 0.5 seconds. This can be changed, of course.

The Debugging bool is perhaps the most important thing to note right now. This has to be set to true for the scene view to display the debugging visualizations, which aid in understanding how the observations are stored. Try setting this to true and start moving the two cubes in and out of the active field-of-view.

As you will no doubt notice, as each cube enters the field-of-view, a purple line is drawn towards them, and they change colours to blue and red. The blue colour is for “Tag1“-tagged objects, the red colour is for “Tag2“-tagged objects:

When moving the cubes outside of the field-of-view, the red and blue representations stay inside. This is because the observer no longer has the cubes within observable range – the red and blue cubes indicate the observers observations of the observed objects, rather than the objects themselves.

By reentering the cubes into the field-of-view at distances beyond the redundancy range, it is possible to add multiple observations of the same objects. With the default configuration, up to three observations are possible. Just note that they currently disappear again after 50 seconds, as the decay threshold of 100 is reached (incrementing every 0.5 seconds).

This should give a pretty clear picture of how the system works in practice. I will also give a few hints as how it can be expanded to not only include additional tags, but have more than just two type-flags (like the red and blue right now). The type flags are not used for anything in this prototype except rendering the cubes different colours in debugging. For how I will actually use these in handling the memories stored, will be for a different prototype. For now, this is simply focused on the sight and memorization of objects.

Currently known issue: When moving an object at a high velocity through the field-of-view object, multiple observations may be recorded. This is due to the speed at which OnTriggerStay is executed, as the object may move beyond the redundancy range since the previous step, registering as a separate observation. To remedy this issue, try increasing the redundancy range, or reduce the speed of the object.

Code:

The script files for this prototype, are fairly easy to segment. One file for each the two custom classes, one for the memory logic and one for the viewcone object (to detect trigger collisions and pass the reference to the memory script for processing). Additionally, something that was not previously addressed, a “Tag” class will be created. This is for the list of tags, as each tag will also indicate if it should be heatmapped or not. This comment will be removed and the class will be backfilled in the documentation, once this post has been fully written:

1. – MemoryClass

This class contains the data which defines a memory. Memories can contain multiple observations of the same object, but there can only be one memory per object, per observer. This class is data-only and therefore does not inherit from Monobehaviour

2. – ObservationClass

This class contains the data which defines an observation. Observations are added to memories, as defined above. An observation contains the location it was made, as well as a counter for how long ago the observation happened. This class is data-only and does not inherit from Monobehaviour.

3. – TagClass

This class is used by the MemoryScript to verify if an object should be memorized or not. Additionally, it contains an indicator (a bool in this case) to help identify how the object should be treated, when memorized. This aspect is not addressed in this prototype, but know that the bool indicator can easily be swapped with an enumerator (or whatever you prefer) if you want more than just true or false.
4. – MemoryScript

This class handles almost everything listed in the problem definition. From redundancy checks to decay timers, line-of-sight checks, observation limits, creating and deleting both memories and observations etc. Anything related to handling the memories and observations, happens here.

5. – ViewConeScript

A simple class, attached to the field-of-view objects on the observer. It handles collision detection with objects in the scene, passing references of those objects to the MemoryScript for evaluation.

6. – SightHandlerScript

This class emulates the state changes in the finite state-machine governing AI behaviours, in the final build. By listening to button-presses on the keyboard, it illustrates what functions should be called to enable/disable sleep, as well as changing between the different field-of-view meshes.

1. MemoryClass:

Since this class is only supposed to store data and not be attached to a game object in the scene, it does not inherit from monobehaviour. This allows for the use of constructors (inheriting from monobehaviour does not allow this) when creating new instances of the class, which makes the code easier to read.

1.0 Fields:

Public GameObject reference

A reference to the GameObject being memorized. This is used in the memory system, when verifying if the object has been seen before (in recent memory, as the AI is built to forget over time).

Public bool shouldMap

This is an indicator that isn’t really used elsewhere in this prototype (except debugging). It is relevant for how the AI may handle the memorization in the final build, and can easily be changed to something with more flagging possibilities (eg. an enum instead of a bool). How this is implemented, really depends on the needs of the AI using the memory system.

Public List<ObservationClass> observations

A list of all observations associated with this memory. All observations listed in a memory, are for the object in the reference variable.

1.1 public MemoryClass(GameObject _reference, bool _shouldMap, List<ObservationClass> _observations):

This is the class constructor. All it does is take the function parameters and initializes the fields listed above.

2. ObservationClass:

As with MemoryClass, no inheritance is used here, to allow for the use of constructors

2.0 Fields:

Public Vector3 position

The position of the object, when the observation was made.

Public float decayTime

A timer which will be constantly incremented, to indicate how long ago the observation was made.

2.1 public ObservationClass(Vector3 _position)

The constructor for the Observation class. It simply initializes the position variable with the function parameter, as well as setting the value of decayTime to zero, to indicate that it is a fresh observation.

3. TagClass:

The third and final data-only class, which does not use monobehaviour inheritance, to allow for the use of constructors.

3.0 Fields:

Public string tagString

A string used to match against the tags of the objects being memorized.

Public bool shouldMap

A “flag” intended to be used by the AI when handling the memory. It does not have any direct uses in this prototype, except for showing how it works in debugging. That means it can easily be changed for a different type with more than just the two true/false states (like an enum, for example), if more states are desired.

4. MemoryScript:

This script contains everything related to handling memories and observations. Each object in the scene can be stored as an individual memory, but multiple observations of the same object can be stored in the same memory.

4.0 Fields:

Almost all of the fields in this class, are ones that either need to be exposed to the inspector, or be accessed across multiple functions. Choosing this design was due to the complexity of the program, as it is much easier to pass values along chains of functions, than having to keep track of what accesses what at which time.

List<TagClass> verifiedTags

A list of TagClass objects, which contains strings describing the tags that can be memorized, as well as the type flags for how these memories and their observations should be interpreted.

List<MemoryClass> memories

A list of MemoryClass objects, which contains references to the objects memorized, the type indicator for how the memory should be interpreted as well as a list of ObservationClass objects for the observations.

Float redundancyRange

A float indicating how far away observations must be made from each other, before they appear redundant.

Int observationLimit

An int indicating how many observations can be stored within a memory.

Int decayThreshold

An int indicating how “old” observations are allowed to become, before they are deleted.

Bool debugging

A bool indicating whether or not the debugging visuals should be displayed in the scene view, when running the project in Unity.

4.1 Awake():

Awake is called after all objects are initialized. It initializes the memories list, calls the InitializeTags-function ensuring that all the tags and their type flags have been created, then calls InitializeChildren, which connects the field-of-view objects to the observer. Finally, the memory-handling (redundancyRange, observationLimit and decayThreshold) variables have their values defined.

4.2 void Start():

Start handles the call of the looping Decay coroutine, which defines the pace at which the decay value increases for the observations.

4.3 void InitializeTags():

InitializeTags handles the initialization of the TagClass objects, which are used to evaluate and observed objects. If any new types of objects should be memorized, it is within this function that their tags and type flags should be initialized and added to the verifiedTags list.

4.4 void InitializeChildren():

InitializeChildren finds the field-of-view objects and passes a reference of the MemoryScript class to them. This is needed for the field-of-view objects to be able to call Observe function, passing the objects they collide with to the observer for evaluation and possible storage.

If additional field-of-view objects are added to the observer object (in the scene), their directory should be specified in the toFind string list constructor.

4.5 public Observe(Gameobject _go):

Observe is the function which the field-of-view objects call, when another object enters their trigger area. Via the collider of the colliding object, a reference to the gameobject is passed as a parameter, _go, which Observe then passes to the SightCheck function. If there is a direct line of sight to the object being observed, pass the object reference to VerifyTag for further evaluation and possible storage.

4.6 bool SightCheck(GameObject _go):

SightCheck performs a raycast from the observer object to the object passed as the _go parameter. If the raycast returns a reference to the same object as _go, the line of sight is clear. This result causes SightCheck to simply return true. If the raycast returns a reference to a different object however, the line of sight is obstructed. This result causes SightCheck to return false.

4.7 void VerifyTag(GameObject _go):

VerifyTag looks at the tag of the GameObject _go, comparing it to each of the strings stored in the TagClass objects in verifiedTags. If a match is found, the GameObject is passed onto the UpdateMemory function for storage, alongside the type flag from the TagClass object which matched the tag.

4.8 void UpdateMemory(GameObject _go, bool _shouldMap):

UpdateMemory updates the memory of the observer. A call to CompareMemory is made, passing _go as a parameter. The result is stored in an instance of MemoryClass named toRemember.

If no memories of the current object exists (the call returned null), a new one is created with a call to CreateMemory, passing _go and _shouldMap as parameters for appropriate storage.

If a memory of the object already exists, an observation will be added to that memory, by a call to CreateObservation, passing a reference to the memory which was found in the toRemember variable.

4.9 MemoryClass CompareMemory(GameObject _go):

CompareMemory checks for existing memories of the _go object. This is done by examining the GameObject reference stored in each memory, to see if any match the _go parameter (meaning that the _go object has already been memorized). If a match is found, a reference to the memory where _go is stored, is returned. Otherwise return null to indicate that no match was found.

4.10 void CreateMemory(GameObject _go, bool _shouldMap):

CreateMemory takes a reference to the GameObject _go which should be memorized, alongside the bool _shouldMap to create the memory with the appropriate type flag. These are simply passed into the temporary MemoryClass newMemory’s contructor. An observation is then added to this memory by a call to CreateObservation, before adding the new memory to the memories list.

4.11 void CreateObservation(MemoryClass _memory):

CreateObservation adds a new observation to the _memory being referenced. Before a new observation can be added, it is first necessary to check the existing observations for redundancy (if any observations of the same object, appear too close to the current observation, the previous observation will be removed). This is done with a call to RedundancyCheck.

Next, a check to ensure that adding another observation won’t exceed the allowed limit of observations in a single memory, by calling LimitCheck. If this limit would be exceeded, the oldest observation in the memory will be deleted.

Finally, a new observation is initialized by passing the current transform of the GameObject reference stored in the memory, into the ObservationClass constructor. This observation is then  added to the list of observations in _memory.

4.12 void DeleteMemory(MemoryClass _memory):

DeleteMemory simply deletes the _memory being passed from the memories list. There is only a single line of code, but since the deletion is considered an individual action, a function was written for it.

4.13 void DeleteObservation(MemoryClass _memory, ObservationClass _observation):

DeleteObservation removes the _observation referenced in the list of observations stored in _memory. Once again, this is only a single line of code, but has its own function due the deletion being considered an individual action.

4.14 void RedundancyCheck(MemoryClass _memory):

RedundancyCheck performs a distance calculation between the current location of the GameObject reference and any existing observations, to check if the previous observations are too close to the new one. If the distance measured is too short (the old observation has become redundant), the old observation will be removed. This is done by adding all observations that are too close, to a temporary list called deleteThis. Once all the distance checks have been made, all entries from deleteThis will be used to remove the observation referenced within, from the _memory.

4.15 void LimitCheck(MemoryClass _memory):

LimitCheck checks if adding another observation to the list of observations in _memory, would exceed the observationLimit. If this is true, each of the existing observations will be checked, to find the one with the highest decayTime value (the oldest one), which is then deleted with a call to DeleteObservation.

4.16 void VerifyMemories():

VerifyMemories checks the GameObject reference variable of each memory stored in memories. This is to check if any of the objects memorized has been deleted since, then delete any memories whose reference returns null.

Additionally, if all observations in a memory have expired, the memory will be deleted as well, as no meaningful data is then considered to be available regarding the reference.

4.17 void IncrementDecay():

IncrementDecay iterates through all observations in all memories, incrementing the value of decayTime in each. Should the value of decayTime exceed the value of decayThreshold, the observation will be deleted.

4.18 IEnumerator Decay():

Decay is the coroutine responsible for calling the IncrementDecay function at a set interval, by waiting a bit, making the call, then calling itself afterwards. Additionally in this prototype, a call to VerifyMemories is made as well. However, it is better suited to be called at a higher clock rate and is only called here as a proof-of-concept.

4.19 void OnDrawGizmos():

OnDrawGizmos is a built-in Unity function, which is used to help with debugging, by drawing cubes in the Scene view that represent the Vector3 coordinates stored in each observation. To help make it clearer what kind of type flag the observations are stored under, the shouldMap variable is used to determine if the cube should be red or blue.

5. ViewConeScript:

This script is attached to the field-of-view objects on the observer. It listens to the colliders on these objects, and passes objects which move into them, to the Observe function in the observer’s MemoryScript.

5.0 Fields:

Public MemoryScript parent

Contains a reference to the MemoryScript attached to the observer parent object. This reference is established by the MemoryScript itself, with a call to its InitializeChildren function.

5.1 public void OnTriggerStay(Collider other)

OnTriggerStay is called each frame that another object is within the collider of the field-of-view object. The other references the collider of the other object, which can be used to find the GameObject associated with it and pass it to the Observe function in MemoryScript.

5.2 private void Start():

This Start function exists purely for debugging purposes. All it does it write the name of the field-of-view object and the name of the parent containing the MemoryScript, to verify that the reference was established properly.

6. SightHandlerScript:

This script allows for toggling between the field-of-view objects, so that only one is active at a time. It also allows for disabling all of the field-of-view objects, to emulate a state of sleep where the observer does not observe anything.

6.0 Fields:

List<GameObject> viewFields

A list of references to all existing field-of-view objects.

Public int activeIndex

An indicator for which one of the field-of-view objects should be active. It is simply used as the index value when accessing viewFields.

Bool sleeping

A bool to indicate if all of the field-of-view objects should be disabled, to emulate sleep.

6.1 void Start():

Start initializes the three fields, as well as giving some default values to activeIndex and sleeping. It also calls FindChildren to establish a reference to all of the field-of-view objects, before calling DisableChildren to turn them all off. Finally, a call to SetActiveIndex is used to turn the appropriate field-of-view object back on.

6.2 private void Update():

Update listens to keypresses to illustrate how it is possible to toggle through the active index and change the active field-of-view object, as well as enabling and disabling the sleep functionality via EnableCurrent and DisableChildren.

6.3 void FindChildren():

FindChildren looks through a list of strings in its child objects, to identify the field-of-view objects and store references to them in the viewFields list.

6.4 void DisableChildren():

DisableChildren iterates through the viewFields list and disables all of the field-of-view objects.

6.5 void EnableCurrent():

EnableCurrent  uses the activeIndex value to enable the field-of-view object stored at that index in the viewFields list.

6.6 void SetActiveIndex(int _index):

SetActiveIndex disables the currently active field-of-view object and enables the one at the index value passed. It does not do anything if invalid values are passed. For testing purposes, it currently just cycles through the activeIndex by incrementing its value and resetting it to zero if exceeding the number of viewFields.Count.