Item Details

Scale-Invariant Temporal History (SITH): Optimal Slicing of the Past in an Uncertain World

Spears, Tyler
Format
Thesis/Dissertation; Online
Author
Spears, Tyler
Advisor
Sederberg, Per
Abstract
In both the human brain and any general artificial intelligence (AI), a representation of the past is necessary to predict the future. However, perfect storage of all experiences is not feasible. One approach utilized in many applications, including reward prediction in reinforcement learning, is to retain recently active features of experience in a buffer. Despite its prior successes, we show that the fixed length buffer renders Deep Q-learning Networks (DQNs) fragile to changes in the scale over which information can be learned. To enable learning when the relevant temporal scales in the environment are not known *a priori*, recent advances in psychology and neuroscience suggest that the brain maintains a compressed representation of the past. Here we introduce a neurally-plausible, scale-free memory representation we call Scale-Invariant Temporal History (SITH) for use with artificial agents. This representation covers an exponentially large period of time by sacrificing temporal accuracy for events further in the past. We demonstrate the utility of this representation by comparing the performance of agents given SITH, buffer, and exponential decay representations in learning to play video games at different levels of complexity. In these environments, SITH exhibits better learning performance by storing information for longer timescales than a fixed-size buffer, and representing this information more clearly than a set of exponentially decayed features. Finally, we discuss how the application of SITH, along with other human-inspired models of cognition, could improve reinforcement and machine learning algorithms in general.
Language
English
Published
University of Virginia, Psychology - Graduate School of Arts and Sciences, MA (Master of Arts), 2019
Published Date
2019-12-05
Degree
MA (Master of Arts)
Collection
Libra ETD Repository
Related Resources
https://arxiv.org/abs/1712.07165
Logo for Creative Commons Attribution LicenseCreative Commons Attribution License

Availability

Read Online