Abstract:
Please see the attached document named "cagri_latifoglu_proposal.pdf" for the actual proposal document which also includes a literature review.
Overview In this project we aim to further develop the Additive Reinforcement Learning Algorithm (ARLA) introduced in [1], so that the inference step will use the information collected in the learning phase to build a posterior probability distribution and make an inference using this distribution. Until now we have successfully applied ARLA in two problems: Binary Matrix Guessing, [1], and Nano Photonic Structure Design, [2].
We aim to: make a new version of ARLA where posterior distributions are used when making inference, improve ARLA so it learns from a lesser number of training examples, apply ARLA to design a new photonic structure, find another suitable problem where ARLA can be applied, and finally investigate the connection of ARLA to Hebbian Learning [3] along with ARLA's capability of learning from adversarially masked (corrupted) training examples.
We will use the following mathematical and computational methods to achieve these objectives: information theoretic characterization of the learning process using statistical learning theory and information theory, usage of exact mathematical formulations to create matrix-reward pairs which are used in training and if exact mathematical relationship between the matrix and the reward is not known (such is the case in Nano Photonic Structure Design), use simulations to create the matrix-reward pairs, use statistical learning approaches to reduce the number of training examples required for learning convergence, and finally use parallel programming software and hardware to improve the training and inference speed.
Intellectual Merit If we can achieve our project objectives, we will improve our's and scientific community's understanding of the exploration vs exploitation dilemma which is one of the central questions of Reinforcement Learning (RL) [4] by establishing the connection between ARLA, RL [5], and Hebbian Learning through the mathematical, statistical and information theoretical study. Furthermore, the scientific community will gain an easy to understand and implement, yet highly parallelizable learning algorithm which can be easily adopted to different learning domains. Finally, a new nanophotonic structure will be designed by ARLA which will furtherthe capabilities of nanophotonic circuits.
Broader Impacts of the Proposed Work If our project is successful, an important step towards understanding additive learning paradigm will be taken. If we can improve the mathematical characterization of ARLA (mainly how and why it works), this will lead to better understanding of a simple yet potent learning algorithm, hence design of better learning algorithms and better decision making by humans and Artificial Intelligence (AI) systems as a result.