Amir Samani

I'm a Senior AI frameworks engineer at Intel working on Intel VPU to accelerate AI workloads. Before that, I was a graduate student at the RLAI lab, where I worked on online representation learning for reinforcement learning. I was advised by Richard Sutton.

My research interest is in deep reinforcement learning and sequential decision-making. More specifically, I am interested in agents that interact with complex environments and continually improve their representations and models of the environment to achieve their goals.

Email  /  GitHub  /  Google Scholar  /  LinkedIn

Research

Online Representation Learning for Reinforcement Learning

The notion of state is fundamental to a reinforcement learning agent. The state is the input to the agent’s action-selection policy, value functions, and environmental model. The interaction between the reinforcement learning agent and the environment results in the agent’s data stream of experience—the alternating sequence of actions performed by the agent and observations provided by the environment. In this short abstract, we discuss the importance of learning the agent state online with computational and memory complexity similar to standard RL algorithms such as semi-gradient TD methods.

Publications

Learning Agent State Online With Recurrent Generate-and-Test

We propose two methods for learning the agent state online based on the generate-and-test approach. We study the proposed methods on two online multi-step prediction problems. The first problem, trace conditioning, focuses on the agent’s ability to remember a cue for a prediction multiple steps into the future. In the second problem, trace patterning, the agent needs to learn patterns in the observation signals and remember them for future predictions. We show that our proposed methods can effectively learn the agent state online and produce accurate predictions.

MaxStream: SDN-based Flow Maximization for Video Streaming with QoS Enhancement

Along with the increasing demand for video streaming, network service providers and video content providers are challenged to maximize their service in terms of both quantity and quality while guaranteeing Quality-of-Service (QoS). In this paper, we propose MaxStream, a SDN-based flow maximization framework. The framework exploits the global view of the network available at the SDN controller to formulate integer multi-commodity flow problems with objectives to maximize the number of streaming sessions accommodated in a network to improve providers revenue (quantity of the service) and to maximize bandwidth provisioning for QoS enhancement (quality of the service). Our simulation results confirm that on average, MaxStream accepts 13% more streaming sessions and offers 50% increase in bandwidth provisioning compared to variations of the widest-shortest path algorithm.

Design and source code from Jon Barron's website