Located in the Whidbey Basin of the northern reaches of Puget Sound, Possession Sound contains the Snohomish River estuary, encompassing a river system that is the third largest contributor of freshwater to the Puget Sound. Myself, and several of my fellow students have had the opportunity to collaborate with the University of Washington, the Washington State Department of Ecology, and a local environmental consultant -- Gravity Marine. Utilizing data collected by permanently moored Sea-Bird CTD probes, during research cruises, and by the United States Geological Survey (USGS), my research partner and I have created a model to facilitate an investigation of how both tides and the discharge of freshwater from the Snohomish River influence the water quality of Possession Sound. To better understand the complex patterns of the water entering Possession Sound from the Snohomish River, we analyzed the relationship between a continuous stream of water quality data (temperature, salinity and turbidity) from a Sea-Bird CTD probe at the mouth of the Snohomish River and continuous discharge data gathered by the USGS at a station 12 miles up the river. The distance between these two sites results in a delay between when river discharge data is recorded up river and when it influences the water quality at the river mouth. From the analysis of these two locations and with guidance from ourcollaborators as well as outside professionals, we used the statistical analysis language R to create a model that predicts the travel time of water from the USGS stream gage to Possession Sound. This model can be applied when considering the effect on the estuary of important factors from the river, such as nutrient loading; influxes of cold water, which promotes upwelling; and the river’s contribution of heavy metals and other pollutants.

Robotics combines machining and artificial intelligence to create real world humanoid models for task automation and industrial applications. We have designed an in-house robot prototype having microprocessor controlled motion. The robot has lasers for eyes and has a position sensor with camera attached. We designed the gear box, track assembly and robot parts and have written software to control the motion of the robot. The robot is good model for Roomba like vacuum cleaner. We create random walls using Monte Carlo simulations and used vector directed motion to control its motion for avoiding these random walls that the robot encounters to simulate real world experience. We have also studied robotic arm kinematics, using matrix algebra and trigonometry to help design a robot arm that we can rotate or translate to any point in three -dimensional space. We study both forward and reverse kinematics and have written software for the arm motion. Our studies provide an elegant educational platform for studies of robot motion along with simulating real-world experience.

Reinforcement Learning (RL) is a subcategory of machine learning, in which an agent (the decision maker) observes its environment and executes the best course of actions to maximize rewards. This is similar to teaching a pet to perform tricks using treats as positive reinforcement. Our research compares different RL methods on low-performance devices like a Raspberry Pi in real-time, real-world environments. RL has gained popularity recently with breakthroughs from DeepMind’s paper, Playing Atari with Deep Reinforcement Learning, where an agent learns to play Atari games from raw pixels and from DeepMind’s AlphaGo (DeepMind, https://deepmind.com/research/alphago) program that was the first computer program to beat a world champion Go player. RL projects like AlphaGo have utilized big data, powerful computing resources, and simulated environments that do not require real-time interaction to train the machine learning models. Our group compares the effectiveness of different RL methods on an accessible level of computing power on offline devices that an average consumer could acquire. The team constructs a physical environment for the robot to navigate, creates an OpenAI Gym environment that our agents will use to control the robot and get feedback from the environment. We train our agents using different RL methods to optimally navigate the environment and avoid collisions. We then compare the performance of the different methods in our physical real-time environment. Reinforcement Learning in small, offline devices could pave the way for a variety of devices that learn over time without being connected to a network. Imagine a small Mars rover that learns to navigate its environment efficiently over time.