|
|
Tutorial 5. Planning using Partially Observable Markov Decision Processes
Topic
Real-world planning problems are often characterized by partial
observability, and there is increasing interest among planning researchers
in developing planning algorithms that can select a proper course of
action in spite of imperfect state information. This tutorial provides an
introduction to research on planning using partially observable Markov
decision processes (POMDPs), a very general and increasingly popular
framework for planning under partial observability. The focus of the
tutorial is on planning algorithms for POMDPs (in contrast to
reinforcement learning approaches), and one of the goals is to show the
continuity between traditional planning research and research on planning
using POMDPs. In particular, coping with the challenge of partial
observability requires development of techniques of knowledge
representation, abstraction, problem decomposition, and heuristic search
that generalize similar techniques used by traditional planners.
Intended Audience
No previous knowledge of Markov decision processes is assumed, and the
tutorial will be accessible to anyone with basic familiarity with AI
planning. Those who attend will acquire an understanding of the central
concepts and algorithms for POMDPs, as well as a broad overview of recent
research developments and applications.
Presenter Information
|
Eric Hansen is an
Associate Professor of Computer Science and Engineering at
Mississippi State University. He received a A.B. in Philosophy
from Harvard University, and a Ph.D. in Computer Science from the
University of Massachusetts at Amherst in 1998. His PhD
dissertation introduced a policy iteration algorithm for POMDPs
that represents policies as finite-state controllers, and he has
published several methods for scaling up algorithms for solving
POMDPs. Other research interests include heuristic search and use
of model-checking techniques in search and planning. He received
an NSF Career Award in 2000.
|
 |
Daniel Bernstein is
currently working towards a Ph.D. in Computer Science at the
University of Massachusetts at Amherst. He received a B.S degree
in Computer Science from Cornell University and an M.S. degree
from the University of Massachusetts. His research interests
include planning under uncertainty, reinforcement learning, and
resource-bounded reasoning. His dissertation work focuses on
planning for distributed agents in stochastic environments.
Daniel is a recipient of an NSF Graduate Research Fellowship and a
NASA GSRP Fellowship. |
 |
Zhengzhu
Feng is a PhD student in the Computer Science Department at
the University of Massachusetts at Amherst. He received a
B.S. degree in Applied Mathematics from the South China University
of Technology and an M.S. degree in Computer Science from
Mississippi State University. His research interests include MDPs
and POMDPs, and especially use of various forms of problem
structure in scaling up planning algorithms for these models. |
 |
Rong Zhou
is a PhD student in Computer Science at
Mississippi State University. He received a B.S degree in
Electrical Engineering from Xi'an Jiaotong University, an M.S
degree in Electrical Engineering from Tongji University, and an
M.S. degree in Computer Science from Mississippi State
University. His research interests include memory-efficient
graph-search algorithms, as well as planning algorithms for
POMDPs |
Tutorial 1
Tutorial 2
Tutorial 3
Tutorial 4
Tutorial 5
|
|
