Monday, March 31, 2008

Reminder: Reading for next class: Decision Theoretic Planning

Folks:
A reminder about the reading for tomorrow's reading.
Please note that about half the students who took 471 with me have already
had some introduction to MDPs. So, the discussion is likely to be faster paced.
It is thus imperative that those who haven't had exposure to MDPs read
the material
once before coming to class.

thanks
rao

---------- Forwarded message ----------
From: Subbarao Kambhampati <rao@asu.edu>
Date: Thu, Mar 27, 2008 at 6:12 PM
Subject: Reading for next class: Decision Theoretic Planning
To: Rao Kambhampati <rao@asu.edu>


Here is the paper for the next week.

http://www.cs.cmu.edu/afs/cs/project/jair/pub/volume11/boutilier99a.pdf


You should read at least upto page 35 for next week.


Rao

Fwd: [CSE574 Planning & Learning] Reminder: Seminar today by Dana Nau (11:30AM BY2...

---------- Forwarded message ----------
From: Subbarao Kambhampati <subbarao2z2@gmail.com>
Date: Mon, Mar 31, 2008 at 6:42 AM
Subject: [CSE574 Planning & Learning] Reminder: Seminar today by Dana
Nau (11:30AM BY2...
To: subbarao2z2@gmail.com


Planning for Interactions among Autonomous Agents

Dana S. Nau
Department of Computer Science
University of Maryland
College Park, MD

Date: Monday, March 31, 2008
Time: 11:30 AM - 12:45 PM
Place: Brickyard Building, Room 210

This talk will focus on ways to plan an autonomous agent's
interactions with other autonomous agents. Sometimes it is feasible to
model the other agents' possible actions as nondeterministic outcomes
of our agent's own actions. For this case, we can plan how to achieve
a desired goal using symbolic model checking, HTN planning, or a
combination of the two. Sometimes it may not be feasible to generate a
plan or policy that goes all the way to a goal, either because the
search space is too large or the goal is ambiguously defined. For such
cases, it can work well to interleave planning and execution if we
have a good predictive model of how the other agents are likely to
behave. The talk will present theoretical foundations and algorithms
for the above situations, and experimental results on the
Hunter-and-Prey domain, the Iterated Prisoner's Dilemma with Noise,
and other multi-agent planning domains.


Biography

Dana Nau is a Professor of both Computer Science and Systems Research
at the University of Maryland, and is co-director of the university's
Laboratory for Computational Cultural Dynamics.He has more than 300
refereed technical publications on automated planning, search
algorithms, game theory, and other topics.He has received an NSF PYI
award, several best-paper awards, and several prizes for the
performance of software systems.He is a Fellow of the Association for
the Advancement of Artificial Intelligence (AAAI).

==============

--
Posted By Subbarao Kambhampati to CSE574 Planning & Learning at
3/31/2008 06:42:00 AM

Reminder: Seminar today by Dana Nau (11:30AM BY210) on "Planning for Interactions among Autonomous Agents"

Planning for Interactions among Autonomous Agents

Dana S. Nau
Department of Computer Science
University of Maryland
College Park, MD

Date: Monday, March 31, 2008
Time: 11:30 AM - 12:45 PM
Place: Brickyard Building, Room 210

This talk will focus on ways to plan an autonomous agent's
interactions with other autonomous agents. Sometimes it is feasible to
model the other agents' possible actions as nondeterministic outcomes
of our agent's own actions. For this case, we can plan how to achieve
a desired goal using symbolic model checking, HTN planning, or a
combination of the two. Sometimes it may not be feasible to generate a
plan or policy that goes all the way to a goal, either because the
search space is too large or the goal is ambiguously defined. For such
cases, it can work well to interleave planning and execution if we
have a good predictive model of how the other agents are likely to
behave. The talk will present theoretical foundations and algorithms
for the above situations, and experimental results on the
Hunter-and-Prey domain, the Iterated Prisoner's Dilemma with Noise,
and other multi-agent planning domains.


Biography

Dana Nau is a Professor of both Computer Science and Systems Research
at the University of Maryland, and is co-director of the university's
Laboratory for Computational Cultural Dynamics.He has more than 300
refereed technical publications on automated planning, search
algorithms, game theory, and other topics.He has received an NSF PYI
award, several best-paper awards, and several prizes for the
performance of software systems.He is a Fellow of the Association for
the Advancement of Artificial Intelligence (AAAI).

==============

Friday, March 28, 2008

Need for assessment in the class??--feedback requested (can be on the blog)

Folks

Now that I have an idea of what semester projects are being attempted
by the various students, I am beginning to wonder
what the grade should be based on in the event that the semester
project doesn't end up all that well.

Now, in real world, if your project doesn't work, your paper will just
be rejected and that is that. However, since this is a course
and you need a letter grade, we need some basis for that. [There is
the option of just giving everybody the same high grade--but I am
afraid that might adversely affect my popularity...]

As of now, we had one homework, and one mini-project. I definitely
plan to release a second homework to cover everything since the
first homework.

In addition to that, I am thinking that there should be some sort of
"breadth of understanding" assessment. We can do this assessment
either around now (specifically, there is a good chance I will miss
the class of April 7th-- and that is always a goodtime to have an
exam), or
at the end of the semester during the finals week.

My assumption is that if someone does a great semester project, that
trumps over everything else. However, if the semester project doesn't
work out all that well--these other assessment mechanisms can help in
giving a grade.

I solicit your opinion(s) on this (either on the blog or by anonymous
email to http://rakaposhi.eas.asu.edu/cgi-bin/mail?rao )

thanks
Rao

Over-sensing and LCW actions.. (as well as planning with sensing as internet planning)

1. We talked at length about how over-sensing during execution can
delay things and lead to
decidedly non-intelligent behavior (e.g. Sphexishness).

One relevant issue here is that if you *know* the state of the world
in some aspect, and you know that
you haven't done anything to change it, then there should be no reason
for you to check it again.

Assuming a single-agent world, if you start with complete state and do
deterministic actions, then you never have
to look (as we discussed). Even if we start with incomplete initial
state, we may know "everything" about *some aspects*
of the world. As long as the ensuing actions--of ourselves--don't
modify the completeness of that knowledge, we don't
need to look *for that aspect* of the world.

It all starts to look like managing "closed world" assumptions. In the clasical
planning, we know init state, and so we can start with closed-world
assumption. No actions modify that closed world assumption.
In general belief-space planning, we don't have full closed world
assumption, but may have *partial* closed world assumption. As
we do actions, we may lose or acquire closed world assumption. In a
desktop (or unix) world, for example, we may start knowing
names of the files in the current directly as well as the sizes of all
the files in the directory. After we run a latex command, we still
know
the names of all files in the directory (even though latex makes new
files-- we know what they will be -- .aux, .bbl etc). We however no
longer
know the sizes of all files (since the sizes of .aux and .bbl files
generated will depend on the files you latexed in a complex way and
you can't
model it a priori). So, we lose closed world knowledge of the file
sizes. If we need that, we will have to do an "ls -s" action (a
sensing action).
If we just happen to do "rm *" action in that directory, we again get
full knowledge on both files and sizes of the directory.

In the paper below
http://www.cs.washington.edu/homes/etzioni/papers/xii-aaai94.pdf
Golden et. al. formalize this notion of starting with and tracking
local closedworld assumptions (LCWs) .
Their main contribution is to note not only the normal effects of the
actions, but also their meta-effects on closed world assumptions
(e.g. see the latexing and rm'ing actions above). Neat paper to read.


=========================================

2. When discussing progression planning in the presence of sensing
actions, I pointed out that there are two non-deterministic
branches: one which picks a causative action to execute and the other
which picks a sensing action to execute.
I also mentioned that if you always pick the causative action branch,
you get "conformant" or "no-sensing" plans (if you succeed).

A related question is what happens if you always pick only sensing
action branch? You get a pure sensing plan.
We can see a use for "pure causative plan" (conformant plan)--an agent
which has no sensors has to deal just with those.
Of what use can pure sensing plans be?

Well--if all you can do is sense some database, then your plans will
be just pure sensing actions. In particular, plans whose main purpose
is to
gather information can be thought of in terms of pure (or almost
entirely) sensing plans.

When you do planning on the web--for example--most often, your actions
involve sensing (look at this database, take a value from there and
plug it into a sensing query for this other database etc.) that leave
the world as it is, and only modify *your knowledge* of it.
Of course, you can also sometimes have causative actions (e..g.
updates--credit card
transactions etc) that modify the state of some database, and not just
the state of your knowledge.

So, not surprisingly, planning for information gathering involves
mostly things of pure sensing actions. The good part about sensing
actions is
that there are never any negative interactions among them (your brain
doesn't explode because you learned knowledge in the wrong order ;-).
So, for sensing planning the big issue is not so much about subgoal
interactions, but rather about reducing sensing.

Not surprisingly, the LCW stuff discussed above--wind up being relevant.

See http://rakaposhi.eas.asu.edu/ijcai-ig.pdf
which talks about how LCW information can be used to reduce the number
of information sources
that the agent has to sense to get all answers for a query.

(The paper http://rakaposhi.eas.asu.edu/ig-tr.pdf
provides a somewhat dated tutorial introduction to planning for
information gathering).

Rao

Thursday, March 27, 2008

Tuesday, March 25, 2008

Re: A short little paper for next class reading..

You can also get a much shorter overview by reading pages 73-76 of

http://rakaposhi.eas.asu.edu/pgSurvey.pdf

(the AI Magazine tutorial paper on PG heuristics that you already
printed and read a part of
back when we were doing classical planning heuristics)

Rao


On Tue, Mar 25, 2008 at 5:44 PM, Subbarao Kambhampati <rao@asu.edu> wrote:
>
> http://rakaposhi.eas.asu.edu/dan-jair-pond.pdf
>
> (As an added bonus, section 2 provides an overview of the progression
> and regression with and without sensing actions in belief space)
>
> Rao
>

A short little paper for next class reading..

http://rakaposhi.eas.asu.edu/dan-jair-pond.pdf

(As an added bonus, section 2 provides an overview of the progression
and regression with and without sensing actions in belief space)

Rao

Friday, March 21, 2008

Added several slides to yesterday's lecture

I added several slides to yesterday's lecture to better reflect the
discussion in the class.

rao

Tuesday, March 18, 2008

Why is this still a problem?

What is the main problem in doing an analysis on the ruleset before the planner gets to work to determine which constraints are reasonable to disregard during the planning process? Does it make programming the ruleset too hard up front? The number of available resources has to be defined anyway right? So the only extra work is providing an estimation of the usage of each resource or an estimation of its availability. If there's 10 taxi in the city you have an impression that you won't be able to find one because more than 10 people use a taxi in an your travel window to get from place to place. In a more cognitively plausible sense we just have an impression that taxi's will be available from past experience.

If it is unreasonable to expect someone to tell us this availability information directly there is still easy work that could be done. There are easy cases that could be taken care of up front. In the blocks world example, if the world has four arms, and you only have 3 blocks, its evident enough that the number of arms aren't a restriction and so this responsibility can be passed on to the scheduler.

In a more realistic project planning example you could create a graphplan for a few levels deep and count the number of actions that are done in parallel on each level and compare this to resource availability to determine which constraints are unnecessary. Using this information you could change the domain on the planner side to be simpler and then proceed using whatever method you wish from there.

So, why is this concept at the forefront of research? There seem to be obvious places to make initial progress, I would think it would be standard in implementations by now.

Reading for Thursday's class (belief states...)

This would be a good reading for the next class

http://rakaposhi.eas.asu.edu/cse574/aips00-incomplete.pdf

Rao

Monday, March 10, 2008

Is CSP scheduling better than IP scheduling?

Do I understand correctly that the slides say that CSP scheduling (based on PCP) performs better than IP based scheduling? That sounds strange: because there are mature IP-solvers and there are straightforward IP encodings (that are not part of AI, I believe) for job-shop scheduling problems. More that that, PCP does not manage well multi-capacity resources (which are widespread); and that kind of condition should not matter to IP.
Oleg Bakun

Saturday, March 8, 2008

Planning and Scheduling

Some most important discussions in the class were:

Planning and scheduling are separate problems as such but go hand-in-hand. A scheduled plan is one which has the plan and a schedule for that plan with time and resource constraints.

Scheduling can be:
-called as a special case of planning with resource constraints
-called as an allocation problem that allocates resources to time points and resources

A planner solves the problem and a plan serves as an input to the scheduler. This input is similar to a disjunctive temporal network which can have either unary or multi capacity resources which are nothing but unary or n-ary mutual exclusions.

Even when the resources are available before or while planning, we would want to go in a hierarchical fashion, ignore the resources and produce a plan and leave the rest to a scheduler for computational reasons.

Planning and Scheduling can be considered as two ends of a spectrum with maximum and minimum level of choices respectively. There are a number of interesting problems in-between these two points in the spectrum.

A scheduling problem can be modeled as a CSP.
Start point representation and PCP representations for CSP were discussed.
PCPs do perform well as it converts the the disjunctive temporal constraint problem into simple temporal problems that are really much more feasible when compared to the start point representation.
One problem with PCP representation is they do not scale very well for multi resource constraints.

Thursday, March 6, 2008

Re: CSE574 grades

I hope to give you the projects and homeworks and term paper proposals back
by the time you get back from Spring break.

Regarding the relative weight of the assignments--I am not too sure
yet. In general
I think about [30- 40%] for homeworks etc, [10-20%] for participation
and [40% -50%] for semester project.

By the way, I should tell you that one idea I am toying with is to do
an oral midterm
(Y'all get to come for half-an-hour each and the teaching czars and I
will "interview" you
on what happened until now).

Rao


On Tue, Mar 4, 2008 at 7:59 PM, Oleg Bakun <Oleg.Bakun@asu.edu> wrote:
> Hello,
> When grades and their averages for HW1 and Project 1
> will be published?
> What is relative weight of each assignment?
>
> Oleg Bakun
>
>
>
>
> ____________________________________________________________________________________
> Looking for last minute shopping deals?
> Find them fast with Yahoo! Search.

http://tools.search.yahoo.com/newsearch/category.php?category=shopping
>

Slack-Based Heuristics Scheduling Paper

The slack-based heuristics for constraint satisfaction scheduling paper visits a number of points that we have seen through the semester, although it doesn't call them what we know them as. Here are a couple of them:

1. In the introduction, 2 different methods of constraint-based scheduling are mentioned: that of posting start times, and the method which the authors use, of posting sufficient additional sequencing constraints between pairs of operations contending for the same resource. It should be really clear that (in planning lingo) this is in essence totally-ordered plans vs. partially-ordered plans: as the authors point out, the solutions generated by the latter method "... typically represent a *set* of feasible schedules" (emphasis added). These are but partially-ordered plans; any topological sort of them will yield a total-order that is executable.

2. This second point probably has a lot to do with the fact that the method proposed deals with constraints as a CSP would: the initial configuration of variable and value ordering heuristics as defined by the authors is something we have already seen in CSP.

Min-Slack: This is similar to the Most-Constrained Variable (MCV) selection in CSP.

Max-Slack: This is similar to the Least-Constraining Value selection in CSP.

kartik

Added slides on quotienting and candidate set semantics for temporal planning

I added slides to last classe's lecture notes to reflect the
whiteboard discussion we had about
quotienting and lifted spaces and candidate set semantics for lifted
planning (this was done in
response to a question from Tuan).

FYI
Rao

Wednesday, March 5, 2008

Readings for Tomorrow..

Tomorrows agenda is

--> discuss any questions you might have on TCSPs (especially handling of STPs)

--> Discuss next topic: Scheduling

Reading:

Required: (required and in order)

Dana Nau text book 15.1--15.3

If possible, also read
http://rakaposhi.eas.asu.edu/cse574/smith-cheng-slack-aaai93.pdf (6
page paper on a scheduler)


============

Also, after the break, we will have a class to discuss the paper
http://rakaposhi.eas.asu.edu/cse574/KER00.pdf

(which talks about integrating planning and scheduling)

Tuesday, March 4, 2008

URL for unedited feedback comments...

is

http://rakaposhi.eas.asu.edu/cse574/s08-feedback.pdf

They should give you a more complete idea of what the class thinks and whether
it jives with your own view.

If you think the comments have a different meaning that I spun them for,
feel free to tell me that ( http://rakaposhi.eas.asu.edu/cgi-bin/mail?rao

)

regards
Rao

ZENO and Nondeterministic Choices

In the description of the ZENO algorithm, there is an interesting point (that didn't come up for discussion in class today), which we have covered earlier in the semester. While describing the nondeterministic decisions in the algorithm, the authors list 3 points: decomposing complex goals, choosing actions, and introducing constraints to prevent interference. They then go on to state that completeness requires backtracking on these decisions. That is, just finding a satisficing solution can only be guaranteed if the "nondeterministic" choices are determinised.

However (and this they point out subsequently), we have already seen earlier that for plan-space planners, the order in which the subgoals are selected for satisfaction has no effect on completeness. That is, unless we want to find the optimal solution, we need not determinise (introduce backtrack into) the order in which goals are selected.

This raises some interesting points:

1. Could this dichotomy be seen as a manifestation of the fact that while it is true that *all* subgoals need to be satisfied (at least in non-PSP scenarios) to have *a* solution, a given solution need not contain *all* the possible decompositions, or actions, or constraints?

2. The second part of 1. can perhaps be stated better as follows: any given combination of actions, simple goals and constraints is not necessarily a subset of any given solution, for all solutions in the spaces; however, any given subset of the subgoals is *necessarily* a subset of the subgoal set needed for top-level goal achievement.

These are just some of the things I could think of; I thought this was a very interesting point put forward by the authors. Please feel free to add your own.

kartik

One thought about completeness of ZENO and Decision Epoches planners

When reading these papers, I'm curious about why ZENO is complete but DEPs are not. One of the reasons I found is the difference between how they model the interaction/ordering between actions in the plans. ZENO post constraints between time points of actions, whereas DEPs uses advancing techniques. More interesting is that, in term of refinement planning, both of them can be considered as refinement operators, and advancing to decision epoches is not a complete refinement.
This difference does not exist in classical planning (i.e comparing POP v.s Progression/Regression) because decision epoches are not there and fattening and advancing work together...
A.Tuan

Monday, March 3, 2008

Final chance: Feedback opportunity to tell me how the class is going

Folks

Today is your last chance to send the midterm feedback. I will
summarize what I got tomorrow (and discuss in the
class if needed).

I got feedback from 5 students. The class has 12 registered
students--so this is not even a majority.
If it helps you galvanize to action, let me teasingly say that the
majority feedback received till now says
everything is upsy-daisy and requests if any thing for more homeworks and exams.

Rao

---------- Forwarded message ----------
From: Subbarao Kambhampati <rao@asu.edu>
Date: Tue, Feb 26, 2008 at 9:20 PM
Subject: Feedback opportunity to tell me how the class is going
To: Rao Kambhampati <rao@asu.edu>


Folks

Now that 6 weeks are over, I thought it would be good to poll y'all
on what is working and what is not working.
Please feel free to send your comments.

You can send comments either via anonymous web mail
http://rakaposhi.eas.asu.edu/cgi-bin/mail?rao

or by bringing a printed sheet to the class on Wednesday. Note that
the webmail records your ip address--so
you may want to send it from some ip address that is generic.

Here are some things I am interested in finding out. Other comments
welcome too:

1. Are the lectures too fast/too slow/too high level/too low level etc?

2. Are you actually able to connect your readings to lectures?

3. Are you reading before or after the topics are discussed?

4. Should there be more assessment (homeworks/projects etc)?

5. How is the progress towards semester project coming (qualitatively
speaking--
you will have an opportunity for detailed answers anyway)

6. Are the classes interacive enough or should they encourage more discussion?
If the latter, what can we do other than stopping for questions?

7. Overall, how does it compare to other graduate level courses you took/taking
(positively or negatively)

regards
Rao

Agenda an d readings for tomorrow...

Tomorrow I plan to answer any questions over SAPA and Zeno planning
algorithms (I am hoping you spent time reading them again after last
class), and then shift to temporal constraint networks (which
I mentioned as part of Zeno).

For the first part, I\in addition to the SAPA paper and IJCAI 2007
paper readings that you are already in charge of for last class, you
can also look at the Zeno paper
http://rakaposhi.eas.asu.edu/cse574/zeno.pdf

For the second part (temporal networks) you can look at

http://rakaposhi.eas.asu.edu/cse574/tcn-meiri-dechter-aij.pdf

rao