Merideth Naomi is now 15 months old No new pictures posted. Yet.

- Fritz. (Aug. 6th, 1998)

by Fritz Freiheit

Mar. 13th, 1995

EECS 592

(A background resource for the Guha & Lenat [2].)

"Commonsense" - I can't tell you what it is, but I know it when I see it.

Initially I would like to compare the Cyc project as an Artificial Intelligence version of two other seminal technological events: the first heavier than air flight by man accomplished by the Wright brothers shortly after the turn of the century, and the Manhattan Project as carried out by the United States during the Second World War. Neither of these events is a complete analogy, but each allows some interesting insights into the nature of the Cyc project. In addition, the very process of creating and exploring these analogies illuminates the very problem that the Cyc project was launched in an attempt to resolve. Unfortunately, it is impossible to carry these analogies to far, and in particular to give us any real indications as to the future success, or failure, of the Cyc project, as it has yet reach the end of its planned lifetime.

Like both the rush to accomplish the first heavier than air flight and that to build the first atomic bomb the AI community is striving to build the first example of machine intelligence that is comparable to that of our own. In both cases there are strong camps that were (or are) aligned with the engineering/pragmatic end of things, and those that feel that a firm theoretical understanding of the problems is absolutely necessary. But in both historical cases, the winning camp was the engineering "lets just build it, we'll deal with the problems as they come up" camps. The Cyc project is also firmly in the engineering camp. It is more true of the climate that pervaded the search for the first (heavier than air) aircraft than that of the Manhattan project that there were many competing ideas and experiments, frequently mutually exclusive, as is the case in the AI field today. It was felt by Guha, Lenat & Feigenbaum (see [5]) that the only way to make significant progress towards the goal was to bite the bullet and throw a large amount of resources at it, like the Manhattan project (this is one of those areas where the analogy with respect the first aircraft breaks down, it was possible to accomplish the initial goal using meager resources).

To return to the Wright brothers, it is interesting to note that all three men involved were essentially mechanics, and in fact the ultimate solution was accomplished by a machinist who had no training in how to build gasoline engines. This is not to say that Guha, Lenat and many of the others who have been working on the Cyc project do not have strong theoretical backgrounds, only to say the similarity is derived from the fact that participants in both cases did not have any reason to believe that their theories were/are sufficient, or even close. The gas engine for the Wright brothers plane, like the knowledge in the Cyc the project, was built from scratch and despite of the theorists saying that it would not work (the experts did not believed that a power to weight ratio sufficient to achieve flight was possible then, just as many of the experts today do not believe that sufficient power can be derived from captured knowledge to allow a significant subset of human intelligence to be simulated).

Which brings us to the question of what motivated Cyc project in the first place. Much like heavier than air flight as demonstrated by birds and insects, the example set by "natural intelligence" as demonstrated by animals, and more importantly, by humans, has been held to be a possible "man-made" goal. The immediate motivations for the Cyc project derive from the experience in the AI field in general during the 1970's and the early 1980's, and more specifically from failures to accomplish desired goals, such as natural language understanding and expert systems that demonstrated anything but the most narrow of expertise. These failures included excessively brittle expert systems (i.e. systems which performed well in narrow domains, but which get lost when encountering anything outside of those domains). Another important limitation has been in the area of sharable ontologies. The reason that expert systems can not be combined, in general, because they do not share the same semantics, even if they share a common syntax or implementation language.

So, what is the problem here? In general one must address the limits of architecture, implementation and representation through that of content, i.e. knowledge. Things become more clear when the theoretical foundations for the Cyc project are articulated by Lenat & Feigenbaum in [5]. They sum this up in the following principle and two hypothesis.

1. The Knowledge Principle: If an agent is to perform a task which has any significant degree of complexity, then that agent must bring to bear a large amount of knowledge about the world, not just about the specific domain of the task. Without knowledge, all that is left is search and reasoning, and this is not enough. (Besides, what are you going to search and reason over if you lack knowledge?) In terms of Wright Manhattan Project, this means that we have build the engine that will drive our knowledge base, and do it from scratch, if necessary.
2. The Breadth Hypothesis: For an agent to behave intelligently in unexpected situations, the agent must be able to use increasingly more general knowledge and/or to analogize to specific knowledge from diverse domains. When it comes to developing this breadth of knowledge, the Wright Manhattan Project analogy indicates that we have to be willing to spend the resources necessary.
3. AI as Empirical Inquiry Hypothesis: The premature descent into highly mathematical descriptions of a problem and/or the prevalence of toy problems obscures or even removes the details of reality that later turn out to be significant. It is therefore important to frame our problems experimentally, falsifiably, over large problems. In other words, we have to stop talking about building it, and build it.

To put these three principles/hypotheses into a concrete framework, the Cyc project was founded on the belief that if progress was to be made towards the goal of full machine intelligence, then someone must attempt to capture that ephemeral concept "commonsense". The fact that this was undertaken as an empirical process does not detract from the attempt. Far from it, it presents the opportunity to test the hypothesis that commonsense can be captured, and if so, will it actually produce the desired effects. Thus, the Cyc project is inherently, and intentionally, falsifiable.

The Knowledge Principle and the Breadth Hypothesis combine to create a notion of "commonsense knowledge", the knowledge necessary to understand an encyclopedia entry or a newspaper article. It can also be view as contextual knowledge and "consensus reality", the reality that we all assume is about us. This concept of commonsense knowledge forms the core of what the Cyc project is attempting to capture.

Research in AI has been going on since the inception electronic computers, so why should have taken this long for someone to attempt an undertaking like the Cyc project? One can certainly argue that there was insufficient computational resources to really attack it, but I think there are several other important reasons as well. One important reason is that it took until the beginning of the 1980's to even recognize that there was a problem (as described by Lenat & Feigenbaum in [5]) with the traditional methods of search and general representational systems. Once it was decided that there was problem, it is difficult to find a place to start. The theory of knowledge was vague, and even knowing how to grip on "commonsense" was a non-trivial task, as can be seen by the number of changes that Cyc went through (see below). Some researchers (such as Brian Smith [9]) feel that a much firmer theoretical foundation is required before you can even hope to start a project of this size. This is a trap, as how can you know in advance what sort of interaction between theory and implementation will occur? It is the process of implementation that frequently reveals problems with ontology, representation, and other aspects of theory. And finally, one cannot dismiss the power of the fear of being wrong or failing, as it is much easier to work in areas that are already charted, then to set out into the unknown.

Lenat & Feigenbaum (in [5]) present a grand vision for the future of AI research as initially embodied by the capture of commonsense knowledge. This vision is broken into three major stages (see Figure 1). These are:

1. Slow hand-coding of a large broad knowledge base. The initial capture of commonsense knowledge.
2. Acquisition of knowledge by natural language through reading and asking questions. Use this basic commonsense knowledge to acquire human level knowledge, both in breadth and depth.
3. Explore beyond the reaches of current human knowledge, as human researchers themselves do, by learning through discovery, carrying out research and development projects to expand its knowledge (base).

(Insert figure 1)

Figure 1. Rate of learning vs. amount of knowledge

This grand vision can be directly mapped to the developmental phases of Cyc in the following manner:

Pre-Phase 1. Ontological and representational problem solving. (Continues through the lifetime of the Cyc project.)

Phase 1. Knowledge Entry. This is the primary goal of the Cyc project.

Pre-Phase 2. Application implementation using Cyc - to help define and extend the necessary knowledge, ontology, inferencing, etc.

Phase 2. Crossover to natural language based learning. This is the planned termination of the Cyc project as it is expected to be in use supporting other applications.

Phase 3. (Ultimately) Discovery on its own.

In the process of implementing the grand vision the specific original goals of the Cyc project were:

• Capture commonsense knowledge.
• Create a common ontology.
• Construct a real (and useful) artifact.
• Empirical research.

These goals emphasize the engineering or pragmatic nature that has been emphasized so far. The actual implementation of Cyc essentially follows that of the declarative logicists approach with a heavy dose of pragmatism. For more specific details, refer to [3] and [7]. It is not that the implementation details are uninteresting, but, in part because of the shifting ground nature of many of the details pertaining to the implementation of Cyc, I prefer to stick with the higher level motivational view. Guha, Lenat and Feigenbaum emphasize that Cyc has evolved quickly and pragmatically. Initially the Cyc system was almost entirely a (vanilla) frame language but this has declined in use as the use of a constraint language (based on first order predicate calculus) has risen in use. This constraint language was developed to address the needs for: disjunction, negation, universal and existential quantification, etc. Changes were introduced into the representation language (CycL) and the ontology of Cyc only when it was felt to be absolutely necessary, such as when there was no way found to represent something, or when there was a need for more efficient inferencing. One of the important questions asked by McDermott [7] and Skuce [8] is how could the process of developing Cyc be carried out in an environment of shifting representation and ontology without forcing restarts. The answer to this question seem painfully obvious to me (as state in [4]) and is because of the declarative nature of Cyc's knowledge representation which allows the writing of new procedures over predicates that remain, essentially the same. Changes in the ontology are/were easier than changes in the representation language. The following indicate the major reasons why:

• Context (microtheory) boundaries serve as barriers to change (changes are frequently restricted to one or two ontologies)
• Most knowledge in the KB does not depend crucially on the exact structure of the top level (i.e. most general level).
• Knowledge enterers rarely have to change what they do, and even less so what they have already done, based on changes to high level concepts.

As an indication of this Guha and Lenat [4] state only 5 major changes to ontology/representation occurred (as 1993), the last big change was in 1990 when contexts/microtheories were introduced (the KB had over half a million assertions in it). The others being elimination probabilities, addition of default reasoning based on argumentation, and allowing predicates of arity greater than 2. Not surprisingly Guha and Lenat indicate that tools developed to support these modifications have also helped to make change manageable.

Guha and Lenat emphasize the increasing importance of the Epistemological Level (EL) and Heuristic Level (HL) mechanisms. The EL provides a common interface, so that, in general, the users of Cyc (humans or other applications) don't have to know about the underlying HL mechanisms that are being used to resolve queries and assertions, while the HL allows specialized techniques to be applied to specific domains, contexts, etc. within Cyc so that efficient (and timely) results can be obtained.

Extensions were made to FOPC to prevent "intolerably slow inference" speed. The extensions include meta-level assertions (reification, reflection of internal inferencing), modal operators (Believes, Desires), a context mechanism, limited quantification over predicates. In addition the FOPC use by Cyc evolved to have some n-th-order predicate calculus features.

Some additional changes:

• Addition of reasoning by argumentation (a central mechanism by 1992)
• Additions to the Heuristic Level to increase efficiency
• Microtheories (contexts)
• Speed - non-complete, heuristic level inferencing mechanisms (the speed tripled during the period 1990 through 1993)
• Addition of domain dependent inferencing mechanisms
• Alternative methods of knowledge encoding (versus frames): graphs, neural nets, graphical, etc.

During the time that [3][4][5][6] span many changes have occured. But by the time that [2] was published it would be fair that changes have settled down and are more representative of the maturing of Cyc. During this midterm period, as noted above, the importance of the Epistemological and Heuristic level bifurcation has grown. Another important insight that Cyc seems to bear out is that of the importance of local consistency over global consistency. Instead of trying to maintain some sort of global consistency Cyc maintains local consistency. This is indicated by the fact that despite the inconsitencies that exists in global sense within Cyc (various contexts/microtheories disagree), Cyc is still capable carrying out meaningful inferences. This was made possible by introduction of contexts/microtheories. Guha and Lenat (in [4]) show a strong parallel between the implementation of the ontologies and that of inferencing mechanisms.

Ontologies

Empirically-derived, increasing stable set of collections

Large Count (1993 - 8000 collections, over 5000 predicates, several tens of thousands of individuals)

Additions are easy (good tools, occurs in parallel, Cyc monitors updates to control the effects of each change)

Context/Microtheory is important (allows multiple ontologies, this is useful by allowing multiple views of a given domain, i.e. strictly correct vs. commonly useful, or multiple participant vantage points, local consistency vs. global consistency, new ontologies can be created by "budding" a new one, rather than by modifying "The One True Ontology")

Inferencing mechanism(s)

• Empirically-derived, increasingly stable schema
• Large Count ( in 1990 - 2 dozen, 1993- over 30 different ones)
• Additions are easy - (add a few recognition and articulation rules to the EL <=> HL translator)
• Context/Microtheory is important (constrains search space)

Additional Midterm implementation notes:

• Declarativeness is maintained (when non-declarative things is added to Cyc, a corresponding declarative description is also added)
• The ontology is axiomatized. There are over 1 million (as of mid-1992) independent assertions (the deduction of the axiom would be impossible based on the remaining axioms). The number of non-trivial one step inferences is on the order of a billion (as vs. the infinite number of trivial one step inferences "people have less than 3 heads", "people have less than 4 heads", etc.)
• Axioms are not buried in procedural code, such that any procedure is reflected by assertions about the nature of it. This is important in that it allows Cyc to reason about its own procedural code (as noted above in Declarativeness).
• Contexts (Microtheories) - Important, used to help deal with assumptions, but this introduced the problem of "lifting" information across contexts.
• "Unit Names" have no meaning to Cyc, but are only used by knowledge enterers during the early phases of knowledge base construction. As far as Cyc is concerned they could be replaced with "G0000001", "G0000002", etc.

Some of the criticisms were laid at the door of Cyc at its midterm?

• An experiment whose sole goal is to test hypotheses. No, constructing an artifact is also an important goal.
• It is a standard expert system, albeit a large one. No, capturing common sense knowledge, which complements expert knowledge, is the goal.
• How do you know that you have make the right ontological distinctions for all problems Cyc will ever face? We don't. That's why we are building it.

Use of the KB as a shared information pool, as opposed to Levesque and Brachman's view of the KB as a service that only inference engines have direct access to. (I.e. the reasoning mechanism is coupled to the knowledge that is represented in the KB.)

How can we measure the success of Cyc? This is an important aspect of the Cyc project that has been somewhat neglected. While Lenat and Feigenbaum admit to there being somewhat of a short fall in this area, and suggest some ways to measure it, there does not seem to be any real attempt to measure progress of Cyc except in a "head-count" sort of way. While it is true that we will have to wait until the time that Cyc actually "crossesover" into reading and asking questions as its primary mode of learning to judge a number of these successes measurements, it would seem that the Cyc project team could publish more incremental results along the way.

Success measurement methods:

• Increasing convergence of terms/vocabulary
• Increasing ease of addition, less often have to define new things, but can copy existing ones.
• Does it do the right thing?
• Can we represent (usefully) what needs to be represented?
• How often, and how easily, are extensions made to the system?
• Can it pass the Turing test?

In the end, it is important to keep in mind that the Cyc project is worth doing regardless of its success. Cyc provides an important locus for progress in AI research, something to compare techniques and strategies to. Another reason is that big projects bring out problems that small projects don't, we can thus expect to learn a great deal about the process of engineering large knowledge bases. It is also expected that we can derive some handle on the size a commonsense knowledge base, succeed or fail. Finally, in the assessment of Guha & Lenat in [3] success can be measure in the following way:

• Good - Cyc research (only) provides insights into the issues involved in building large commonsense knowledge bases.
• Better - Cyc forms the core of a knowledge base that can be used by the next generation of AI researchers to help make programs more than theoretical exercise.
• Best - Cyc's knowledge base serves as the foundation of the first full scale, or hard AI agent. Something that truly effective natural language understanding, expert systems, and machine learning can fully exploit.

There are number of things that I did not address due to time and space considerations. These include any significant information on specific inplementation details, such as explicit internal representation, or user interfaces. Instead, I prefered to emphasize the high level and theoretical considerations that surround and support the Cyc project. If this had been a "how to" rather than a "how come" paper, I would have spent more time on them.

Terms

BH - Breadth Hypothesis

CSK - Commonsense Knowledge

EH - Empirical Inquiry Hypothesis

EL - Epistemological Level

HL - Heuristic Level

KB - Knowledge Base

KP - Knowledge Principle

References

[1] C. Elkan and R. Greiner, Book Review: D.B. Lenat & R.V. Guha, Building Large Knowledge-Based Systems, Artificial Intelligence 61 (1993) 41-52

[2] R.V. Guha and D.B. Lenat, Enabling Agents to Work Together, Communications of the ACM, July 1994/Vol. 37, No. 7, 127-142.

[3] R.V. Guha and D.B. Lenat, Cyc: a midterm report, AI Magazine 11 (3) (1990) 32-59.

[4] R.V. Guha and D.B. Lenat, Response: Re: CycLing paper reviews, Artificial Intelligence 61 (1993) 149-174

[5] D.B. Lenat and E.A. Feigenbaum, On the thresholds of knowledge, Artificial Intelligence 47 (1990) 185-250

[6] D.B. Lenat and R.V. Guha, Building Large Knowledge-Based Systems (Addison-Wesley, Reading, MA, 1990)

[7] D. McDermott, Book Review: D.B. Lenat & R.V. Guha, Building Large Knowledge-Based Systems, Artificial Intelligence 61 (1993) 53-63

[8] D. Skuce, Book Review: D.B. Lenat & R.V. Guha, Building Large Knowledge-Based Systems, Artificial Intelligence 61 (1993) 81-94

[9] B.C. Smith, The owl and the electric encyclopedia, Artificial Intelligence 47 (1990) 251-288

Fritz's Home page

Contents

Feedback and comments

http://www-personal.umich.edu/~fritx/cyc.html