Figure 1.1 The IST-05 Reference Model
(A preliminary and partial draft for discussion and development. Latest update 2001/05/06)
A "visualisation system" is a system for presenting interactively some part of a dataspace in such a way that a user with some purpose in mind can visualise the import of the data for that purpose.
This definition of a visualisation system contains several important words, each of which must affect any guidelines for evaluating the system. Most importantly, it is the user who does the visualising. A visualisation system is not a system for making a picture out of some attributes of the data. Indeed, its data presentation need not be wholly, or even partially, visual, provided that it allows the user to visualise --to make a mental picture of--how the data fits with the purpose.
Some of the important words and phrases in the definition:
To evaluate a visualisation sytem is inherently difficult. The very concept of "visualisation" is the antithesis of "logical analysis." One visualises what one does not analyse. Visualisation is synthetic, putting together things that analysis may pull apart. Visualisation and analysis work together to create "understanding", but they are very different things. To analyse a visualisation system is therefore likely to lead to conceptual difficulties.
One way complex systems are often evaluated is by looking at isolated elements of the system, using an underlying assumption that if the elements work well, then so will the system. Sometimes this may be a correct assumption, but it is not guaranteed to be correct. Alternately, systems are sometimes evaluated by how well they perform as an integrated whole in the situation for which they were designed. This is, of course, the ideal situation for evaluation. The problem is that it lacks a metric. It is impossible to say how well the system is performing, in relation to how well it might perform with some little change, and the complexity of an environment that requires the use of a visualisation system usually precludes a meaningful variation in the conditions under which the system is tested. For example, the best test of a visualisation system in support of battlefield command and control is whether the user of the system is more often on the winning side than would otehrwise be the case; but normally only one such test, if any, is performed, and by the time the results are in, it is too late to do anything about it.
Another problem with whole-system testing is that it is hard to isolate what may make a system effective to use. It is not always the system that appeal to the users that work the best. For example, a well designed system may be completely undermined by allowing a user to choose attractive display colours that make it impossible to distinguish critical differences that a less aesthetic colour scheme would clearly differentiate.
These approaches involve testing a system that has been constructed. But it is at least as important to try to evaluate the design of a system, so that it is likely to work well--even if not provably so--when constructed. The present guidelines address this issue--prospective evaluation of the system. They take a part-whole approach to the problem, looking both at the job the system is intended to perform, and at the nature and relationships of the elements of which it is composed.
|
|
Figure 1.1 The IST-05 Reference Model
|
In its Final Report, NATO RTO IST-13/TG-002 used a functional mode,l called (for historical reasons) "The IST-05 Reference Model," to describe the relationship between a visualisation system and its user. This relationship is seen as consisting of three main loops. A fourth loop is taken for granted, in which the user performs the task by interacting with the "Outer World," but this outer loop is not part of the visualisation system. The IST-05 Reference Model has the same form regardless of the kind of task or the kind of data in the dataspace. Nevertheless, it forms the basis for the guidelines being developed here
It is assumed that the data in the dataspace reflect something relating to the task of the user. Those data may have been entered explicitly by the user or some other person, or may have been acquired autonomously. They may vary rapidly, taking input from sensors monitoring some aspect of the outer world, or they may be essentially static within the time scale of the user's interaction, as, for example, a library catalogue usually would be. Whatever the nature of the data, the outer loop shows the user creating an understanding and acting to change the content or possibly even the structure of the dataspace.
The user does not act directly on the dataspace, nor does she understand it directly. The Reference Model shows that Understanding comes through visualising. However, it is recognized that visualising is only one means toward understanding, logical analysis being another. Logical analysis and visualising mutually support each other, the combination beign more powerful than either alone. However, the Reference Model and these guidelines are concerned with the visualisation process, and leave logical analysis aside, as a necessary, but unconsidered, part of what the user does to achieve understanding.
As an intermediary on the computer side, the Reference Model incorporates the concept of "Engines." Engines interact with the dataspace on the one hand, and with the input-output systems on the other. The engines select data from the dataspace, perform algorithmic operations on the selected data, and perhaps insert data into the dataspace or modify data that are already there. On the otehr side, the engines organize the selected and manipulated data for presentation, mapping them, for example, into locations, colours, and transparencies in a 3-D space, or into textual descriptions or compex auditory signals. The user communicates with the engines to affect the data selection, the data manipulation, and the preparation of the presentation.
The actual interaction between the computer and the user happens through physical devices, of which there are many. Output devices include display screens, holographic systems, immersive 3-D systems, headphones and loudspeakers, haptic devices that affect the user's tactile exploration of the data world, and so forth. Input devices include keyboards and speech recognizers, mice and wands, gloves and cameras (cameras can be used to examine the user's face for detecting eye movements or even indications of emotional responses).
Each of the three levels of loop, as well as the interactions by which the inner loops affect the next outer ones, are potential subjuects for guidelines.
Guidelines relate to the ability of a user to achieve the purpose of the visualisation. They are therefore necessarily related to how the element under consideration relates to that goal. Typically, if the user has some influence over the element, the over-riding question will be:
This question spawns two other basic questions:
Each of these questions in turn may spawn others that are particular to the element in question. For, say, a Web search engine, the questions might be asked like this:
|
Table 1.1 Basic Questions applied to a Web search
engine (the "element")
|
|
| What is the user trying to achieve? | Selection of documents that contain information useful to the task. |
| Can the user perceive whether the engine is tending toward achieving the goal? | Depends on whether the presentation shows enough of the nature of the selected documents for the user to make that judgment (leads to specific questions about numbers of hits as compared to numbers of useful hits, manner of presenting information about document content, and so forth) |
| Does the user have the power to influence the element? | Depends on whether the user can refine the search criteria, or needs to initiate wholly new searches based on perception of the initial results (leads to further questions about what the user can do to modify the selection criteria) |
Generally, a sub-question under "does the user have the power" will be "in principle could the element support the goal". For example, if the element is the dataspace itself, and what the user is trying to achieve is to visualise the distribution of ages in a community, a dataspace could not support that goal if it did not contain birthdates or data from which they could be derived.
Another sub-question often is "Are there external influences that may interfere with the user's ability to perceive or with the user's power to influence the engine's behaviour?" External, here, means external to the user's interactions with the specific element under consideration. Often, the user's concurrent interaction with another element is an interfering influence, and so the evaluation guidelines must consider the user's ability to deal with the behaviour of many different elements of the the visualisation system, either simultaneously or cuncurrently. Two further questions therefore must be added to the list:
If the element in question does not involve direct interaction with the user, the basic questions are analogous. "What is the element supposed to be doing?" "Does the element have access to the infoprmation required for it to do what it is supposed to do?" "Does the element have the resources to allow it to do what it should do?" And again we have to consider "Are there external influences that might affect the element's ability to do its job." In this case, the concern may be simple access to computing cycles, or to screen real-estate, or it may be more arcane such as issues of data-locking between data items that might be read and written by different asynchronously operating processes.
Finally, the system might be able to help the user to perceive the element's behaviour by supplying algorithmically defined Alerts. When a condition exists that confomrs to something the user might need to perceive, and the system is able to detect that fact, it might present an Alert to the user in some manner adapted to the human alerting systems. We can add, then, a sixth question, which is not in the direct evaluation loop, but which could be important to the effectiveness of a visualisation system:
 |
| Figure 1.2 A loop illustrating the points of influence of the five questions within the model. |
Five of these six questions can be diagrammed in the form of a loop of effects (Figure 1.2) This loop is very much like the so-called OODA (Observe, Orient, Decide, Act) loop of a military command and control system. The initiating construct in the command and control loop is the Mission. It is what the commander is supposed to try to achieve. In the loop of our five questions, it is "what the user is trying to achieve" whether that be a major mission imposed from outside or a minor sub-task such as "aligning a cursor with an on-screen object" in support of some higher-level task. No matter what the level of the achievement goal, the supporting questions remain the same.
To determine what action needs to be taken, the user must determine how the current state of affairs related to the goal state in respect of the computer-side element that the user's actions are intended to affect. The user must be able to perceive the state or the ongoing development of the element in question. The evaluator must therefore examine the means by which the user is afforded this perception. However, the "means" or mechanism is a question to be answered at a different level of analysis. Whether it be visual, auditory, pictorial or textual is of no import at this level. At the level of this loop, what needs to be answered is what information is required from the display mechanism.
The user's ability to perceieve the current state may be inhibited by outside influences, which can take a wide variety of forms. For example, in the real world outside the computer, there might be fog. In the interaction with the computer, there might be a clutter of irrelevant data on the screen, obscuring the interpretation of the relevant information. Or, the external influence might come from inside the user, in the form of a diversion of the user's attention toward another task. The evaluator must consider what possibilities there might be for such interference on either side of the human-computer interface.
Having determined how the current state deviates from what the user wants to achieve, the user can act so as to influence the computer-side element to do what is necessary to bring the user's goal closer to being achieved. The evluator must examine the means by which this action is effected, but as with the perception, the actual mechanism is a question at a different level of analysis. At the level of this loop, the evaluator must determine what influence the user needs to bring to bear on the element in question, and whether the mechanism is adequate to support that influence.
As with the perceptual side of the loop, external influences may affect the user's ability to act effectively. In a moving command vehicle, for example, it is hard to control a mouse running on a flat surface. At a higher level, navigation in a dataspace may be made more difficult if the system likes to shift the display focus to places where new information is available. Inside the user, actions that require the hands to do something may be inhibited by the requirements for action in other tasks.
When evaluating such possibilities, there is a natural tendency to confuse the levels at which these external influences act. If the external influence affects the mechanism of action or of perception, then it belongs as a lower level of analysis. For example, if the user is trying to navigate to a part of the display space with certain characteristics, an inhibiting influence might be a requirement in a different task to navigate to a different part of the dataspace. It would not be appropriate at that level to consider the bouncing of the command and control vehicle that make the use of a mouse difficult, though at a lower level this influence might be very important to the evaluation, since it could make navigation extremely difficult.
The IST-05 Reference Model shows three nested loops. The outer loop shows the user's interactions with the dataspace, the middle loop the user's interactions with the engines that manipulate the data, and the inner loop with the displays and inpute devices. In one sense, these are levels appropriate for analysis, but each of them can be refined further. For example, the Engines can be subdivided into components that address the dataspace, components that operate on the data relationships, and components that prepare data for display.
Considering the outer loop first, it is very hard to specify what the user might want to do, because this is the level at which every task is unique. However, the IST-013/TG-002 Final Report (The HAT Report) lists four kinds of perceptual function that are applicable across all tasks, and it might be profitable to consider these. The four kinds of function are:
The perceptual requirements for any task can be broken down into these characteristic types, and to the extent that this is done, something can be said about the outer loop of the IST-05 model.
At one end of the outer loop is the user's Understanding, which is supported by both visualisation and logical analysis; we are deliberately ignoring the logical analysis branch, and dealing only with visualisation. At the other end of the outer loop is the dataspace. In the HAT Report, six dimensions of description of data were proposed:
These descriptive dimensions can take on a restricted set of specific possible values, as shown in Table 1.1 (Table 3.1 in the HAT Report):
Table 1.1 Summary of Data Types |
|||
Acquisition |
Streamed | regular | |
| sporadic | |||
| Static | |||
Sources |
Single | ||
| Multiple | |||
Choice |
User-selected | ||
| Externally imposed | |||
Identification |
Located | ||
| Labelled | |||
Values |
Analogue | scalar | |
| vector | |||
|
Categoric (classic) |
symbolic | linguistic | |
| non-linguistic | |||
| non-symbolic | linguistic | ||
| non-linguistic | |||
| Categoric (fuzzy) | symbolic (non-linguistic) | ||
| non-symbolic (non-linguistic) | |||
Interrelations |
User-structured | ||
| Source-structured | |||
The attributes in Table 1.1 do not necessarily apply to all the data in a dataspace. Indeed, it is rare that they do. Typically, for example, a dataspace may contain archived (static) data, which is being continually augmented by input from external sources (streamed).
When considering the relationship between the user's perceptual tasks and the nature of the data, the most important aspect of the data probably in the Acquisition dimension. Only if new data can arrive on a time-scale commensurate with the user's execution of the main task will "Controlling/Monitoring" be involved at the outer loop level of analysis. But even in a static dataspace, provided it is large enough that the user cannot perceive it all at once, "Controlling/Monitoring" may be important at the "Engine" level while the user is "Exploring" or "Searching" a static dataspace at the outer loop level.
Whether a data set is "static" or "streamed" depends on the time scale of the user's task more than on any abstract measurement of the dataspace. It is streamed if the user is concerned with changes in the data as a part of the task. It is static if the data change too slowly to interfere with the proper performance of the task. If the data are changing significantly during the performance of the task, it is pointless for the user to explore them, since the results of the exploration may be out of date by the time they are required for a real-time task, and will need to be checked at the time they are to be used. Exploring therefore tends to be opposed to controlling/monitoring. The user may explore static parts of the dataspace while controlling/monitoring those parts that change, but will not explore and control/monitor simultaneously the same part of the dataspace.
Thus far, "Alerting" has not been mentioned. In principle, an alert is the result of an automated system (in the human) detecting some potentially significant event or pattern in a part of the dataspace not being currently controlled/monitored. The reason for the alert is that the human might find it profitable to shift from the current focus of control to some new focus. In our deep evolutionary history, it might have been a matter of life and death for a berry-picking ancestor to be made aware of the danger of a predator signalled by an unexpected noise or a flicker of colour seen through the bush. In the context of the present discussion, algorithms in the computer might detect potentially significant events, and generate display events that suit the human user's built-in alerting systems. It is not clear whether this is likely to occur at the outer loop level, but it must occur at the level of the engines.
To develop any guidelines, we must consider the six questions. To paraphrase, they are:
It is impossible to say in advance what the user's task may be, but we can divide they types of task into those that are basically control/monitoring, those that are basically searching for something, and those that are at heart exploration. It is not very reasonable to suppose that a task is basically alerting, because the notion of alerting presupposes that the person is primarily doing something quite different.
Quite often the user will be doing several simultaneous tasks with the database in support of some external task in the outer world. The questions must be asked in connection with each of these individual tasks, but they should not ordinarily be treated as one complex task when evaluating the system.
The first question here is "In principle, does the dataspace contain data that could allow the user to perceive what is necessary?" For example, if the user is trying to track changes in the stock market in real time, a database that is updated at the end of the trading day could not, in principle, allow the user to perceive the necessary changes in prices. But this question is not always so easily answered. The dataspazce may contain the data, but in such a way that complex methods are required to extract it. The methods might be as yet undiscovered, but that would not mean that the data, in principle, were inadequate to allow the user to perceive what was necessary. Indeed, it is often the very lack of available algorithms that suggest visualisation by the human might be the appropriate way to extract the information from the data.
Assuming that the data can, in principle, support the task, the next question is to list the various things that the user needs to perceive in order to perform the task, and to ask whether the system allows the user to perceive those things. Most of these questions become the basis of evaluation at the next level--the engine level. A question such as "Can the user see the data that would, allow the task to be performed" will later be restated at the engine level as the following tasks (among others):
In conjunction with the evaluation of what the user needs to perceive, the evaluator might well consider question 6: whether the possibility of deploying algorithmic resources to provide Alerts has been taken. Could the computer side of the system help to point the user toward those spects of the data that would best support the task?
Each of the four tasks listed are potential answers to question 1 of the basic five, and for each of the four tasks, the other five basic questions must be answered. But the other five basic questions also have to be answered at the top, outer-loop, level, an issue that we now address.
When dealing with nested structures such as those of the IST-05 Reference Model, it is often hard to keep straight the level of abstraction at which a question must be answered. At this point, we are dealing with actions that the user needs to perform on the database, and the answer to the question depends very much on the task. As pointed out in section 2.1.1, it is impossible to say much about the extraordinarily wide range of tasks for which visualisation may be helpful, but we can divide them into three classes--Controlling/Moinitoring. Searching, and Exploring. These classes imply different kinds of action.
These statements really imply questions the evaluator must ask in connnection with each task. How well does the system satisfy the stated requirements?
There are three classes of things that might be detrimental to the user's ability perceive:
Of these, the first two are issues that probably need to be dealt with at a lower level, since they deal with the machinery of constructing the display. At this level the issue needs to be noted, for resolution at the lower level, whether that lower level be the level of the engines or the level of the display devices.
The third possibility, however, is one that must be addressed at this level. Are there going to be other tasks that must be performed silutaneously with this one, and will their perceptual requirements interfere with the requirements of this task, either because of limitations in the human or because of limitations in the processing or display systems on the computer side of the loop. In otehr words, in the IST-05 Reference Model, teh question is "Are there any bottlenecks in the side of the outer loop that lead from the dataspace to the user's understanding?
The likely sources of difficulty impeding the user's ability to act appropriately parallel those that are likely to inhibit perception:
Again, the first two potential problems are noted at this level, and analysed at a lower level, whether that be the level of engine or the level of the input devices, whereas the third is a problem at this level. Do other parallel tasks require the user to be doing something that is physiologically or psychologically difficult to do in parallel with the actions need for the task under consideration?
This question may apply at the present level, inasmuch as the task may involve detecting when or where in the dataspace certain definable patterns occur, If those patterns can be algorithmically described, then there is an opportunity for providing alerts to signal their occurrence. Two problems arise. One is that the algorithic description of "interesting" patterns may be broad and imprecise enough to generate too many false alarms, and the second is that every alert potentially interferes with the perceptions required for other ongoing tasks. Alerts can hurt as well as help. The balance depends on the likelihood that the alert is worth pursuing, the degree of interference with the perceptions involved in ongoing tasks, and the ease of navigating to and back from the relevant aspect of the dataspace. The questions to ask, then include:
At the Engine level more can be specified, since many different kinds of task may be supported by similar Engine operations.
What may the user want to achieve using an Engine? Obvious possibilities include:
Very often, selection precedes or forms part of any other use of an Engine, so it may be useful to treat it separately.
Table 1.1 summarizes the data types identified by IST-013 in the HAT Report. These data types determine the range of possibilities for data selection. For example, selection might be of data satisfying a certain criterion of value. If the data are, say, analogue scalar, the user might want to see those with values between 3 and 5. More interestingly, if the data are analogue vector--as might be the case for text described by context vectors--the user might want to see those data whose value vectors correlate more strongly than 0.85 with the vector representing a query.
Equally, the user might want to select data by their identities. If the data are, say, located, the user might want to see those located within 5 km north and east of a reference point, or if labelled, those with labels that include some combination fo keywords such as "airport not military". Each of the six different kinds of data attribute offers possibilities for selection, and they can be combined to refine the selection possibilities further.
With respect at least of the selection capabilities of the Engines, we are therefore in a better position to pose the six basic questions than we are in respect of the outer loop. To repeat, the six questions at any level are:
We have implicitly answered the first of these questions by saying that we are dealing with data selection. However, it is worth noting that the nature of the selection task differs according to the data attributes used for selection. In particular, if the data are treated as located, that is to say that they are identified by their position within an N-dimensional space, then one aspect of selection becomes navigation through the dataspace. The user may want to move the viewpoint to a location in the dataspace near which interesting data are likely to be found. The issue then is how the user arranges for the system to create a display from such a felicitous viewpoint.
The user needs to perceive two different kinds of thing.
Firstly, the user needs to perceive the data selected in enough detail to determine whether they are likely to contribute to the performance of the primary task for which the visualisation is being performed. In other words, something about the displayed data must help the user either to perform the visualisation or to determine what difference in the selection would be likely to lead to such data. Either the data are the ones that are themselves useful or they are a way station in a Search procedure.
Secondly, the user needs to perceive how the Engine may be controlled so as to bring the displayed data more nearly to contribute to the primary task. There are many different possible ways an Engine might be controlled to change the data selection, depending the nature of the data according to the six kinds of descriptive attributes, and on which attributes of the data are of primary interest to the user. But no matter how the data are to be selected, if the user cannot perceive what must be done in order to effect or to alter the selection in desirable ways, the selection Engine will be of little use.
The evaluator must determine how readily the user can (a) determine the usefulness of the selected data, and (b) determine what needs to be done to control the selection Engine.
The actions the user must perform are those that control the selection Engine so as to improve the usefulness of the selected data for the primary task. It is not possible to specify what those actions might be. They might involve "flying" the viewpoint through an abstract 3-D representation of located data, they might involve alphanumeric subselection from a pre-selected data set, they might involve choosing isolated data from a display, or any of a myriad of other possibilities.
The evaluator must assess whether, having perceived what needs to be done, the user is given the power and tools to do what needs to be done. The evaluator might also enquire as to the cognitive complexity of the required actions, but the present guidelines cannot approach the methods that the evaluator might use for such an enquiry, because of the huge number of possibilities.
The usual suspects that may inhibit the user's ability to perceive something are: unsuitability of the display, and competition for the user's perceptual resources. Unsuitability of the display may come in many different guises, again hard to specify, but most often unsuitability involves unnecessary complexity, a mismatch between the display and the user's expectations or training, or emphasis of the unimportant over the important elements of the display. Competition usually means that the user's attention is distracted or that important aspects of the display are obscured or masked by something extraneous.
The evaluator should check the degree to which the displays are suitable for the user both to evaluate the usefulness of the selected data, and to perceive what needs to be done so that the Engine will produce more useful data.
The design of the system may simply not provide the user with the tools to control the selection appropriately. For example, one can imagine a geographic database system that does not allow the user to select those elements within a drawn outline that are of a class "residence building assessed at less than $300,000". But this may be the selection criterion that the user perceives to be necessary.
Failure of the system to provide the necessary tools is one extreme example, but more subtle are the effects of bottleneck--the user must use the same low-level input system for simultaneous but disparate actions at a higher level. Bottlenecks are not always easy to detect, since their effect is at a different level from their cause, and because their existence is transitory, depending on the exigencies of the moment.
In evaluating this issue, the evaluator should be careful to distinguish implementation at a low level from problems in principle at the level of the selection Engine.
It is not obvious what alerts are likely to be appropriate in a selection system. One possibility exists if a selection Engine operates algorithmically on the data in the dataspace, but the user's navigation of the dataspace is decoupled from the display of the selected data. In such a case, data that satisfied the selection criterion might be highlighted for the user to find during navigation. This is a variety of alert, even though it does not fit the pattern of notifying the user immediately an alerting condition is satisfied. Other than this, it is hard to see occasions for alerting during the selection process.
The evaluator should be alert that alerting might be appropriate for the system and primary task at hand, and check whether it has been effectively done if it is appropriate, but ordinarily this will be of little concern.
Even though there are innumerable possibilities for data manipulation, nevertheless the same six questions apply to evaluating the process. The task for the user is to ensure that the data manipulations create derived data that allow the visualisation to support the primary task effectively. Rather than detailing the other five questions the evaluator should assess, they can be summarized as follows. The evaluator should check (1) whether the user can see what is being done and how to alter what is being done, (2) whether the user can perform the actions required to affect what is being done, (3) and (4) what might be detrimental to the user's ability to perceive or to act, and (5) whether opportunities for alerting are being effectively used.
Note: There may be special difficulties for the evaluation when the task primarily involves the perception of relationships among the attributes of elements of the dataspace, since the space of relationships is inherently of higher dimensionality than that of the data. Whether this potential difficulty is real will have to be determined in actual attempts to apply these guidelines.
The only special consideration here would seem to be that if the Engine is to add data to the dataspace, it must find a place to put the new data. In other words, the scope or structure of the dataspace must be altered. If this is to be under the control of the user, rather than implicitly done by the engine for the user to discover in a later slection operation, then the user must be able to perceive those aspects of the dataspace and of the data that would permit the appropriate placement of the new data, and must be able to perform the actions that allow the desired placement.
To modify data in the dataspace, the user must be able to perceive the initial state of the data and the controls for the mechanism of changing them, as well as the data state after the change. Otherwise, the evaluator must consider the five questions (modifying the data being the answer to the question "what is the task"), in the same way as for the other Engine-related tasks.
The formal dataspace within which the user performs the primary task is not the only dataspace of concern. As illustrated above, the user must also be able to perceive the ways to influence the views onto that dataspace provided by the Engines and the actions that affect the Engines. The possibilities afforded by the interface form a dataspace of their own, which the user may visualise in order to use the system. Sometimes this visualisation is called "forming a mental model of the system."
In principle, visualising the interface can be analysed in the same way as visualising any element of the system. The same six questions apply as apply to the system as a whole and to any of its parts. The evaluator must determine what the task is, whether the user can perceive what is required to perform the task, and whether the user has the power and ability to act on the basis of that perception.
Particularly in the case of the interface, much of the perception may reside in the user's memory, earlier Exploration having provided the data for present control (i.e. the user has learned something of how to operate the interface). This fact has implications for the effectiveness of the intrerface. Two tendencies oppose one another. If the interface is structured in a uniform manner, so that similar actions based on similar perceptions cause similar effects under different circumstances, it is easy to learn that set of conditions and apply them under the different circumstances. However, if those results are not applicable in some particular curcumstance, then the generic learning may be detrimental.
Considering this issue as one of navigation within the dataspace of the interface, an anlogy might be between learning to find one's way in a grid-structured New World city as compared to an Old World city in which the street pattern depends largely on the footpaths used by long-ago farmers. The Old World city is potentially very confusing to newcomers, but much easier to navigate for those who grew up in it. The reason is the same in both cases: every corner looks different, which confuses to novice, but guides the resident. Unattended cues in the environment likewise may guide the expert into using a complex interface accurately, when the same cues may utterly confuse the novice user. The "resident" may be able to visualise the complex intrerface more easily than the simplified one, whereas the opposite is likely to be true for the novice.
Beyond the questions of whether the interface effectively enables the perceptions and the actions of the user, the evaluator should assess whether the designer has taken advantage of possibilities for alerting the user to events in the interface, and if so, whether the alerting mechanisms have been designed so as to detract from or to aid the main control task being analyzed.
An evaluator must consider six basic questions about any element of a visualisation system, or about the system considered as a whole. The manner in which these questions are addressed may differ when considering the outer "understanding the dataspace" loop of the IST-05 Reference Model, the Engine-level loop, or the input-output systems. In particular, though a question may be posed at one level, such as "can the user perceive what is necessary to perform the task defined for this element", the answer may lie at a lower level, such as "No, because the designer has used blue forr detail or text on a dark background". Such an answer would indicate that the system structure may be all right if a low-level problem that makes it unusable is fixed.