The role of the motivation in conditioning

One of the most important questions in the field of learning theory is what is it that makes a stimulus a reinforcement? What is the reason for animals and humans to be engaged in an activity such as lever pressing? In the majority of experiments food serves as the reinforcement for a hungry animal, or water for a thirsty animal. For those explorers of animal intelligence who went beyond a set of diversity limited by rats and pigeons, it rapidly turned out that for some species the greatest reward is to leave an animal alone, while for others (such as chimps) this may be, say, a possibility to look at children’s railway through the window. For example, for an ant in a maze a worthy tribute is a possibility to return smoothly to its laboratory nest. Isolated chicks will quickly learn a simple maze in order to rejoin their nest mates (Gilbert et al., 1989). Even such a favourite and usual subject for laboratory experiments as a rat may surprise experimenters by its preferences for a relevant award. It turned out that rats very quickly learn mazes if they are presented with an opportunity to kill a mouse in the final chamber (Myer, White, 1965).

Basing on Pavlov’s concepts of conditioned and unconditioned stimuli, Konorski (1967) suggested that unconditioned stimuli possess two different characteristics – specific and affective. Specific characteristics are those that make the US unique: the place where it is delivered, its duration, intensity, and so on. Affective characteristics, by contrast, are those that the US has in common with other stimuli and reflects its motivational quality. Thus food, water, and an opportunity to mate have the common appetitive characteristics that animals will actively search for them. Conversely, electric shock, illness, and loud noise possess the common aversive characteristic that animals will do their best to minimise their contact with them.

These characteristics of the US come to the concept of motivation that was suggested in early 30-th by one of the most famous Pavlov’s progenies, P.K. Anokhin. He included the motivation into a general statement of properties of learning. This notion is helpful for completing characteristics of stimuli that dictate whether or not they will function as reinforces. Anokhin first suggested a method of suddenly substitution of US in experiments on classical conditioning and thus revealed that animals possess something like “predisposed stimulation” in their central nervous system in a situation when they are waiting for a definite stimulus. This enabled him to develop a theory of functional systems, which was based on a concept of motivation (see Anokhin, 1961, 1968, 1974). The role of the motivation is to form a goal and to support the goal-directed forms of behaviour. Physiologists consider the motivation as an active drive, which simulates nervous processes during searching for such a decision that is adequate to animal needs in given environmental conditions. The motivation is closely correlated with the notion of dominanta, a term that was later introduced by another Russian physiologist, A.A.Uchtomsky (1950). The dominanta means that animal resources are mobilised to reach a given goal. In particular, the nervous resources are correspondingly mobilised, so the animal attention is aimed at the goal during purposeful behaviour.

As Anokhin supposed, that behavioural program starts first which answers purposes of an organism solving its vital problem. If the reflex is conditioned on the base of foraging behaviour it should be supported by a strong motivation for satisfying hunger. A hungry dog always displays a complex of foraging behaviour nearby a food source. But if an arbitrary hungry dog is thrown into deep water, it would rather manifest another pattern, just trying to reach land instead of eating even an appetising bit of food that is floating near the dog’s mouth.

It is important to note that, unlike in Pavlov’s theory, intelligence was included into Anokhin’s theory of functional systems. The functional system was introduced to explain the animal purposeful behaviour and that included “intelligence resources” which should be mobilised for seeking the priority goal, recognition of a situation and “planning” of actions.

Several authors later have developed concepts very close to Anokhin’s theory of functional systems. They have proposed that the determinants of conditioned behaviour are organised into functional systems that are concerned with such activities as feeding, mating, defence, and parenting (Davey, 1989; Timberlake, 1994). These systems are activated by the appropriate stimuli, and serve to coordinate patterns of behaviour that are both innate and learned. Each system is assumed to control a wide range of actions, the selection of which is determined by the stimuli that are present.

In 1930s, Hull and Spence introduced motivation as an intervening variable in the form of homeostasis, the tendency to maintain equilibrium by adjusting physiological responses (see: Hull, 1932, 1952; Spence, 1936, 1960; see also: Pearce, 2000 for detailed analysis). An imbalance creates needs, which in turn create drives. A drive, by Hull, is a single central motivational state activated by a certain need. Actions can be seen as attempts to reduce these drives by meeting the associated needs. This is the drive-reduction theory: the association of stimulus and response in classical and operant conditioning only results in learning if accompanied by drive reduction.

Premack (1959, 1962, 1965) suggested a solution to the problem of deciding whether a stimulus could be a reinforce. His idea was also based on difference in preferences of different kinds of activities. The Premack Principle, often called " grandma's rule, " states that a high frequency activity can be used to reinforce low frequency behaviour. Access to the preferred activity is contingent on completing the low-frequency behaviour. Premack proposed that reinforces were not stimuli, but opportunities to engage in behaviour. Thus the activity of eating, not the stimulus of food, should be regarded as the reinforcement when an animal has been trained to press a lever for food (see Pearce, 2000).

In order to determine if one activity will serve as reinforcement for another activity, Premack proposed that the animal should be allowed to engage itself freely in both activities. For example, a rat might be placed into a chamber containing a lever and a pile of food pellets. If it shows a greater wiliness to eat the food than to press the lever, then we can conclude that the opportunity to eat will reinforce lever pressing, but the opportunity to lever press will not reinforce eating.

This seems to be completely naturalistic and trivial. To demonstrate a relative property of reinforcements, Premack conducted an experiment with rats placed into a running wheel and demonstrated that the opportunity to run could serve as a reinforcement for drinking in rats that were not thirsty. In this experiment, animals were placed into a running wheel for 15 minutes a day. When the rats were not thirsty, they preferred to run rather than to drink. For the test phase of the experiment the wheel was locked and the rats had to lick the drinking tube in order to unlock the wheel and so to gain the opportunity to run for 5 seconds. Thus the animals had to drink in order to earn the opportunity to run.

This and other experiments (see, for example, Allison and Timberlake, 1974) led researchers to a concept of relative value of reinforces in dependence of environmental circumstances and internal state of an animal which forms motivation. We have to adopt a point that there are no absolute reinforces and in each experiment their properties should be considered directly.

6.3.2. Common properties and rules of associative learning

Although in previous sections of this chapter we emphasised difference between displays of Pavlovian and Skinnerian conditioning, it is still not completely clear whether mechanisms of these forms of learning differ in principle. Many experiments have been effectively performed with the use of mixed technique when properties of classical conditioning are studied on subjects being placed into operant chambers. For example, in experiments by Rashotte et al. (1977) second-order conditioning was studied on pigeons which first received auto shaping and had to peck keys being illuminated by different colours. Even in pure Skinnerian experiments the distinct evidences were obtained that pigeons form conditioned reactions for a key they peck, i.e. they associate the key with food or drink, depending on experimental condition (Moore, 1973). When pecking for food, the pigeon’s eyes are closed and the beak open (typical for feeding); when pecking for water, the pigeon's eyes are open and its beak nearly closed (a typical drinking motion), so their behaviour seems to follow the reward given. When students having seen video records of these experiments were asked what sort of reward each pigeon was waiting for- water or food - they never made a mistake in their answers. The same, although not so impressive, situations have been observed with rats and monkeys. It seems that operant and classical conditioning can not be considered principally different forms of learning.

Let us consider general principles common to all forms of associative learning. Some of them serve as elements of useful experimental techniques which help experimenters to discover more and more facts concerning animal intelligence.

Timing of the stimulation. We have already seen when considered concrete examples that both for classical and instrumental conditioning the order and timing of stimuli are important. Only a few seconds delays between a rat pressing a bar in operant chamber and the delivery of food greatly impedes the rat’s ability to learn. Such phenomena have presented thorny problems for animal trainers. In many instances the reward could not be given fast enough. As we have seen above, a method of secondary reinforcement was developed that we considered in details above.

Classical conditioning is based on the temporal continuity of presentation of the US and UCS. Pavlov also found that learning becomes poorer if the time gap between the presentation of the NS and UCS increases beyond the optimal half a second. This format of presentation is referred to as delayed conditioning. Basing on this experimental technique many important results have been obtained concerning short-term retention in organisms which made processes of memorisation and forgetting more clear.

An early demonstration of animal memory in the laboratory is provided by Hunter (1914) in his experiments with racoons, rats, dogs and children. The main principle of Hunter’s experiments is that UCS is absent just in that moment when a subject perform a reaction. A stimulus is substituted by “something else” which was named “ an idea ” by Hunter and this may also be named “reminiscence about a stimulus”.

In those experiments an object, say, a racoon, being retained in an observational chamber, was allowed to observe three exits. A light above one exit was then briefly illuminated and some time later the animal was released. If it chose the exit that had been indicated by the light it received reward. By this time the light is not illuminated, so the subject reacts for its “reminiscence about the light”. The time of the delay is important in these experiments. Hunter found that with sufficient training the subjects were able to tolerate a delay of as much as 25 seconds between the offset of the light and their release. This technique was called by Hunter the “ delayed reaction test “and later transformed by Harlow with co-authors (Harlow et al., 1932) and applied to different species of primates, from lemur to orang-utan. These experiments demonstrated different organisms as being able to retain information about past events and raises many interesting questions about how much information can be retained and for how long. These questions will be considered in Chapter 10.

Stimulus discrimination. Pavlov discovered that stimulus discrimination was learned in very much the same way as the acquisition of an association between things. In many experiments animals demonstrated their ability to detect the difference between those stimuli that are connected with a reward and those that are not. For example, if a dog is presented with the bells of different decibels in a number of occasions but only receives food when the bell of 5 decibels is presented, it only exhibits the CR (salivation reflex) with the bell of 5 decibels. Therefore the dog only responds to the stimulus that will result in reinforcement.

Stimulus discrimination learning appears to occur in both classical and instrumental conditioning in a very similar way. If only a very specific response is reinforced eventually the animal will only exhibit this specific response.

This experimental technique has been immediately appreciated as an excellent and practically universal tool for determining the sensory abilities of many animals as well as their cognitive and communicative skills. For example, honeybees can be conditioned to seek food on a piece of blue cardboard. By offering other colours to a blue-conditioned bee, Karl von Frisch as early as in twentieth found that honeybees see four distinct colours: yellow-green, blue-green, blue-violet, and ultraviolet. Even earlier, in tenth, Ladygina-Koths used the same method to reveal colour vision in birds.

Dramatic consequences for animals, which face fatal difficulties in stimulus discrimination during the process of investigation of their sensorial capability, were discovered first in Pavlov’s laboratory. For example, experimenters flash a circle on a screen and follow it with food. Then they flash an ellipse on the screen, but leave the bowl empty. The dog quickly learns to distinguish between the two shapes. Then, they flash a circle, followed by an ellipse that is a few degrees closer to the circle. The dog still manages to detect the difference. After that, though, the experimenters keep changing the ellipse, so that it gets closer and closer to resembling a circle. When they reach a certain point, the dog starts whimpering and wriggling, and it bites through the tubes on the collection device. With the use of such experiments it was possible to bring the animal to a serious neurosis (Fig. II-2).

As it was already noted, capacity of many organisms to give a definite answer to a question about differences between stimuli has been used up to now in many experimental schemes applying for studying animals’ cognitive abilities.

One of the earliest studies of animal cognition based on discrimination learning belongs to Kö hler (1918) who applied the transposition test to chickens. Birds were trained with two cards, one of which was darker than the other. Pecks at the dark card (S+) but not at the light card (S-), were rewarded with food. Once the discrimination had been mastered, a transposition test was given in which the subjects had to choose between the original S+ and even more shadowed card. If the original discrimination was solved on the basis of relational information, then Kohler reasoned the new card will be chosen in preference to S+ on the test trials. This prediction was confirmed, even though it meant that subjects rejected just the card they had originally been trained to select.

Kö hler interpreted these results in terms of Gestalt principles which have been considered in Chapter 4 and will be again analysed in Chapter 17. Here we only mention that the Gestalt explanation says that during training the organism learns a principle related to the two stimuli. The principle might be “the darker shade of grey is better”. Chicken thus had followed the rule “choose the darkest card”. When presented with other stimulus, the animal “transposes” the solution from the original problem to the new problem and chooses the lighter grey even though the medium grey was associated with food originally.

Although a number of more recent theorists have preferred to interpret Kö hler’s findings in other and maybe more parsimonious ways as we will see in Chapter 17, it is no doubt that discrimination learning underlies such cognitive abilities as abstraction, categorisation and rule extraction. These domains of animal intelligence will be considered in Part VII. Here we only briefly describe one more class of experiments based on stimulus discrimination and thus add learning- set to the list of basic classes of learning nominated in this section.

Experiments for studying learning set formation were elaborated by Harlow (1949) and involve a succession of discriminations with different stimuli, and the focus of interest is whether there is an improvement in the rate at which each discrimination task is solved (see also: Harlow and Bromer, 1938). Harlow suggested Wisconsin Test Apparatus (WGTA) which is still actually used for a battery of tests for studying cognitive skills in primates including humans but a general principle of this test is applicable for many species. In the experiment a subject is presented with two containers of different shapes under one of which a food item is hidden. The subject, say, a monkey, can lift either or both, but is only rewarded in one case. The subject is then successfully presented with several more pairs of containers with the same shapes as the first pair, and with the food always under a box of a certain shape. After this set of experiments, a new set starts in which the subject is shown another pair of containers with different shapes from those used in the first set. In several more trials, the food is again always under a box of particular shape. After several such experiences, the monkey begins to make a steady improvement in the percent of correct choices in the successive trials until, finally, once the animal learns which box the food is under, it chooses the right box every time. In effect, the monkey has learned to learn.

The method of learning set formation is based on more general methodological approach called matching-to-sample (MTS), which in turn came from the concept of stimulus discrimination. This method came from mental diagnostic elaborated for human children (Baldwin et al., 1898; Binet, 1905). Ladygina Koths (1923) was first to apply this method for studying perception and learning abilities in experiments with her famous young chimpanzee Ioni. The chimpanzee was presented with a sample stimulus such as a simple geometric figure, and, a short while after the chimpanzee was taken away, he was presented with two figures, one of which was the same as the sample. In other case the sample stimulus could be of a particular colour, and from two comparison stimuli one will be the same colour as the sample. In order to gain reward Ioni had to choose the stimulus that matched the sample (Fig. II-3). This method has become very popular as a very effective mental test for comparative studying of animal intelligence. Many species including not only mammals but also birds and even insects seem capable of solving discriminations on the basic of relational information (Fig. II-4). We will consider the concrete results in Chapter 14.

Stimulus generalisation. As an element of animals’ ability to judge about things, their aptitude for generalisation is of the same importance as for discrimination. Stimulus generalisation underlines processes of integration of information and determines the procedure of discrimination learning.

In Pavlov’s experiments a dog trained to salivate (CR) in response to 1000 Hertz tone (CS) also matched to similar tones, although salivation was not so intensive. The dog had “generalised” its responses for all stimuli that were similar to an original stimulus. It turned out that the more similar the other stimuli were to the original CS, the less similar the stimuli were to the original CS, the weaker the CR. This is called a generalisation gradient.

Generalisation appeared to be a characteristic of both classical and operant conditioning. An animal exhibits a very similar response to stimuli that resemble the original stimulus but not identical to it. If a rat is reinforced whenever it presses the red lever it will also attempt to press a number of coloured levers in a Skinner box. Furthermore, the more similar the stimulus is to the original the greater the frequency of responses the animal will emit. Thus if the original lever was deep red then the animal will also press levers of varying shades of red. However, it will press a lever that is nearest to the original shade of red more times, per minute, than a lever that is least like the original shade of red.

The use of the principle of stimulus generalisation allowed explaining some phenomena of associative learning. Thus, Spence (1936), by referring to the effects of stimulus generalisation, was able to explain transposition study by Kö hler in a more efficient way. He argued that when animals are presented with a discrimination task between two pairs of stimuli from the same dimension (say, brightness), there will be a measure of generalisation between them. As a consequence, the excitatory tendency to approach S+ (dark) will also be elicited by S- (bright), but to a weaker extend; and the inhibitory tendency to avoid S- will be aroused, albeit slightly, by S+. The strength of approach to either stimulus will then be determined by the interaction between these sources of generalisation. The surprising prediction of Spence’s theory is that if the test stimuli are far away from the training stimuli, transposition will break down. The subject will choose the lighter of two stimuli. Gestalt theory predicts that the subject will continue to choose the darker of stimuli. Experimental data confirmed Spence’s theory. Transposition breaks down when the test stimuli differ greatly. Despite this success of Spence’s theory, there are a number of problems that it is unable to overcome. This shortcoming is partly connected with the rule that is used to determine to which extend the associative properties of a stimulus will change on any trial. It turned out that these change took place not independently of the properties of the other stimuli that were presented (Rescorla and Wagner, 1972).

One could find detailed analysis of many theories of discrimination learning in Pearce’s “Animal Learning and Cognition” (2000). Despite their difference, they share a common feature. They assume that when animals are presented with a set of stimuli they learn about each one separately. According to configuration theory (Pearce, 1987, 1994) if a compound stimulus is presented for conditioning, or discrimination, then a configural representation of the entire pattern of stimulation will be formed.

Briefly speaking, the configuration theory emphasises the importance of generalisation between different patterns of stimuli. This theory assumes that when two or more stimuli are presented together for conditioning, only a single association will develop. This association is between a unitary, configural representation of all the stimuli and the US. By making the additional assumption that generalisation will occur between configurations, the configuration theory can explain most of the findings from the discrimination studies basing on stimulus generalisation.

The ability of organisms for generalisation causes dramatic drifts in nature. For example, many predators quickly learn to withdraw when meeting with brightly coloured and venomous insects. Generalising key features of an image they also avoid similar but harmless insects. Well known phenomenon of mimicry is based on evolutionary changes of species towards similarity with dangerous prototypes.

Repetition, extinction, recovery and attention. An important characteristic of associative learning is that it improves with repetition. Thus, a rat learning a maze reduces its number of errors with each trial until it runs directly to the food box with no wrong turns at all. After this its performance cannot be improved upon. However, if the rat is continually permitted to run the maze after it consistently achieves perfect scores, it remembers the route longer.

In Pavlov’s experiments salivation production increased slowly with every pairing of CS with UCS until it achieved the same level as when only UCS was presented alone. For example, a dog produced the same amount of saliva when hearing the bell ring as if it would have been presented with meat itself. As it has been already told, the performance of the reaction can not be improved, however, the longer the stimuli will be presented together after the top level of reaction had been achieved (this is called over-training), the more solid it becomes regarding to extinction. In relation to classical conditioning, this term refers to the disappearance of a given response when the conditioned stimulus (CS) is repeatedly presented without the unconditioned stimulus. In relation to operant conditioning it refers to the elimination of a response by withholding all reinforcements of the response (Cartwright, 2002).

In Pavlov’s experiments quantitative characteristics of extinction had been obtained. If a CS (say, a bell ring) was presented on a number of occasions in the absence of UCS (food), the salivation production began to lessen on each presentation of the CS without UCS until it simply disappeared. Hence extinction is not a sudden process, but occurs slowly so that the CR slowly becomes progressively weaker until it no longer occurs.

It is interesting that the extinction may not be permanent. Pavlov found that carrying out an extinction procedure did not lead to total loss of the learned association. It turned out that even after a response has been completely extinguished in a series of trials, if the animal is allowed to rest for a few hours and is then given the conditioned stimulus again, the conditioned response may appear through what it called spontaneous recovery. Such recovery can be elicited several times before the extinction become permanent. To extinguish a CR fully the CS should be presented on a number of occasions. Nevertheless, research has shown that even when the CR has been completely extinguished, compared to naive animals, previously conditioned animals that have had the CR completely extinguished will re-learn the response much faster than naive animals learn it. Therefore it may seem that what has been learnt once is never entirely forgotten.

Another way to call extinguished reactions into being is the use of a new stimulus together with a known CS. For example, extinguished salivation as a reaction of a dog for a tone may be restored if blinker light will be switched on together with the tone. Similar data were obtained in operant conditioning. Pavlov called this process disinhibition and considered extinction a new process of conditioning that inhibits the first conditioning reflex. Thus neutral stimuli, which act at early stages of learning together with conditioning stimuli often inhibit learning and decrease effectiveness of a process. When a new processes, namely extinction, starts new stimuli inhibit it and thus call previously extinguished reactions into being. Pavlov’s logic of explanation is quite natural (Pavlov, 1927). He suggested that when facing a novel stimulus an animal immediately activates the orienting response, OR (or as he referred to it, an investigatory reflex) and turn its perceptive organs to a new source of information. This just suppresses development of conditioned reaction. This process is called external inhibition. If the novel stimulus appeared during a process of extinction, then that extinguished CR is increasing. This is called Pavlovian disinhibition. It is important to note that the disinhibition is not caused by competition of two CRs but excitation of central nervous system.

Many data confirm Pavlov’s hypothesis that extinction is a process of learning. During the process of extinction an organism learns that CS is no more followed by UR. Now CS is connected with the absence of UR, hence CR is extinguished. It is important that in experiments animals meet the absence of US in the combination with CS under circumstances when that CS has been rewarded for many times. This is an unexpected event and being repeated it will form the new CR, namely, “do not pay attention to this stimulus any more”. In real life an organism meets lots of stimuli which are not connected with any reinforcement and ignore them. Only meeting the absence of the reward suddenly, the individual learns to connect certain stimuli with the absence of reinforcement (Mackintosh, 1976).

It does not mean that it is necessary to reinforce carefully every correct movement of a family dog when teaching it to follow commands. As it was explained before, very rare awards will do in order to keep animal’s attention active.

In principle, extinction is closely connected with attention. Pavlovian investigatory reflex (orienting response, OR) mentioned above is a consequence of the animal attending to the stimulus and the vigour of this response may well provide an indication of the attention a stimulus receives.

An experiment by Kaye and Pearce (1984) shows how the simple experience of being presented repeatedly with a stimulus might influence this measure of attention. This experiment was based on the Skinner’s technique in combination with Pavlov’s method. Two groups of rats were placed into a conditioning chamber containing a light bulb and a food dispenser. For the first 12 sessions nothing happened for Group Novel, whereas for Group Familiar the bulb was illuminated for 10 seconds at a time at intervals in each session. Both groups were then given a single pre-test session in which the light was occasionally illuminated for 10 seconds. A typical OR to the light was performed at the outset of training by Group Familiar. But with repeated exposure to the light the frequency with which this response occurred declined progressively across 12 sessions. Such a decline in the responsiveness to a stimulus as a result of its repeated presentations is referred to as habituation, a phenomenon discussed in 6.1. The strength of the OR for both groups was considerably more vigorous in Group Novel than in Group Familiar, which suggests that the groups differed in the amount of attention they paid to the light. All subjects were then conditioned with the light serving as a signal for food. There was no difference in the strength of the CR of magazine activity during the light for both groups on the pre-test session. As predicted, on test sessions conditioning was more rapid in Group Novel than in Group Familiar, for which extended exposure to the light reduced its conditionability. This effect is known as latent inhibition and supports the claim that Group Familiar paid rather little attention to the light as a result of the pre-exposure stage.

Lubow (1973) reviews experiments showing latent inhibition in goats, dogs, sheep, rats and rabbits, but beyond these studies of latent inhibition are rare even in mammals. There have been several attempts to demonstrate this phenomenon in pigeons, but these have led to conflict findings. No success has been achieved in attempts to show latent inhibition in honey-bees (Bitterman et al., 1983) or goldfish (Shishimi, 1985). Pearce (2000) suggests that if future research should confirm that changes in attention, as indexed by changes in conditionability, are unique to mammals, then this will be an important discovery. Obviously, the opposite claim is also true: this would be a challenge to find these characteristics of attention in other groups of animals but mammals.

It is of great importance to know details of attentional process in human. This is a dream of agents belonging to many trades - from teachers and artists to advertising accounts and presidential contenders - to catch people’s attention and to hold it. It is clear that the attentional processes governed by the same rules in many mammalian species including humans. This concerns generation of reactions to novel stimuli, selective attention paid for different stimuli, processes of inhibition and recovery which may lead to revival of attention and interest and so on.

Let us consider a simple funny example. Nowadays pairs of lovers can present each others with velvet flowers provided with a button as well as an audiocassette fitted. When press a button, a happy lover may observe a flower bowed and listen magic words “I love you” articulated by the partner’s voice. After may be a hundred pressings of the button extinction will come but an interval should make attention being restored. Isn’t it a good chance to measure degree of love in values of attention? I suggest that many quantitative characteristics could be measured here such as number of pressures until the first display of inhibition, the time duration which is needed for returning interest, the rate of increase of this time from one session to another and so on.

Unfortunately none of present theories of attention is able to explain all the relevant experimental findings, although the development of these theories has led to the discovery of a wide range of experimental findings that now show the importance of attention in animal and human learning and cognition.

7. LEARNING CLASSES BEYOND “SIMPLE” ASSOCIATIVE LEARNING

A title of this chapter looks paradoxically because, as a matter of fact, the rest of this book is fully devoted to forms of animal learning beyond “simple” associative learning. The aim of this chapter is to give readers an estimate of application of rules of associative learning and briefly review learning classes which will be considered further in this book, just to complete a list of learning classes given in the beginning of this Part.

Indeed, when analysing rules of associative learning, learning theorists consider some notable exceptions to the main principle of association formation. The fact is that on some occasions and in some species conditioning occurs in “wrong” time and with different probabilities for different stimuli.

Before considering some exceptions to the rules, let us first concentrate here the main rules of “simple” associative learning.

1. Making associations is a matter of connecting events that occur together in time, and it is true that usually an increase in the interval between CS and US presentation slows the formation of associations.

2. The speed or strength of learning increases with:

The intensity of the CS

The size of the reinforce or US

3. When reinforce is withheld, the learned response declines in frequency or intensity (this is called extinction).

4. All pairs of events can be associated with equal ease e.g. it should be just as easy for a rat to associate the arrival of food with a light or a buzzer as with any other CS. This is called equipotentiality.

The first rule is now known to be a more general phenomenon i.e. time is not necessarily the most important factor for learning. Even events far apart in time can be associated if there is a highly predictive link between them. The best example of this is in food aversion learning. The first experimental results in this field were obtained by Garcia and colleagues in their attempts to learn why are rats so hard to poison (Garcia and Koelling, 1966; Garcia et al., 1972). In those experiments US = feeling ill, CS = novel flavour of food. The learnt response is avoidance of the novel food. The interval between tasting the food for the first time and feeling ill may be several hours for the rats. In the wild it is impossible to tell when the animal starts to feel ill. But food aversion learning can be studied experimentally. Illness can be induced at a set time after the animal has eaten a harmless but novel-flavoured food e.g. by injecting lithium chloride. In rats, food aversion learning can take place with delays of up to 12 hours between consumption and illness onset. It was suggested that the US-CS interval was not as long as 12h due to lingering flavour of the novel food. But even if other (familiar) foods are eaten in the intervening period, aversion to the novel food is still demonstrated. Obviously such learning ability has a high biological importance. It indicates that it is the predictability between two events that is important in establishing learning. Predictability between US and CS is more important than the time interval between CS and US, although in many cases a short time interval will be strongly correlated with a high degree of predictability.

Other behavioural routines also need to be taken into account here. For example, rats show a cautious behaviour to novel foods. Usually, they will taste only nibble at any new food the first time it is encountered (Barnett, 1970). Then, if nothing happens, they may eat a bit more the next time, until, finally, it becomes a normal part of their diet. This coupled with the time lag on associations made between US and CS is why many rat poisons take much greater than an hour to have any effect.

Garcia and colleagues (Garcia et al., 1977) have found once more effective application of the obtained results concerning slowed food aversion: they taught coyotes and wolves to co-exist peacefully with sheep. Predators were made ill by feeding them chopped mutton that was wrapped in raw sheep and laced with lithium chloride. A dramatic effect was observed when the animals were allowed to approach live sheep. Rather than attacking them as they normally do, the wolves after their characteristic flank attack immediately released their prey.

Equipotentiality (the fourth rule listed above) is also problematic as a universal law of learning. It has been found in many studies that animals do not associate all events with equal ease (Mackintosh, 1974). Chickens will easily learn to peck a key to obtain food but take longer to learn to peck a key to obtain access to wood-shavings for dust bathing. They seem to find it easier to associate a foot-related response with obtaining wood-shavings. For rats only five trials are sufficient to run from one section of a box to another in order to avoid electric shock but hundreds combinations are required on those occasions when animals should press a lever with the same purpose. For pigeons on the same occasion it is much more difficult to learn to peck a key than to press a bar.

This is caused by the difference between displays of animal’s innate reactions of avoidance in dependence of a concrete situation. Thus, this is nearly impossible to shape avoidance in those experiments in which animals, say, rats should draw near a source of danger, for example to press a lever situated just under a bulb alighted for signalling about electric shock. It is easier to shape this association when the bulb is situated far apart from the lever to be pressed (Biederman, 1972).

Returning to the set of examples concerning food aversion, rats associate a novel flavour with subsequent illness much more readily than an auditory-visual compound (Garcia and Koelling, 1966). Pigeons easily learned that a red light signals food, but they learned with difficulty when the same stimulus was used to signal shock. Conversely, conditioning with a tone progresses more readily when it signals aversive rather than appetitive US (Shapiro et al., 1980).

The term selective association refers to this general finding that some CS-US relationships can be learned about more readily than others (Mackintosh, 1974). In the case of Garcia and Koelling study cited above, taste is more likely to provide information than sound about whether or not a certain food is poisonous.

Selective associations may, at least in some species, be evident from birth. One of the best examples is behaviour of newly hatched chicks of precocial birds. Chicks have innate tendency to peck at small conspicuous objects such as coloured beads. When a new object is edible or positively rewarding, chicks will show enhanced pecking. However, when the object tastes bitter, chicks will subsequently learn to avoid similar beads even after a single experience. In this manner, chicks quickly make up a directory about edible, neutral, and aversive objects encountering within days after hatching. This is used as the basis for a variety of experimental tasks developed for studying memory formation in chicks (see Chapter 10 for details).

In experiments of Gemberling and Domjan (1982) rats were conditioned when they were only 24 hours old with either illness, induced by an injection of lithium chloride, or electric shock. The rats are unlikely to have learned anything during their first 24 hours of life. Instead, the most likely explanation for these findings is that rats are genetically disposed to learn about some relationships more easily than about others.

The concept of selective associations forms a physiological basis for the special form of learning, namely, “ guided ” (“ selective ”) learning, which means that species realise innate predisposition to form definite associations in accordance with their ecological and evolutionary traits (Gould and Marler, 1984; Griffin et al., 2002). We will consider significant implication of this state in Part VII.

Concerning the second and third rules listed above, we should note that relations between stimuli and reactions are not so simple and many questions concerning the nature of stimuli representation, attention, memory and learning are still not clear although the great progress has been achieved. In contrast to theories of classical and operant conditioning, intervening variable theories proceed from the assumption that learning can involve knowledge without observable performance. This will become apparent if we consider such classes of learning as latent (exploratory) learning, insight, and imprinting.

Latent (exploratory) learning. In his program article “The new formula of Behaviourism”, 1922, Tolman suggested to develop “ a new non-physiological behaviourism” aimed to objective study of internal processes concerning learning such as motive, purpose, determining tendency, and the like. Tolman developed this concept in many experiments, and some of them led to the theory of latent learning. This theory describes learning that occurs in the absence of an obvious reward. Latent learning is “hidden” internally rather than shown in behaviour.

For example, Tolman and Honzik (1930a, b) showed that rats could learn a rout in a maze without obtaining reinforcement. Three groups of rats were trained to run a maze. The control group, Group 1 was fed upon reaching the goal. The first experimental group, Group 2 was not rewarded for the first six days of training, but found food in the goal on day seven and everyday thereafter. The second experimental group, Group 3, was not rewarded for the first two days, but found food in the goal on day three and everyday thereafter. Both of the experimental groups demonstrated fewer errors when running the maze the day after the transition from no reward to reward conditions. The marked performance continued throughout the rest of the experiment. This suggested that the rats had learned during the initial trials of no reward and were able to use a cognitive map of the maze when the rewards were introduced.

Indeed, what is learned in such a situation is not a series of S-R connections, rather, the organism learns “what leads to what” and this may be considered a form of “Stimulus –Stimulus” learning. These stimulus relationships get organised into a cognitive map that is obviously more than a series of individual routs to a goal. Once an organism develops a cognitive map of its surroundings, it can get to a goal from any location. It follows the “principle of least effort” by choosing the rout requiring the least effort. This form of learning has been revealed in different species, from anthropoids to insects.

The term latent learning refers to only one phase of complex learning process. Thorpe (1963) preferred the term exploratory learning and defined it as the association of indifferent stimuli, or situations, those without patent reward. The motivation in latent learning situations seems simply be a desire to get to know the surroundings. Latent learning is undoubtedly very important for organisms in their real life. The survival value of exploratory behaviour may not be obvious at first, but its adaptiveness becomes apparent when a need arises and is quickly met. After establishing the location of a commodity such as food or a hole, the animal can reasonably expect it to be there when it returned (see Wallace, 1979). In a laboratory experiment by Metzgar (1967) two groups of mice were turned loose in a room with an owl. One group was given a few days to familiarise themselves with the room before the owl was introduced. The second group was put into the room at the same time the owl was. The mice that were familiar with the room fared much better – that is, fewer were caught.

A set of tests was elaborated by Reznikova (1981, 1982) for studying exploratory behaviour in ants in their natural surroundings (see Fig. VI-2).

Ants were presented by an “enriched piece of environment” that included several sorts of mazes, all lacking of food reward. These tests were aimed to compare levels of routine exploratory activity in different species which reside in the same territories and belong to the same species community. Exploratory activity in ants was measured by registration of time duration spent by the ants within mazes as well as numbers of visits. Ants of different species showed significant difference in intensity of exploratory activity. It turned out that explorative behaviour correlates with agility in hunting and searching foe hidden victims in different ant species.

Insight. A “what-leads-to what” expectancy is considered a basis for insight (Tolman, 1938). In the case of insight learning confirmation of expectancies lead to learning. Learning refers to cognitive knowledge generated by confirmed expectations, such as knowing how to navigate the maze or how to use things (such as put sticks together for getting a banana) after exploring them.

As it was mentioned in the Part I, W. Kö hler, one of founders of the Gestalt school of psychology, had argued for the place of cognition in learning. In particular, he suggested that insight played a role in problem-solving by chimpanzees. Rather than simply stumbling on solutions through trials and errors, the chimpanzees seemed to demonstrate a holistic understanding of problems, such as getting hold of fruit that was placed out of reach, by arriving at solutions in a sudden moment of revelation or insight. One of the best demonstrations of true insight in primates came from an experiment in which food was placed outside the arm’s reach of a chimpanzee; the animal used short sticks which were placed within its cage, to reach longer stick placed outside, and then used the longer sticks to reach the food. These results support the Gestalt assertion that it is the “whole” rather than S-R elements that guide learning. A chimpanzee in such situations tries a number of possibilities mentally, and then tries it physically (see Fig. II-5).

In general, the learner ponders a problem thinking about possible solutions (this is called a pre-solution period) until one suddenly becomes apparent (insight).

There are several characteristics of insight learning:

- Transition from pre-solution to solution is sudden and complete

- Insight is based on performances are error-free

- Solutions gained by insight are retained longer

- Solutions gained by insight are easily applied to other problems (transposition)

For a long time, it was believed that insight learning is not possible below the primate level. Recently it has been shown that at least several bird species can learn this way and that “Folk Physics for Apes” (Povinelly, 2000) may be adapted for crows, finches and jays. This encourages investigators to take a look at insight learning in various other species (see Chapters 17 and 18 for detailed analysis).

Imprinting. Imprinting as one of classes of learning implies both features of conditioning and insight. One could name this “ontogenetic insight” as during a particular short period of its ontogenesis an organism suddenly “learns” what to do. At the same time imprinting implies both features of learning and instinctive behaviour. Imprinting occurs when innate behaviours are released in response to a learnt stimulus. This form of learning has been described as a rapid learning of certain general characteristics of a stimulus object. Most imprinting promotes survival of newborn animals and shapes their future breeding activities. There are still many discussions and criticism of the imprinting paradigm. Many researchers believe that the imprinting phenomenon has too many special qualities for it to be considered just a special form of learning. The phenomenon of imprinting will be considered in details in Chapter 24.

CONCLUDING COMMENTS

In the studying of animal intelligence, we can discern basic forms of learning some of them can be considered relatively simple (such as habituation and classical conditioning), some (such as imprinting and guided learning) have special qualities which make difficult analysis of them in isolation from innate forms of behaviour, and others (such as “insight”, “latent learning” and social learning) require more cognitive explanations that will be given further in this book.

Although more than a century has elapsed since researchers turned from naturalistic studies of animal intelligence to experimental investigations and measuring characteristics of learning, there are probably no experimental paradigms in this field about which one can say that they have only historical importance. For example, habituation and conditioning still serve as the basis of experimental paradigms widely used by experimenters for studying abstraction and concept formation in a wide variety of species including humans. Relatively simple rules of operant conditioning can be used not only for shaping and govern complex behaviour but also for enabling organisms to exhibit creative abilities.

The whole picture of interaction between different forms of learning in subject’s mentality is yet far from completion. Even for “simple” forms of learning its mechanisms and correlation with memory processes are still not completely clarified.