A Role for Dorsal and Ventral Hippocampus in Inter-Temporal Choice Cost-Benefit Decision Making

Orbitofrontal cortical (OFC) lesions affect how long rats are willing to wait for rewards (Mobini et al., 2002). Compared to controls, rats with OFC lesions display an increased preference for the goal arm of a T maze containing an immediately available low reward (LR) over the goal arm containing a higher reward (HR) that is only available after a delay (Rudebeck, Walton, Smyth, Bannerman, & Rushworth, 2006). Rats with hippocampal (HPC) lesions also exhibit impulsive choice and prefer immediately available rewards over delayed rewards (Cheung & Cardinal, 2005; Rawlins, Feldon, & Butt, 1985). For example, Rawlins and colleagues trained rats with HPC aspiration lesions and controls on a task in which one arm of a Y maze was continually reinforced (CRF) and the other reinforced on only 25% of trials (partially reinforced arm – PRF). During the training phase, when no delays were present on either arm, all rats showed a clear preference for the CRF arm. During the test phase, the reward in the PRF arm (when present) was still available immediately, but access to the reward on the CRF arm was delayed by 10 seconds. After the introduction of the delay, controls continued to choose the CRF arm, but HPC lesioned rats switched their preference to the immediate PRF arm.

As the hippocampal lesions employed by Rawlins et al. (1985) were aspiration lesions, it is not clear whether the impulsivity observed on this delay-based, cost-benefit decision making T maze task was due to hippocampal cell loss or due to damage to fibers of passage or cerebrovasculature caused by this kind of surgical manipulation. A recent study by Cheung and Cardinal (2005) has shown that cytotoxic hippocampal lesions do produce impulsivity in an operant version of this task. Therefore, the first aim of the present study was to see if the preference of HPC lesioned rats for an immediately available low reward over a delayed high reward on the T maze task was also present following cytotoxic, fiber-sparing lesions of the HPC.

The second aim of the present study was to assess the effects of hippocampal lesions on a T maze paradigm similar to that employed by Rudebeck et al. (2006), and thus allow direct comparison with the effects of orbitofrontal lesions. There are a number of procedural differences between the Rawlins et al., (1985) and Rudebeck et al. (2006) studies (e.g., continuous/partial reinforcement vs. high/low rewards; preoperative vs. postoperative training on the task). We now tested the effects of hippocampal lesions using the same paradigm that has previously demonstrated impulsive choice in OFC lesioned rats.

Furthermore, in the study by Rawlins et al. (1985), HPC lesioned rats showed a clear preference for the CRF arm when there was no delay on either arm. The third aim of the present study was to extend this investigation to see whether or not HPC lesioned rats would be impaired when using a double-delay procedure in which a 10s delay was introduced into both goal arms, and thus interposed between the choice point and receiving either the LR or HR (e.g., Rudebeck et al., 2006).

The fourth aim of the present study was to assess the effects of hippocampal lesions on a similar version of the T maze decision making task, but in which the cost associated with the HR was in terms of physical effort rather than delay to reinforcement. Anterior cingulate cortex (ACC), but not OFC, lesions affect how much effort rats decide to invest for rewards, as measured by a reduced willingness to climb over a barrier to obtain a high reward when the alternative is a lower reward requiring less effort (Rudebeck et al., 2006). In the present study, we now assessed whether any role of the hippocampus in cost-benefit decision making was specific to tasks in which the cost was in terms of delay to reinforcement, or whether it also extended to other kinds of costs such as physical effort.

The final aim of this study was to investigate the relative contributions of the dorsal and ventral HPC subregions to cost benefit decision making. The HPC has long been associated with learning and memory, particularly within the spatial domain (O'Keefe & Nadel, 1978), but more recent studies suggest that these spatial functions are largely subserved by the septal portion of the HPC (dorsal in rodents, posterior in primates), whereas a lesser role is played by the temporal region (ventral in rodents, anterior in primates). Septotemporal differences in spatial processing are consistently found following selective cytotoxic lesions in rats, with dorsal but not ventral lesions resulting in robust and reliable deficits in spatial reference (Moser, Moser, Forrest, Andersen, & Morris, 1995) and working memory tasks (Bannerman et al., 1999).

This functional differentiation is consistent with the anatomical connectivity along the septotemporal axis. The dorsal HPC receives highly processed sensory information via the entorhinal cortex (Dolorfo & Amaral, 1998), consistent with a role in spatial information processing. In contrast, the ventral HPC shares greater connectivity with structures such as the amygdala, hypothalamus, and orbitofrontal cortex (OFC), which are more commonly associated with emotional processing. Indeed, ventral but not dorsal HPC lesions produce anxiolytic effects on unconditioned tests of anxiety (Deacon, Bannerman, & Rawlins, 2002; Kjelstrup et al., 2002; McHugh, Deacon, Rawlins, & Bannerman, 2004). These findings are consistent with a more general role for the hippocampus in the integration of multimodal sensory and emotionally salient contextual information to guide response selection (Gray & McNaughton, 2000). Given the involvement of the OFC in the delay-based, cost-benefit decision making task (Rudebeck et al., 2006), it is interesting to note that the direct projections from the CA1 field of the hippocampus to the OFC are only present in the more ventral/caudal regions of the hippocampus in the rat (Jay & Witter, 1991). This suggests that the ventral HPC may play a key role in the delay-based, cost benefit decision making T maze task, in contrast to other mnemonic versions of the T maze task where it is not required (Bannerman, Yee, Good, Heupel, Iverson, & Rawlins., 1999; Hock & Bunsey, 1998). Therefore, the final aim of this study was to test the hypothesis that the vHPC may be the critical subregion for performance on the delay-based, cost-benefit decision making task.

We therefore compared the effects of complete, dorsal and ventral, cytotoxic HPC lesions on a T maze task in which rats chose between an immediately available LR and a delayed HR (Rudebeck et al., 2006). In addition, performance was also compared under conditions in which an equivalent delay was present in both the HR and LR arms. For comparison, a second experiment examined response choices when the HR was associated with additional effort (climbing over a barrier), with less effort (no barrier) required to obtain the LR (Rudebeck et al., 2006; Walton, Bannerman, & Rushworth, 2002).

Materials and Method

Animals

This study used 40 male Lister hooded rats (Harlan Olac, Bicester, U.K.) that were ~2 months old and experimentally naïve at the start of training. Rats were housed in groups of two or three on a 12hr light/dark cycle (lights on at 7:00 a.m., with testing during the light phase). During the experiment rats were maintained on a restricted diet at ~85% of their free-feeding weight but had access to water ad libitum in their home cages. The experiments described were conducted in accordance with the United Kingdom Animals Scientific Procedures Act (1986) under project license number PPL 30/1989.

Apparatus

The rats were tested on an enclosed, high-sided wooden T maze placed 42 cm above floor level (Rudebeck et al., 2006). The start arm joined two goal arms. Each arm was 60 cm long, 10 cm wide, and had 40 cm walls. A raised metal food well (2.5 cm diameter, 2 cm high) was situated at the far end of each goal arm, 2.5 cm from the back wall. Runners were installed in each of the goal arms permitting one “guillotine” door (50 cm high × 9 cm wide × 0.6 cm thick) to be inserted 5 cm along the goal arm (from the junction in the T maze) and a second identical guillotine door 5 cm before the food well (Rudebeck et al., 2006). Doors delayed access to the HR food well after the animal had made a choice (Experiment 1) and also prevented access to the other goal arm during forced trials (Experiments 1 and 2). The maze and doors were painted a uniform gray color throughout.

In Experiment 2, animals had to exert additional effort to obtain the HR by climbing over a barrier (Rudebeck et al., 2006; Walton, Bannerman, Alterescu, & Rushworth, 2003; Walton et al., 2002). Six different barriers were used (15 cm, 20 cm, 25 cm, 30 cm, 35 cm, or 40 cm in height), each constructed from wire mesh in the shape of a 3-dimensional right-angled triangle. The rat had to scale the vertical face of the barrier and descend the slope to get to the food well.

Preoperative Training for the Delayed Reward Task

During food deprivation, the rats were weighed and handled by the experimenters on a daily basis and then thoroughly habituated to the T maze. Over the first 10 days of preoperative training, rats learned to associate one arm of the T maze with the LR and the other arm with the HR in the absence of any delays. In all trials in Experiment 1, the food well in the HR arm contained 10 food pellets (45 mg Formula A/I; Noyes, Lancaster, NH), whereas the food well in the LR arm contained 2 pellets. Allocation of the HR and LR to the left and right arms of the T maze was fully counterbalanced (50% HR = right, LR = left; 50% HR = left, LR = right).

At the start of every training session each rat received two forced trials (one to the HR arm and one to the LR arm) during which a door prevented access to the other arm of the T maze. On each choice trial, the rat was placed in the start arm and allowed to enter either the LR or HR arm. Upon entering either goal-arm, the door behind the rat was closed and the door preventing access to the food was immediately lifted. In other words, there was no delay cost associated with the HR arm during this initial stage of training. For the first 6 days of training, rats received two choice trials per day and for an additional 4 days the rats received five choice trials per day.

Once all the animals reliably chose the HR on more than 80% of trials, a 5s delay was introduced to the HR arm. On trials where rats entered the HR arm (on choice or forced trials) the door would close behind them and they would be detained in the arm for 5s. At the end of this detainment period the door preventing access to the food well was opened and the rat could eat the 10 pellets. No delay was associated with the choice of the LR arm.

Preoperative Testing for the Delayed Reward Experiment

After three days (15 choice trials) of testing with a 5s delay in the HR arm, the delay was increased to 10 seconds and 6 days of testing began. These 30 choice trials (3 blocks of 10 trials) provided a preoperative baseline (Figure 1a). As with all other training days, each session began with two forced trials (one to each arm, with HR first or LR first, counterbalanced with respect to day). Rats were run in squads of five or six giving an approximate intertrial interval of 10 minutes.

Figure 1

Mean high reward (HR; 10 pellets) choices out of 10 (+/- SEM) in the intertemporal choice task for rats with complete hippocampal (cHPC), dorsal hippocampal (dHPC), ventral hippocampal (vHPC) or sham lesions. Rats received 2 pellets for choosing the low reward.

Surgery

The assignment of rats to surgical groups was counterbalanced with respect to preoperative performance at the 10s delay and with respect to the right-left allocation of the HR and LR. Rats received either excitotoxic bilateral lesions encompassing the dorsal hippocampus (dHPC; n = 10), ventral hippocampus (vHPC; n = 10), or complete hippocampus (cHPC; n = 10); or sham surgery (sham; n = 10). At the time of surgery, the rats weighed between 327 g and 438 g. All rats were anesthetized with Avertin (0.29 g/kg, i.p.) and placed in a stereotaxic frame with the head level between bregma and lambda. An incision was made along the midline, and a drill was used to remove the portion of bone overlying the injection sites. Lesions were made by injecting N-methyl-D-aspartate (NMDA; Sigma Chemical, Poole, U.K.), dissolved in phosphate buffered saline (pH 7.4), at a concentration of 10 mg/ml, at the coordinates specified in Bannerman et al. (2002; see also Supplemental Information, Table S1). Injections of between 0.025 and 0.1 µl were made over 15-60s (0.1 µl/min) with a 5-µl microsyringe (Scientific Glass Engineering, Milton Keynes, U.K.) mounted on a stereotaxic frame using a modified 34-gauge needle. The syringe was left in place for 60s after each injection to allow diffusion of the neurotoxin away from the injection site. Sham surgery involved anesthesia followed by a midline incision, craniotomy, and then suturing. One rat died shortly after surgery leaving a final experimental cohort of 39 rats (cHPC, n = 9; other 3 groups, n = 10).

Procedure

Experiment 1: Intertemporal choice on the T maze (postoperative testing)

After 14 days postoperative recovery, rats were returned to a restricted feeding schedule (~85% of free feeding weight) and testing resumed. Postoperative testing was divided into three main stages. The first stage followed an identical protocol to the preoperative baseline testing except that four blocks (40 trials) were run (Figure 1b). In brief, following two forced trials at the start of each session, rats received five choice trials per day in which access to the food in the HR arm (10 pellets) was delayed by 10 seconds whereas access to the food in the LR arm (2 pellets) was granted immediately. In stage 2, the delay in both the HR and LR arms was set to 10 seconds (20 trials; Figure 1c). In stage 3 (Figure 1d), the original parameters (HR = 10s; LR = 0s) were reinstated. During the 1st three blocks of testing in stage 3 (30 trials), the majority of rats continued to select the HR arm and therefore received very little exposure to the new contingencies in the LR arm (i.e., that reward was no longer delayed in this arm). The rats were therefore given only forced trials for two days (20 forced trials in total, 10 to HR arm, 10 to LR arm) to expose them fully to the contingencies in both arms before testing continued for a further six blocks (60 trials).

Experiment 2: Effort-related reward on the T maze

The purpose of Experiment 2 was to examine the effect of associating extra effort with the HR arm by inserting a barrier. Training followed directly from Experiment 1 and the same HR/LR arm allocations were kept, for example, if the HR was in the left arm in Experiment 1, then it remained in the left arm for Experiment 2. However, the size of the HR was reduced to 4 pellets with the LR staying at 2 pellets (based on Walton et al., 2002). As with Experiment 1, all sessions began with 2 forced trials (1 into each arm) but 10 choice trials were run per day. Rats were given a minimum of 30 trials to regain the HR/LR associations without any cost being associated with the HR arm. Those who did not choose the HR arm on 80% of trials on three consecutive days were given additional training. Once all animals met all criteria, a further 3 days of testing (30 trials) provided a period of baseline testing with no barrier in place (Figure 2a). The smallest barrier (15 cm) was then introduced and 3 blocks (30 trials) were run (Figure 2b). Following these 3 blocks, all rats received forced trials only over two days (10 forced trials per day, 20 in total) with the 15 cm barrier in place to ensure adequate exposure the reward/effort contingencies. Thereafter, the rats received 30 trials at each barrier height (15 cm (after forced trials), 20 cm, 25 cm, 30 cm, 35 cm, 40 cm; Figure 2, c-h), giving a total of 240 choice trials over 24 blocks.

Figure 2

Mean high reward (HR, 4 pellets) choices out of 10 (+/- SEM) in the effort-related choice task for rats with complete hippocampal (cHPC), dorsal hippocampal (dHPC), ventral hippocampal (vHPC) or sham lesions. Rats received 2 pellets for choosing the low reward.

Data Analysis

Data were subjected to analysis of variance (ANOVA) using a general linear model in SPSS (Version 11, Chicago). All tests of significance were performed at a = .05 and full factorial models (SPSS's Type III sum-of-squares method) were used. All graphs show group means plus/minus 1 standard error of the mean (SEM).

Histology

At the end of behavioral testing, rats were injected with Euthatal (200 mg/ml sodium pentobarbitone; 200 mg/kg) and perfused transcardially with physiological saline (0.9% NaCl) followed by 10% formol saline (10% formalin in 0.9% NaCl). Their brains were then removed and placed in formol saline solution and subsequently transferred to a 30% sucrose-formalin solution for 24 hr, frozen, and sectioned (50 µm) horizontally. All sections were then stained with cresyl violet.

Results

Histology

Complete (cHPC), dorsal (dHPC), and ventral (vHPC) hippocampal lesions were highly reproducible and highly selective and consistent with previous studies in this laboratory (Bannerman, Grubb, Deacon, Yee, Feldon, & Rawlins, 2003; Bannerman, et al. 1999; Deacon et al., 2002; McHugh et al., 2004). In all cases, cell loss was restricted almost exclusively to the hippocampal subfields, with little if any damage to adjacent structures such as the subiculum or entorhinal cortex, and no damage beyond these structures. In the cHPC and dHPC lesioned rats, there was complete loss of pyramidal and granule cells in the dorsal part of Ammon's horn and the dentate gyrus, respectively. In the dHPC group there was little if any evidence of damage beyond Plate 108 (Paxinos & Watson, 1998). In the vHPC lesioned rats there was little or no damage evident before Plate 109. In the cHPC and vHPC lesioned rats there was some very restricted sparing at midhippocampal levels, involving the most posterior portion of the CA1 subfield and the dentate gyrus, at the apex of the hippocampus as it starts to curve downward. In the cHPC and vHPC groups, the lesion then became complete once more as it extended into the ventral hippocampus, until the most ventral tip of the hippocampus at which point some sparing once again became apparent, and mainly involved the most posterior portion of the dentate gyrus. No rats were excluded on histological grounds. Reconstructions and photomicrographs of the lesions are available in the supplemental information (Figures S1-S4).

Experiment 1: Delayed reward

Data for each block of Experiment 1 is presented in Figure 1, which also serves as a timeline for each stage of delayed-reward testing. Three blocks of testing were run prior to surgery. As group allocation was determined by performance at this stage, there were no differences between the groups as confirmed by a repeated-measures ANOVA