Shaping Ratna Day 1: Training Session 2

                       Training Session 2 performed 2/25/2016 @ 4:00 PM.

 

 Goal 
Since Ratna successfully completed magazine training, the next process was to shape her. Shaping, as used in training Sniffy, is to reinforce a behavior by successive approximations of that desired behavior. In this case, I want to shape Ratna until she can associate to press the bar on her own to get reinforcements from the magazine. 

Procedure
In order for Ratna to seek the reinforcements in the Operant Box, food deprivation was continued to maintain her target weight of 199 grams. At the time of training, Ratna weighed approximately 200.4 grams, which is about 1.4 grams over her target weight. In Operant Box 5, I began training Ratna from 4:00 PM and ended 28.51 minutes later around 4:28 PM. During the duration of shaping Ratna, I only reinforced successive approximations of bar pressing behavior.  Ratna was first reinforced if she either smelled the bar or got near the bar. The instant she reared up near the bar or touched the bar, I instantly gave her a reinforcement. If Ratna displayed other variant behaviors such as smelling the light or rearing up away from the bar, I did not reinforce her because shaping would be trained improperly.  If she got close to the underside of the bar, I did not reinforce her for that behavior. During the duration of this training session, I gave 95 manual reinforcements and she pressed the bar 5 times. ( See the video below of Ratna during part of her 1st Shaping session)

Results
  As evidenced in the video above, Ratna did develop a medium association between the action of pressing the bar and the receiving the pellet of reinforcement. The first half of the training sessions Ratna was reinforced for behavior near the bar. The last half she was only reinforced for rearing up the near the bar or putting her hands on the bar. Ratna fairly quickly responded overall based on general observations. 

Discussion

Through the process of shaping, Ratna did perform a variety of behaviors that did effect her training session. Ratna is a clever rat and is pretty hyperactive, which makes it easier for operators like me to train rats.  Ratna quickly picked up the contingency between pressing the bar and the pellet reinforcement.Thus, she continued to have a vast variety of behaviors such as: rearing up throughout the cage, sniffying/touching the lights, floor, and hopper. One thing I did notice was that Ratna understood she had to do something with the bar. She got to the point where she put her hands on it, but wasn't pressing it most of the time. My plan for tomorrow is to continue shaping her but to be more specific in the behaviors I reinforce. For example, I would only reinforce her if she reared up over the lever, as she came down, and then when the front feet were on the bar. If I did these more specific successive approximations, Ratna will understand the motion better, and hopefully, if I do so, Ratna will put more pressure to press the bar more. Even though this is only my first day of shaping, I have noticed that although shaping as learned in class is pretty similar, it is going to take a while for Ratna to develop the association. In turn, it has shown me that the operators have to be very careful when exactly they reinforce so that the wrong behavior will not be reinforced.

Magazine Training Ratna: Training Session 1

Training Session 1 performed 2/24/2016 @ 1:30 PM.

 Goal: Magazine train Ratna so she develops an association between the magazine, aka the food hopper, and the reinforcement. My main goal is to condition Ratna’s behavior so that after she sees the light in the operant box flash on/off and hears the pellet drop into the food hopper she approaches the food hopper in a systematic manner. 

Procedure: The week before I began training Ratna, she was food deprived to approximately 90% of her original body weight. Ratna originally weighed approximately 221 grams, but after a week of food deprivation she reached her target weight of 199 grams on training day. My training sessions are scheduled to usually last from 4-4:30 PM five days a week, however my first training session started at 1:30 because I worked with Dr. Trench my first time training Ratna. As pictured below, this is the operant box I will be using to train Ratna. It is very similar to the one I used to train Sniffy the Virtual Rat. To deliver a chocolate pellet to the magazine, I had to used a manual trigger. To begin with I delivered one pellet and as soon as Ratna ate one, I gave her another one. Throughout the duration of training session, that lasted approximately 21 minutes long, I gave Ratna reinforcements constantly. Shaping was starting to be performed during this training period but was not fully completed. In total, Ratna pressed the bar 3 times on her own, and I gave 47 manual reinforcements. 

Operant Box (left pic)/ Pellet Dispenser & Manual Trigger( right pic).

Results: One observation I noticed was that when reinforcement was delivered, Ratna consistently would go to the food magazine. In turn, the results of magazine training was complete because Ratna developed a strong association between the light turning on and off & sound of the pellet with receiving the food pellet reinforcement in the magazine. She also developed a minimal association between pressing the bar and receiving an reinforcement.

Discussion: Overall, Ratna maintained good attention when I was training her. She did illustrate a variety of behaviors such as scratching her nose or tail, walking around the box, jumping up in different regions, and even sticking her head in the hopper. However, whenever I did give her a reinforcement she went immediately to the hopper. If she did wander off from the hopper, it was very easy to bring her back by providing a reinforcement. Shaping was slightly achieved towards the middle and end of the session. Ratna began sniffing the bar and did press the bar a few times.  Overall, magazine training Ratna was similar to how we learned about the concept in class and how I trained Sniffy. Ratna picked up the association between the sound of the pellet and the on/off light to the reinforcement of the food. Tomorrow, I intend to work more on shaping Ratna to press the bar.

Part 2: Training Sniffy the Virtual Rat Fixed Interval & Extinction

 Goal 3: Fixed interval schedule was put on Sniffy after magazine training and shaping were completed. Reinforcement delivered after a set amount of time passes. 

Procedure: In this schedule, Sniffy was given reinforcement after 15 second intervals (FI-15). Sniffy pressing the level were more far out in this schedule compared to the CRF (continuous reinforcement). Time was sped up and data was recorded in the cumulative recorder.

Results/ Discussion: Instead of rewarding Sniffy every time he pressed the lever/bar, Sniffy was rewarded after a set time. Sniffy got kind of confused when he pressed the lever and no food came out. He started pressing the bar fast and several times in a row. Before this schedule, Sniffy in shaping pressed the lever in a methodical manner, however in fixed interval he was pressing the bar more aggressively. After 5-10 minutes, Sniffy figured out he would have to wait longer to get a pellet. The cumulative recorder, evidenced in figure 3 below, shows that Sniffy started to receive a pellet of food after waiting 15 seconds.  Under fixed interval, the performance is not steady like under fixed ratio schedule. Sniffy increased the rate of pressing the lever when he closer to the end of the interval. Overall, what I learned about fixed interval schedule is behavior is reinforced the first time it occurs after an interval.

Figure 3: Cumulative Recorder for Fixed Interval (FI) Reinforcement (sloped lines with dashes represent when Sniffy presses the bar-presses.FI-15 indicated the start of fixed interval reinforcement for 15 seconds.

Part 1: Training Sniffy the Virtual Rat Magazine Training & Shaping

This week I trained a Sniffy the Virtual Rat before I began training my real rat, Ratna. This computer program I used simulates an operant box environment, much like the one I will use with Ratna. The operant box, shown in the picture below, has a food hopper, a bar that will release pellets of food when pressed, a water spout, a speaker, and a signal light.  It allowed me to train a Sniffy with variable behaviors via operant conditioning. Overall, this program allowed me to feel comfortable with the steps taken to train my own rat. Before completing this assignment, I was not fully aware how to exactly begin magazine training, shaping, or other reinforcement techniques. However after working with Sniffy on these various training measures, I am more confident how to exactly train Ratna. I do realize that Ratna will not behavior/ learn in a similar way as Sniffy, but I have a better understanding on how to get better results when I start training Ratna.

Sniffy in Operant Box

---

Goal #1: Magazine train Sniffy so he improves his association between the bar sound of the food pellet falling into the hopper and the reinforcement, the food.

Procedure: The duration to magazine train Sniffy was approximately 15-20 minutes. I thought magazine training would take a long time to do but it was easy, quick, and straightforward. First, I let Sniffy walk around the cage to get used to the environment. I offered Sniffy reinforcements, in this case the pellets of food, when Sniffy approached anywhere close to the hopper. I increased the rate of reinforcement the closer Sniffy got to the hopper.

Results/ Discussion: As repeated reinforcements were given to Sniffy, there was increase in the association between the sound of the bar and receiving pellets. The software had a bar graph, which allowed me to see how and when to deliver reinforcements in a timely manner. The last 5-10 minutes Sniffy barely moved to other quadrants of the operant box, but he remained close to the hopper. Figure 1 below shows the cumulative recorder for magazine training, and the bar graph for magazine is also shown. 

Figure 1: Cumulative Recorder for Magazine Training (dash lines on the top of the graph denote reinforcements controlled by me, the operator. Each divided section of the graph signifies five-minute intervals. CRF Press Bar directs the start of Sniffy’s shaping.)

                                          

  

                                                 

                                                  Bar graph of Sniffy’s Magazine Training.

Goal #2: Shaping Sniffy to press the bar on his own by using reinforcement. Shaping is achieved by reinforcing successive approximations of Sniffy’s bar pressing behavior.

Procedure: The duration it took to shape Sniffy to bar press was approximately 1 hour.
I began shaping by rewarding any  behaviors performed near the back of the cage. I placed Sniffy on variable ratio reinforcement, meaning I gave him a reinforcer after a specified number of correct responses. I reinforced Sniffy more specifically every time he either got close to the hopper,reared up on the back wall, or passed the front of it. After shaping Sniffy for 3-5 minutes, he began to press the bar on his own. I tracked Sniffy's progress for his association between bar pressing and the reinforcement. I continued for 20 minutes on an FR1 (fixed-ratio one) schedule of Sniffy rearing up and pressing the bar at least once. Slowly, I began reinforcing Sniffy until he began pushing the bar more times in a consecutive fashion (2 times in a row, 3 times in a row, etc). I also gave Sniffy reinforcements when he wandered off from the hopper because shaping could continue to progress when he returned to the location. Soon Sniffy was pressing the bar about 25-30 times per a five-minute period. I let the program run for the last 25- 30 minutes without me reinforcing him, so Sniffy could reinforce his own behavior. 

Results/ Discussion: Shaping Sniffy was definitely more difficult because it requires more time and accurate timing when reinforcements are given. I had to reinforce carefully because I did not want him to be fixated on a particular behavior beside pressing the lever. Once Sniffy realized he had to stay close by the hopper, shaping him progressed much faster. He soon learned to rear up in order to receive a pellet and figured out how to press the lever in order to get food. Overall, shaping has showed me that one can get higher, steady response rates. By using fixed ratio reinforcement, rewards were provided after an unpredictable number of responses with only brief pauses in behavior after reinforcement.  As one can see in the bar graph below, Sniffy associated reinforcement with the bar sound at a max strength. Figure 2 below shows the cumulative recorder for shaping. This part of the cumulative recorder data was taken toward the end of shaping where acquisition of the bar pressing behavior had a high strength, Sniffy pressing the bar 20+ times in a 5 minute period.  

                        

Bar graph depicting Sniffy’s bar press for shaping.

    Figure 2: Cumulative Recorder for Shaping (dash lines on the top of the graph denote reinforcements controlled by me, the operator. The sloped lines with dashes represent when Sniffy presses the bar-presses).