Skill Retention Research

Today I am looking up some research on skill retention and skill decay for a proposal.

References I looked up:

1. http://www.tatrc.org/conferences/MMVR_2011/ppt/ONeil-MMVR-CCC-2011.pdf

The above is a set of slides on the topic of skill retention in medicine. They advocate an ITS-like approach to addressing skill decay. A couple of points:

1. School knowledge decays less than (what? not mentioned).
2. Motor skills decay less than cognitive skills. Fall steeper and faster.
3. They recommend an approach of modeling learning skills and targeting decaying skills with lessons targeting a single skill.

2. http://www.owlnet.rice.edu/~antonvillado/courses/12a_psyc630001/Arthur,%20Bennett,%20Stanush,%20&%20McNelly%20(1998)%20HP.pdf

A meta analysis of factors that influence skill retention/decay.

1. Overlearning leads to better retention
2. Closed-loop task knowledge decays faster than Open-loop task expertise (close-loop task = those with a fixed sequence of tasks that have a definite beginning and end)
3. Speed tasks decay less than accuracy tasks
4. Physical skills decay less than cognitive skills
5. Skill on artificial tasks decays more than natural tasks
6. Studies that used recognition tests reported less skill decay than those that used recall tests
7. Skill decay is more apparent when the retention test does not have the same context and the learning context.

Research at NCSU Intellimedia Center

Spoke to James Lester today and he spoke about a couple of projects. Looked up the following papers.

Kristy Elizabeth Boyer, Robert Phillips, Amy Ingram, Eun Young Ha, Michael D. Wallis, Mladen A. Vouk, and James C. Lester. Investigating the Relationship Between Dialogue Structure and Tutoring Effectiveness: A Hidden Markov Modeling Approach. International Journal of Artificial Intelligence in Education, 21(1-2), 65-81, 2011.


http://people.engr.ncsu.edu/keboyer/papers/boyer-ijaied2011.pdf


Eun Young Ha, Jonathan Rowe, Bradford Mott, and James Lester. Goal Recognition with Markov Logic Networks for Player-Adaptive Games. InProceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, Toronto, Ontario, Canada, pp. 2113-2119, 2012. (invited paper)


http://www.intellimedia.ncsu.edu/papers/ha-aaai-2012.pdf


The first paper describes a project where they gathered massive amounts of data from one-on-one human tutoring interaction. This data was tagged by humans into various dialog act categories. They then used HMMs to discover tutorial interaction modes from the data. These modes were correlated with student learning data to identify the most effective modes.

The second paper is about using Markov Logic Networks for plan recognition in free-play games. Again generated lots of data by getting lots of folks to play a game. Player actions were then manually tagged with an associated goal. A standard MLN learning algorithm was used to learn a network from the data. Comparisons with other techniques (strawman, unigram, bigram models), MLN was found to be most accurate.

Progressive formalizaton

This is an approach to developing learner-entered environments for formal disciplines such as math.

Here the idea is to startt with the informal knowledge that kids bring to school and show tem how to transform these to formal knowledge. Initially students use their own words and picture to describe mathematical situations. Over time, kids are then encouraged to formalize these represntations. Initially, they are allowed to invent their own notationss etc. Gradually, they are encouraged to replace these with conventional, formal notations.

This path is not always one way. Learners may back and forth between these levels.

The design in educational environments: learner-centered environments

Environments that pay attention to the knowledge, skills, attitudes, and beliefs that learners bring to the educational setting.

Also called "culturally appropriate", "culturally responsive"...

Also called "diagnostic teaching": a key strategy is to prompt children to explain the reasons for predictions, their answers etc. Create congnitive conflicts and then have discussions about conflicting viewpoints. Test for known misconceptions.

Important to be sensitive to cultural practiced. In Hawaii, a reading program included discussions of learners' personal experiences and gave them opportunities to practice "talk-story"-a native custom of constructing stories jointly. This led to improvements in standardized test scores.

Not all kids come to school with practice in "school talk" - I.e. Impersonal and expository way of talking without relating to personal experiences or stories.

In one case, African American students were shown that many of their forms of everyday communication are examples of high forms of literacy.

In learner centered environments, teachers pay attention to what learners know and can do, and in their interests and passion - what do they want to do.

Teaching for transfer: Part 2

Continuing with the book: How People Learn

1. Transfer learning should be viewed as increased speed of lesrning in a new domain. So sometimes you might have to wait to see the effects of transfer. Thus it may not be apparent on day 1 that students have transferred their learning from a previous domain. But by day 2 or 3, the effect may start becoming motivator.

2. Students may need prompting at first to map their previous learning to current task. Prompts such as "does this remind you of something you have done recently" can be used and gradually withdrawn.

3. Metacongitiv lesrningcan facilitate transfer without prompts.

A. Reciprocal teaching for reading comprehension is an example of metacognitive teaching.

B. Alan schoenfeld (1991) has developed an approach for teaching math problem solving using polya's heuristics and teaching them using a reciprocal approach. Initially, the teacher demonstrates the processes for generating alternative courses of action, evaluating which course is appropriate for the problem, and assessing one's progress. Gradually students learn to ask themselves these questions and the teacher's modeling is withdrawn.

Learning and transfer

From the book "How People Learn: Brain, Mind, Experience, and School"

1. Successful transfer is determined the degree of mastery attained over the original material.

2. Transfer requires that people learn material with deep understanding rather than say memorizing facts and such.

3. Acquiring expertise takes a long time. It can take 50000-100000 hours of practice, for example, to become world-class chess master. Much of the time is spent in lesrning pattern recognition skills.

4. A training program that tries to cram too much into a short amount of time may lead to shallow learning.

5. Deliberate practice that involves active monitoring of ones's learning is important for achieving expertise. This involves practice with feedback that focuses in understanding. Feedback should be provided on the degree to which learners understand when and where and how to use the knowledge they are learning.

6. Understanding when and where to use knowledge can be taught by the use of contrasting cases.

7. Transfer is also enhanced when students can see the potential use of the knowledge to other areas or for other purposes. E.g. Students learned Logo debugging skills better when they were expected to write a user manual on debugging at the end of training.

8. Motivation is an important consideration while lesrning. Activities must be at the appropriate challenge level. Not too easy, not too hard.

9. Goal orientation plays an important role in motivation. Students with a learning orientation will be motivated by more challenging and new problems than those with a performance orientation.

10. Goal orientation may change over time and depend on the area of study. This aspec has not yet been studied.

11. Lessens are motivated when they see that their work has a social use beyond the classroom. Example, one study found that inner city kids were motivated by activities such as tutoring other children, making presentations to an outside audience, making blueprints for playhouses that were then built and donated to local schools... Contributing to their social groups and having an impact on their local communities tends to be motivating.

Faded Examples

1. Recent research shows that showing worked out examples to novice learners in well-defined domains (math, physics etc.) is critical for effective learning. (Notes: Find out what research shows exactly. How did they should the worked out examples. What does critical mean? What improvements in learning gains did they find?)

2. Self-explanation of worked out examples is critical to the effectiveness of this approach.

3. Two effective self-explanation strategies:

      a. Anticipatory self-explanation: Learners anticipate the next step. A high degree of prior knowledge is  prior knowledge required to use this strategy effectively.

      b. Priniple-based self-explanation: Learners explain the current step in terms of principles. Works for learners with low prior knowledge.

4. Some studies did not show improved learning with learners being prompted for self-explanation while reviewing worked out examples.

5. Renkl showed that:
      a. A sequence of instruction which starts out by showing worked out examples and then progressively withholding solution for some steps for the student to solve improved near-transfer (compared to an approach of using example-problem pairs. i.e. where you show an example, then ask them to solve a similar problem.)

      b. The number of problem-solving errors generated mediated the effectiveness. (Note: Not sure what "problem-solving errors generated" means.

      c. The procedure was most effective when the steps were faded from the back (last step first and so on progressively backwards).

6. This paper discusses an experiment to study the effects of combining worked out examples with self-explanation prompts.

7. Four conditions were compared: Example-Problem (EP) pairs only, EP+prompts, Backward-fading (FD), FD+prompts. The first experiment involved college students learning to solve probability word problems.

a. Self-explanation prompts were implemented for each worked solution step provided to the learner. Each of this steps showed a list of principles. Learners had to select the ones that applied to the step. Once learners made their choice, the tutor displayed both their choices and the correct choices for them to see.

b. They found that backwards fading had a significant effect on both near transfer problems and far transfer problems in terms of problem-solving success (regardless of prompting).

c. They found that prompting had  a significant effect on both near transfer and far transfer (for both EP or BF conditions)

d. The condition of BF+Prompting didn't do any better than just BF or just prompting. So while fading and prompting have significant effects, these affects do not combine synergistically.

8. They conducted a similar experiment using the same computer-based tutor on a population of high-school students. They found the same results, and in fact, the effect on prompting on performance gains was larger in this case.

9. They argue that their approach only works in domains where there are clear principles underlying each step. This may not be true in ill-structured domains. For the latter, he suggests that prompting for goals and subgoals achieved by steps rather than principles. Refers to studies by Catrambone (1996, 1998) that show that this approach is effective.



Source:

[Atkinson et al., 2003] R, Atkinson, A. Renkl, M. Merrill, “Transitioning from Studying Examples to Solving Problems: Effects of Self-Explanation Prompts and Fading Worked-Out Steps,” Journal of Educational Psychology, v95 n4 p774-83 Dec 2003. Web: http://www.cs.pitt.edu/~chopin/references/tig/AtkinsonRenklM_03.pdf.

References to follow:


Anderson, J. R., Fincham, J. M., & Douglass, S. (1997). The role of
examples and rules in the acquisition of a cognitive skill. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 23, 932–
945.


Catrambone, R. (1996). Generalizing solution procedures learned from
examples. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 22, 1020 –1031.

Catrambone, R. (1998). The subgoal learning model: Creating better examples so that students can solve novel problems. Journal of Experimental Psychology: General, 127, 355–376.