Teaching and Learning Research Framework

Teaching and Learning Research

As the desire to measure the effectiveness of blended learning at SUSS increases, it is apparent that there is a need for a comprehensive, cohesive and adaptable framework around which to frame teaching and learning research at SUSS. Such a framework will not only define the parameters of research, but will also determine the direction that TLC research should take, while maintaining a focus on the needs of SUSS.

A suitable research framework should engender studies that (1) provide knowledge and understanding of how learners learn and how instructors facilitate learning at SUSS; (2) interrogate and evaluate the appropriateness of the methods, modes and tools being used; and (3) identify gaps in our knowledge and focus on specific areas of interest that remain unexplored.

By mapping the explorations undertaken and keeping track of possible gaps and biases in its research efforts, TLC would be better positioned to plan a concerted and systematic set of research goals and projects to cater to both present and future needs. The ultimate outcome of such a systematic, framework-based approach is that all research will result in the provision of evidence-based strategies and methods for the improvement of instruction, course design, and the teaching and learning experience at SUSS.

In particular, TLC will:

  1. consolidate current knowledge of teaching and learning (T&L) at SUSS
  2. identify gaps in our current knowledge of T&L at SUSS
  3. focus its research on:
    • how students learn at SUSS
    • ways to improve student learning
    • responses (positive and negative) to new pedagogical initiatives
    • responses (positive and negative) to new technological initiatives
  4. provide recommendations on course(s) of action to take with regard to course design, instruction, and support for learners, using evidence-based research
  5. study students’ responses to and evaluate corrective actions taken so far, with a view to making further improvement

Research Framework

Our search for a possible research framework to structure TLC’s research efforts led us to Meyen et al.’s research framework for e-learning.

In 2006, Meyen et al. noted that despite the proliferation of online courses, a systematic approach to framing pedagogical research in online instruction, interface of technology and pedagogy, and definition of related constructs was noticeably absent at the time. The authors and fellow researchers at the University of Kansas thus set about “creating a research construct as a context both for structuring their own research and as a framework for stimulating research on e-learning among potential collaborators” (2006, 5).

Based on the research interests and research questions posed by their fellow researchers at University at Kansas, Meyen et al. further categorized these parameters into three main variables – outcome variables, input variables, and in-situ variables.

Outcome variables are all possible variables that may be changed, or affected, by a particular e-learning initiative, or changes to the learning experience. This includes, but is not confined only to, learner performance.

In-situ variables are conditions under which learning takes place. These variables may constrain, or enable, one’s learning experience. Examples include learners’ attributes – e.g., learner demographics or learner motivation – and technological infrastructure (e.g., hardware, software, and bandwidth).

Independent variables are “variables that can be manipulated for the purposes of research” (Meyen et al., 2006, p.16). Examples of these are instructional design, or learner interactions.

At SUSS, we have identified three dimensions associated with each variable (input, in-situ, output) that are present in SUSS’s teaching and learning landcape (please see Figure 2).

  • For input variables: teaching / learning interactions; integration of technology; components of course design.
  • For in-situ variables: learner / faculty and staff characteristics; enabling technology; enabling environments
  • For output variables: learner satisfaction; T&L efficacy; outcome implications

The figure below provides a systems approach to conceptalizing the T&L research at SUSS.

Figure 1: Proposed systems approach to conceptualizing T&L research at SUSS

Contextual Factors:

Inputs
Independent (Input) variables:
  • Teaching/learning interactions (e.g., learner-learner; -technology; -content; -instructor; COP; instructor-content)
  • Integration of technology (e.g., online discussion; eTextbook)
  • Components of course design (e.g., iStudyGuide; ePub; eTextbook)
Institutional actors and setting (structural, social, cultural, economic)
In-situ (Intervening) variables:
  • Learner / faculty characteristics (demographics; prior experience; academic qualifications)

  • Enabling technology (mobile device; desktop; laptop; online/offline access; software)

  • Enabling environments (supporting systems / infrastructure; broadband/wifi connection; social/cultural: online or f2f; economic: digital divide)

Outputs
Dependent (Outcome) variables:
  • Learner / faculty satisfaction (behavioral; emotional; cognition in adult learners)

  • T&L Efficacy (Teacher & Learner performance; academic/non-academic gains)

  • Outcome implications (policy; intervention programs; training for instructors; support programs

Feedback loop for reiterative improvement and refinement to input variables

The feedback loop linking the output variables to the input variables indicates that TLC’s research will likely result in evidence-based findings that provide the impetus for informed decisions on further improvements or refinments, or propose new initiatives associated with the initial input variables of components of course design, teaching/learning interactions and/or integration of technology.

We thus do not conceptualise TLC’s research – or indeed, any research on teaching and learning at SUSS – as a one-off, standalone venture; rather, research may also probe at whether iterative changes or revisions to the input variable – e.g., components of course design (such as the enhancement of the iStudyGuide) – did indeed result in, for example, the desired enhancement to our students’ learning experience.

The three identified dimensions for each of the input, in-situ, and output variables result thus in 27 cells (i.e., 3x3x3 cells) to be further explicated by T&L research (please see Figure 3).

Figure 2: Twenty-seven dimensions of TLC research

Independent Variables

In-situ Variables Teaching / Learning Interaction Components of course design Integration of technology Outcome Variables
Learner / Faculty characteristics       Learner / Faculty satisfaction
Enabling environments      
Enabling technology

2

   
Learner / Faculty characteristics  

3

  T&L efficacy
Enabling environments      
Enabling technology      
Learner / Faculty characteristics

1

    Outcome implications
Enabling environments      
Enabling technology      

For the next three years, TLC’s research efforts will focus on evaluating our students’ learning experiences with the iStudyGuide and eTextbook initiatives, and to recommend adjustments and refinements where necessary to the conceptualisation of those initiatives, provision of required student support, or instructional training (please see Appendix 1 for the list of proposed projects).

The following is an illustration of how research questions related to the iStudyGuide project may be mapped to the relevant dimensions of research framework (see pink cells in Figure 3):

Research Aims of iStudyGuide project:

  1. To determine the usage of iStudyGuides (Pink cell #1)
    1. How often do our students / instructors use iStudyGuides?
    2. To what extent do SUSS students / instructors use iStudyGuides?
    3. Where do the SUSS students / instructors use iStudyGuides?
    4. What devices are used to access iStudyGuides?
  2. To determine level of learner satisfaction with the iStudyGuides (Pink cell #2)
    1. What is SUSS students’ perceived satisfaction level with iStudyGuides?
    2. What do students like about the design features and elements (e.g., introductory video, activities, formative assessment, powerpoint slides, glossary) in the iStudyGuides?
    3. What do instructors like about the design features and elements in the iStudyGuides?
  3. To evaluate the T&L efficacy of iStudyGuides (Pink cell #3)
  4. To determine level of learner satisfaction with the iStudyGuides (Pink cell #2)
    1. How are students using the design features and elements (e.g., introductory video, activities, formative assessment, powerpoint slides, glossary) in the iStudyGuides in their learning?
    2. How effective is the iStudyGuides in helping students learn?
    3. How have these design features and elements enhanced student engagement with the course content?

As mentioned previously, the iStudyGuide project encompasses particular research strengths and interests of TLC faculty. We will be developing a quantitative survey instrument, which will capture baseline data for the iStudyGuide initiative. The data collected will be analyzed to provide for user and learner experience.

Additionally, TLC faculty will hold focus group discussions with both students and instructors, to elicit and document their personal reactions of the learning and user experience with the innovative tool (iStudyGuides). Findings from both the qualitative and quantitative approaches will be triangulated, to validate as well as to provide a multi-pronged, multi-dimensional evaluation of the effectiveness of the iStudyGuide on student learning in SUSS.

We intend to further refine the survey instrument developed in this pilot study and develop it into an innovation evaluation protocol to be administered in future research on innovative learning initiatives at SUSS. A baseline evaluation protocol would allow further comparative studies between innovative learning initiatives developed and implemented at SUSS.