The Ecology of Support

Print Friendly

Summary

Technology Story – Guiding Questions

Northeastern has been involved with ePortfolios for many years, and in that time the institution has experimented with a number of platforms, models for support, and approaches to professional learning. Based on this experience we believe that effective support and collaboration requires a holistic approach, attending to technical, pedagogical, and strategic concerns that are often co-mingled. As the Catalyst model suggests, success hinges on development across multiple sectors in the institution.

Our Platform

Helpful Northeastern Links

Digication tutorials
http://tinyurl.com/Northeastern-eP

Learning, Technology, & Evaluation Links

– SAMR Model: youtube.com/watch?v=_QOsz4AaZ2k
– TPACK Model: www.tpack.org
– Kirkpatrick Levels of Evaluation & Beyond Read Article

Northeastern has been involved with ePortfolios for many years. The University was one of the first institutions to experiment with Open Source Portfolio (OSP), but ran into challenges when customized code did not function properly with upgrades to that system. This is a cautionary tale about open source. The good news is that it can be tweaked to meet the needs of the institution and its use scenario. The bad news is that there is no corporate infrastructure of support to assist with the inevitable challenges associated with upgrades.

After OSP Northeastern transitioned to Taskstream, a system chosen primarily for its robust assessment capabilities. However, Taskstream proved difficult for faculty and students to use.

gail_reduced_white_eye_cropped_vert200Gail Matthews-DeNatale, Ph.D., Graduate School of Education at Northeastern University

In 2011 The College of Professional Studies contracted with Digication, including single sign-on authentication, because of its reputation for ease of use, a hosted format that allows students to retain their ePortfolios as alumni, and and ability to download ePortfolios. Authentication means that the username and password for Digication is the same as the login information for other university software such as email and the Learning Management System.

In  Fall 2012 the rest of the University transitioned from Taskstream to Digication. In the beginning this account did not authenticate with the University’s information system. In addition, it was not centrally funded and therefore programs were reluctant to formally integrate ePortfolios into the curriculum, given uncertainty about ongoing access. As of Fall 2013 funding has now been centralized, authentication is supported, and it is expected that program-level use will increase.

The University has two sites, one for the undergraduate day school and another for the College of Professional Studies (http://myneufolio.digication.com and http://northeastern.digication.com). At times this has proved challenging, because the authentication does not function properly if a student or faculty member tries to login at the wrong site portal.

In addition, the current installations of Digication are not integrated with the University’s system for enrollment. This makes it difficult to access the portfolios of students in a given course. That raises the bar for faculty who want to use portfolios on the course level, and for programs that want to access the portfolios of all active students. On the plus side, this emphasizes the point that the ePortfolio is separate from the learning management system — and that creates an opportunity to talk about the difference between “learning within courses” and “learning across courses.”

Digication Strengths: In addition to being easy to use, Digication is also more visually appealing than most of the other products on the market.  Students appreciate having the flexibility to customize the look of their ePortfolios, and those with more sophisticated technology skills enjoy being able to tweak the style sheets. Other valuable features include the capacity to have multiple portfolios, copy portfolios, and provide varying levels of access (private, viewable within the school, and viewable to the public).

Digication Weaknesses: Digication’s ease of use is both a blessing and a curse.  For example, when a student sets her ePortfolio to private and adds in a viewer, the ePortfolio link is displayed on the viewer’s homepage, alongside ePortfolios owned by the viewer.  There is no way for the viewer to folder, categorize, hide, or sort ePortfolios on the homepage.

The template tool is also limited.  For example, you cannot embed default permissions in a template so that, for example, ePortfolios created off of a template are all set to private, or all have a set of people with customized access.  The assessment tools for Digication are also not as robust as other tools on the market.

It would be helpful to have a more nuanced set of options for access, for example to make an ePortfolio only accessible to peers enrolled in a given course.

While it is great that Digication supports the embedding of some multimedia (e.g., YouTube), the options don’t keep pace with developments as rapidly as we would wish.  For example, as of fall 2012 Digication did not automatically support the embedding of VoiceThread by default.  Users need to write to Digication to request that capability.

Support and Collaboration

The following description pertains to faculty and student support within the College of Professional Studies. In addition to the description below, see the “Letting Learning Serve as the Driver,” a piece that describes the Honors Writing Program approach to faculty professional learning (http://neu.mcnrc.org/pd-story-and-practice).

Lessons Learned: The key to effective technology support and collaboration is understanding the concerns and contexts of the people who are being served. For example, effective support for students at a “commuter” school will differ from that at a residential college. The separate colleges within a large University often have unique needs, and this is true at Northeastern. Northeastern’s day school is largely residential and therefore face-to-face workshops are sufficient, but the College of Professional Studies hosts many fully-online programs that necessitate a robust, virtual strategy for the support and learning of administrators, faculty, and students.

Effective support and collaboration also require a holistic approach, attending to technical, pedagogical, and strategic support concerns that are often co-mingled. It’s a chicken-and-egg situation. As the Catalyst model suggests, success hinges on development across multiple sectors in the institution. Unfortunately, many novice institutions focus on technical “training,” overlooking equally important support for professional learning in the other two domains of pedagogy and strategy.

Which comes first, clarity regarding pedagogical strategy or conversance with how the technology functions? The need for professional learning is multi-dimensional:

  • technologists need to increase the sophistication of their understanding about curriculum, priorities for student learning, and assessment,
  • faculty and academic administrators need to carefully investigate the functionality (and pedagogical implications) of technology,
  • all participants need to cultivate characteristics associated with effective collaboration: patience, openness, humble curiosity, courage to venture into the unknown, proactive problem-solving, and tolerance for bumps and bruises.

This learning process is also often iterative, as evidenced by a recent CPS initiative to use VALUE rubrics in the assessment of student work in the context of fully online programs. Institutional researchers, assessment specialists, faculty, and instructional designers are debating whether data should be gathered through the Learning Management System, the ePortfolio system, or a combination of the two. This initiative requires a sophisticated understanding of the technical capabilities of each system, pros and cons of various use scenarios, and the long term implications of implementation strategy for bringing the initiative to scale and ensuring integrity of data.

Technical Support: At the College of Professional Studies, college-wide technical support for faculty takes place primarily through one-on-one consultation with instructional designers located in the academic technology unit. As part of the recent Master of Education Redesign initiative, a working group of academic technology instructional designers and faculty collaborated with Digication staff to develop a series of online tutorials that are primarily intended for students (see helpful link above and more on tutorial use below).

Program-wide initiatives, such as the M.Ed., are still grappling with the most effective way to provide opportunities for faculty professional learning and support. ePortfolio use is embedded throughout the curriculum and so faculty need to learn how the technology works,why it’s being implemented program-wide, and to have time to discern the relevance of portfolio pedagogy for their own teaching. “Training” that takes place in advance of teaching is of limited use, because it is decontextualized and the relevance is not immediately apparent. In addition, most people do not prefer to attend a separate technology training (in the same way that very few people read the manual before using a piece of software).

In contrast, “just in time” learning takes place closer to the point of relevance, for example at critical junctures during the term, but that can also be overwhelming because the faculty are in the midst of teaching. The program’s current strategy is to embed professional learning in gatherings such as faculty meetings, check-in regularly with individual faculty while they are teaching, and create opportunities for peer sharing (see below).

Pedagogical Support: Many of the faculty in CPS are remote, located throughout the country, and therefore professional development needs to be offered in a format that meets the needs of online participants. In fall 2013 the Graduate School of Education instituted bimonthly “Share and Cares.” These sessions are faculty-led, open to everyone in the college, and described as an “opportunity to discuss our practices and pedagogical passions (3Ps), with particular attention to the role that technology can play in fostering and improving learning.” They take place on site with a simultaneous presence in GoTo Meeting (web conferencing software).

Sometimes participants are asked to read materials in advance (recent examples include an article on the Community of Inquiry model for improving learning in online settings, a piece on Integrative Knowledge Portfolios from the International Journal of ePortfolio). The faculty members who are presenting usually open their session by sharing pedagogical conundrums and discoveries, using their own practice as case studies. This is followed by conversation among the virtual and on site participants, during which participants contribute examples from their own experience. In addition to the faculty member who is leading the gathering, a second person monitors GoTo Meeting to feed questions and comments back to the live group, and virtual attendees cycle in and out of the video feed so that everyone can put a face to a name. These strategies create continuity between the virtual and on ground attendees, and foster substantive conversation that extends beyond the presentation component of the session.

Strategic Support: In addition to technical and pedagogical support, a faculty member located in the Graduate School of Education who dedicates approximately 15% of her time to strategic consultation on the development of ePortfolio programs and initatives. This strategic consultation typically takes place in the context of special project work groups that include directors, managers, and teams of faculty. For example, a recent initiative to design workplace-based experiential learning opportunities that meet the needs of adults enrolled in fully-online programs.

Student Support: The Master of Education program in the College of Professional Studies recently launched a five credit, required gateway course entitled Education as an Advanced Field of Study that helps students

  • take stock of their motivations and perspectives on education (positionality)
  • learn how research contributes to understanding of controversies on education (and vice versa), and
  • become acclimated to the use of ePortfolios for documenting, reflecting upon, and integrating learning.

Education as an Advanced Field of Study:
Sample ePortfolio Assignments

Week 1 – Create portfolio based on program template, write homepage, document aspirations for course learning in the “My Program” area of the portfolio

Week 3 – Review program competencies, reflect upon vision for the future in the Goals page, incorporate early coursework into My Program area

Week 5 – Read about digital media literacies (including fair use and creative commons licensing), find and embed images within ePortfolio

Week 7 – Share ePortfolios, use Hallmarks of Excellence document as a framework for providing peer feedback

Week 11 – Embed signature assignment (e.g., final presentation) in My Program

Week 12 – Review work, peer feedback, etc. and write a reflection about learning and connections with other domains

Note: The program’s ePortfolio Resource site serves as a “text” for the assignments http://tinyurl.com/Northeastern-eP

The ePortfolio Resource site is used as a “text” within this course (see sample assignments). In addition to assignments focused on positionality and controversies research, students are given ePortfolio assignments every other week to instill the habit of regular use and to carefully scaffold their learning about portfolio pedagogy.

As mentioned above, Digication’s hosted platform encourages students’ sense of ownership.  In the gateway course they learn about granting and rescinding access, hiding items, and retaining access to ePortfolios after graduation.  It’s important to make this information explicit to the students, because it  establishes the ePortfolio as “theirs.”

Education as an Advanced Field of Study was implemented in Fall 2013, and therefore evaluation of the new system is in process, but preliminary results are promising. All students successfully created portfolios, with little to no need for additional support. In addition, there was a dramatic improvement in the caliber of writing and design, as contrasted with ePortfolios created by students in earlier cohorts before the required course came into effect.

The quality of students’ reflections is a work in progress, not significantly improved over previous years, and for this reason the revised course that goes live in Winter 2014 will distribute that work more evenly throughout the course, with multiple opportunities for feedback and modeling of that process.

Connections with the Catalyst

As mentioned earlier, the Catalyst describes a synergy that is essential for realizing the promise and potential of ePortfolios. It’s not about the technology, but technology selection and support can also make or break an ePortfolio initiative. It’s important to embed conversations about technology into all other dimensions of the catalyst (e.g., pedagogy, outcomes assessment, professional development, and scaling up). It’s also important for technology specialists to increase their awareness and sophistication of understanding of the other domains so that they can participate as collaborators and partners in the process.

Pedagogy

When positioned properly within a conversation about pedagogy, “ePortfolio as tool” can expand dialogue about teaching and learning.  For example, in explaining the difference between a learning management system and an ePortfolio, the conversation shifts from “learning within courses” (LMS) to “learning across courses” (eP).  Likewise, the capacity to embed mutiple forms of media can serve as a springboard for discussion about different ways of knowing, and linking leads to an exploration of connected learning. This is sometimes disorienting for people who haven’t used ePortfolios before, but if the dialogue stays focused on the pedagogical implications of the functionality it can open up new windows of insight.

Outcomes Assessment

Assessing and improving an institution’s sustem for technical, pedagogical, and strategic support is a complex challenge. On the one hand, it’s better to do do something than nothing (e.g., follow-up surveys with people who attended a workshop). However, this component of the Catalyst model is particularly complex because it involves increasing the capacity of the institution to

  • make wise decisions regarding instructional selection, design, and implementation
  • improve learning in ways that would not be possible without technology (e.g., SAMR level of “redefinition” and/or the TPACK model that describes the synergistic integration of technology, pedagogy, and content)
  • collect and interpret evidence about progress toward identified strategic goals (e.g., increased retention, student engagement, quality of work, digital media literacy, employment), and
  • anticipate and plan for future needs (e.g., scale, sustainability)

Assessment needs to take place on many levels (e.g., Kirkpatrick’s four levels that span the gamut of learner satisfaction, short term learning, and long-term results, and Kaufman and Keller’s fifth level of lasting contribution to society). Each level challenges us to consider a unique set of questions, and to gather different forms of evidence. For example,

Question Example of Evidence
Are the students engaged? Is the learning experience gratifying? Course evaluations, survey and focus group feedback, evidence of investment such as changing the default color settings in a portfolio, references to their portfolios in other contexts
Are the support materials and tutorials usable and accurate? Help Desk trouble tickets, follow-up questions posted in course discussion forums, incidence of error in performing key tasks such as submitting an ePortfolio for review
Is the use of ePortfolios transformative from a pedagogical perspective? Discussions and examples referenced by faculty during “share and care” gatherings, student portfolios that are substantive and integrate media in synergistic ways
Is the initiative improving learning? Correlation of portfolios with program competencies or national measures such as the AAC&U’s VALUE rubrics or Lumina’s DQP
Does the portfolio experience continue to be valuable for alumni after graduation? What is the long term impact of the program? Requests for ongoing access, follow-up surveys and focus groups, use of portfolios in presentations, publications, job applications, interviews and annual evaluations
Is the program financially viable, scalable, and sustainable? FTE of staff and/or peer mentors dedicated to the project, logs of actual staff hours, percentage of faculty time alloted to the project, cost of licensing

 

We are in the beginning stages of collecting and interpreting this evidence. The “What We’ve Learned” section of this site includes quotes from from surveys and evaluations. Earlier in this piece we reference early assessment of the strengths and opportunities for improvement in the ePortfolio component of the Education as an Advanced Field of Study course. The Master of Education program is slated to begin outcomes assessment of student work in Summer 2014. The will include faculty review of portfolios and work samples in relationship with program competencies. Finally, work is underway to assess student work in relationship to national instruments such as the VALUE rubrics, and to follow up with program graduates.

Additional Reading

The SAMR Model Explained by Ruben R. Puentedura – http://www.youtube.com/watch?v=_QOsz4AaZ2k

TPACK – http://www.tpack.org

Kaufman, Keller, and Watkins – What works and what doesn’t: Evaluation Beyond Kirkpatrick – http://home.gwu.edu/~rwatkins/articles/whatwork.PDF

Skip to toolbar