Measuring Digital Professional Development: Analytics for the Use of Web and Social Media

Journal of Applied Research in Higher EducationMy colleagues and I at NIU were recently notified of the publication of our article, Measuring digital professional development: Analytics for the use of web and social media.

Purpose

As faculty professional development increasingly occurs online and through social media, it becomes challenging to assess the quality of learning and effectiveness of programs and resources, yet it is important to evaluate such initiatives. The purpose of this paper is to explore how one faculty development center experimented with using analytics to answer questions about the use and effectiveness of its web and social media resources.

Design/methodology/approach

The case study was based on direct observation of the center’s practice and review of selected data generated by the analytic tools.

Findings

Unfortunately, while some analytics are available from a variety of sources, they are often distributed across tools and services. The center developed an analytics strategy to use data from Google Analytics and social media reporting tools to assess the use of online and social professional development resources. Initial results show that the center’s online and social professional development resources are widely used, both within and outside the university. However, more work is necessary to improve the strength and scope of the available analytics.

Practical implications

As a result of the analysis, the center has streamlined online resources, targeted social media use, and has begun developing methods to allow faculty to report online resource use as professional development for academic personnel purposes.

Originality/value

Many faculty development centers have not explored methods of evaluating online and social media resources. This paper outlines a strategic evaluation plan to measure the usage of online resources as well as engagement and interaction through social media.

Citation

Rhode, J., Richter, S., Gowen, P., & Krishnamurthi, M. (2015). Measuring digital professional development: Analytics for the use of web and social media. Journal of Applied Research in Higher Education, 7(1), 19-31.

2015 #et4online Conference Preview

2015 Emerging Technologies Symposium

Recently, I had the opportunity to join my colleagues from the Online Learning Consortium (formerly Sloan-C), MERLOT, and the Emerging Technologies steering committee to share a sneak peek of what lies ahead at #et4online. We hung out to share details about the upcoming conference being held April 22-24, 2015 in Dallas, Texas.

Who attends #et4online?

who-attends-et4online

image c/o @brocansky

6 Reasons Why You Should Join Us for #et4online

Or hear what the #et4online Steering Committee Members have to say in our 1/30 Google+ Hangout ON AIR (recorded):
Michelle Pacansky-Brock Conference Chair – @brocansky
Jason Rhode, Assistant Conference Chair – @jasonrhode
Jane Moore, MERLOT Program Chair – @janepmoore
Laura Pasquini, OLC Program Chair – @laurapasquini

Here are just a few of the MANY highlights for the #et4online program that we shared:

  • The Unconference – dig into topics and direct the agenda as you like it
  • Technology Test Kitchen – a maker space to explore, play & learn for ed tech
  • Keynote & Plenary Speakers – talks about connection to learning, networked identity, collaborative knowledge, and then some
  • Featured Sessions & Workshops – are just a few of the program items NOT to miss and learn from
  • Discovery Sessions with VoiceThread – to augment interaction and learning between presenters and attendees
  • The Launch Pad & Teacher Tank- Where #edtech start ups can show case their wares to our Ed Tech “sharks.” Submission Deadline Closes February, 13, 2015 – apply now!

Interested in attending (virtual or on site)? Register TODAY! Early bird pricing ends on February 25, 2015. Do you have questions about the conference or program? Leave a comment or hit me up on Twitter @jasonrhode

Beyond the Headlines: A Decision-Making Rubric for the Next Phase of Online Program Growth

As the number of online degrees have proliferated in recent years, it is even more important for institutions to take a more thoughtful approach to program selection. Often, a university must consider key issues like course offerings and schedules, degree specialization, and admissions requirements in order to offer a competitive online degree.

In this webinar offered 11/12/2014, academic leaders from the University of Alabama at Birmingham and Ohio University discussed the key market forces that are present today, and how universities can best position their prospective degrees in the market. Panelists provided practical insights and recommendations on how to handle the change management for a university to offer competitive degree programs.

The participants in this webinar learned about:

  • How to effectively plan for the online marketplace, validate program expansion, and reduce risk
  • How to influence change across the institution to prepare to take degrees online
  • Key considerations when evaluating online program management services

More details and a copy of the webinar slides, are available here.

NIU’s Customizations and Building Block for Blackboard Enterprise Surveys and Course Evaluations

Since the open discussion at BbWorld where NIU shared that we’ve been working on some system tweaks as well as a custom building block for providing an alternative reporting format for Bb enterprise surveys and course evaluations, we’ve conducted a very small pilot of the tool this summer and continued efforts on an initial draft of a building block that provides an alternate report format.

For those interested in learning more about our efforts, I recorded this screencast demo of the functionality at this point that I can share with those who would like to see the building block deployed within our dev environment. I tried to avoid as much technical-specifics as possible and rather just show the workflows and functionality tweaks that we’ve envisioned.

A decision still hasn’t been made institutionally whether we will continue to pursue a wider pilot of the tool with additional building block enhancements to be requested, or look at other 3rd party tools.

For follow-up on any additional technical details about the tweaks and building block depicted, please contact Ruperto Herrera (rherrera@niu.edu) and Matt Kacskos (mkacskos1@niu.edu) who have done all the technical development work.

2014 Inside Higher Ed Survey of Faculty Attitudes on Technology

Inside Higher Ed’s third annual survey of college and university faculty members and campus leaders in educational technology aims to understand how these groups perceive and practice online learning and other emerging opportunities for delivering course content.
Faculty Attitudes on Technology
Some of the questions addressed in the study are:

  • Can online courses achieve learning outcomes that are equivalent to in- person courses?
  • What are the most important quality indicators of an online education?
  • How does the quality of online courses compare with the quality of in- person courses?
  • To what extent have faculty members and technology administrators experienced online learning themselves, as students?
  • To what extent have faculty taught online, hybrid, and face-to-face courses? For those who have not taught online, why is that?
  • How supportive are institutions of online learning?
  • Which should cost the student more — online degree programs or those delivered face to face?
  • Who should be responsible for creating and marketing online degree programs?
  • Are institutions expanding online learning? Should they do so? To what extent do faculty feel that they are appropriately consulted in this decision- making process?
  • How do faculty use learning management systems (LMS) and early warning systems?

Snapshot of Findings

  • Few faculty members (9 percent) strongly agree that online courses can achieve student learning outcomes that are at least equivalent to those of in- person courses. Academic technology administrators are more likely (36 percent) to strongly agree with this statement.
  • Asked to rate the importance of factors reflecting quality in online education, faculty members and academic technology administrators alike say it is “very important” that an online course or program “provides meaningful interaction between students and instructors” (80 percent for faculty, 89 percent for administrators), “is offered by an accredited institution” (76 vs. 84 percent), “has been independently certified for quality” (66 vs. 52 percent), and “leads to academic credit” (50 vs. 68 percent).
  • While a larger proportion of technology officers than faculty members say online courses are of better quality than in-person courses in a set of eight areas, in neither group did any of the eight areas garner a majority reporting this view. But faculty members thought online courses could be at least as good as in-person
    during class and 77 percent of faculty say the same about the ability to reach “at-risk” students.
  • Very few faculty members (7 percent) believe the tuition for online courses should be higher than for face-to-face degree programs. A much smaller proportion of faculty who have taught online courses believe online courses should have a lower tuition than face- to-face programs (20 percent), while nearly half of their peers who have never taught an online course (48 percent) believe that this should be the case.
  • More technology administrators (53 percent) than faculty members (32 percent) have taken an online course for credit. Nearly half of those who have taught an online course (49 percent) have also taken an online course as a student, compared to less than a quarter (23 percent) of those who have never taught an online course.
  • About one in three professors say they have taught an online course, with some variation across position type. Among those who have never taught an online course, the three main reasons they give are never having been asked, not being interested, and not believing that online classes have educational value.
  • More than 8 in 10 instructors say they have converted a face-to-face course to a hybrid course. The majority report that this conversion decreased face-to-face time.
  • Half (51 percent) of faculty believe improving the educational experience for students by introducing more active learning in the course is a very important reason for converting face-to-face courses to blended or hybrid courses.
  • Nearly three-quarters of faculty believe that professors own the online course content and material they create.
  • Less than half of faculty and technology administrators strongly agree that their institution offer instructors strong support for online learning, as measured by eight indicators.
  • Nearly all professors (96 percent) agree that institutions should produce their own online degree programs and be responsible for marketing them (85 percent).
  • About one-third of faculty strongly agree that their institution is planning to expand online course offerings, though only about one-sixth strongly agree that their institution should do so. A larger proportion of those who have taught an online course than their peers who have never done so strongly agree to the above two statements. Most faculty do not feel that they have been appropriately involved with decision making surrounding the expansion of online course offerings.
  • A small fraction of faculty believe that spending on IT infrastructure (8 percent) and digital initiatives (7 percent) is too high. Faculty are split on whether spending in these areas are too low or just about right.
  • The majority of faculty always use learning management systems (LMS) to share syllabus information with students (78 percent), record grades (58 percent), and communicate with students (52 percent). Only 20 percent of faculty members always use the LMS for lecture capture.
  • Only 15 percent of faculty strongly agree that digital humanities has improved their teaching, 14 percent strongly agree that it has improved their institution, and 23 percent strongly agree that digital humanities has improved their research.
  • The vast majority of faculty (89 percent) say their institution uses an early warning system, and 81 percent believe that those warning systems help students make significant learning gains.

The full report can be downloaded here.