OpenEd User Perspective - My survey results

Thanks for all of you who responded to my request to fill out a short questionnaire! In this blog I will summarize and discuss the results.

Research question
In a recent post I vented my frustration with the limited interaction in the Intro to Openness in Education course. Realising that there are different ways in which people participate in this course--e.g. on/off campus, grad students, professionals--I wondered if there is a difference in perception about factors such as interaction.

Hypothesis
Given some of the comments I had read in some blogs about class discussions, I would even go so far to assume that on campus participants in the course would be more positive about the interaction than off campus participants.

Method
Not wanting to reinvent the wheel (or better: knowing how difficult it is to develop a good instrument) I decided to see if one had been developed before. I selected one by Ward, Peters and Shelley (2010). They asked participants rate dimensions of effective instruction for different course formats. These dimension were used again and subjects were asked to rate them from low to high as to how they related to IOE12.

Three questions were added about location (on/off campus), intentions (university credit, badges, etc.) and a field for comments on interaction in the course.

Subjects
Participants in the course were targeted for the questionnaire. This was announced through my personal course blog, through twitter using the course #-tag and to individual participants where possible through email, blog comments or direct tweets.

Results
Forms in google documents produces a summary which includes a link to all data. In five days after the announcement of this short survey, eleven fellow participants filled out the questionnaire; two on campus and nine off campus participant, with a variety of intentions.

For further analysis, one submission was excluded. The participant had selected all 1's, stating "[..] I was very interested in the course but work overload made me difficult to follow it properly. [..]"

The extent to which the dimensions of effective instruction relate to the course is shown in Table 1 and graphic 1. 

Graphic 1 - Dimensions means and standard deviation. 

Table 1. Dimensions of instructional effectiveness. 

The participants score Intro to Openness in Education high on dimensions quality and amount of content, encouraging active learning, respecting diversity, ease of access, and minimising costs. The other dimensions score considerably lower. But there are big differences on opines in the individual scores. 

To examine differences between on and off campus participants, means for respective groups are shown in Graphic 2. It appears that the on campus participant score all dimensions considerably higher than off campus participant. Because of the small number of subjects, it is impossible to establish any statistical significance at this moment. 

Graphic 02. Dimension means per group. 

Comments about the interaction were mainly focussed on two areas; one technological and one about human interaction. The 'blog broadcasting' on the course page was mentioned a couple of time, as being a hinderance, for example "the DS106 posts were overwhelming and diluted the actual content of the course." In general participants commenting on the human contact had expected more interaction: "I would have loved to build more meaningful relationships with other participants, but it just didn't happen."

Discussion
The survey discussed in this blog originated from a feeling of lack of interaction between all actors in the course Introduction to Openness in Education 2012, combined with an interest in the user perspective on the different topics in the course (anyone else considering completing the user perspective badge? yes? ;-)) I assumed there might be similar frustrations in other participants, and suspected it might be different for on-campus students, who could talk about the topics at least once a week in class ( I assume; correct me if I'm wrong.)

To be honest, I feel the dimensions of effective instruction are quite a shaky part of this survey. Rating them and how they relate to the course ... low to high ... does anyone really know what that means? I assume you are all expert survey answerers, and went along nicely. Thanks for humouring me! One could question though if people had the same ideas when doing the rating. It would be good to develop a better instrument when repeating this with a larger audience.

Obviously the small sample size is the other reason why results need to be taken with a pinch of salt. OK and the fact that it was quite the convenience sample perhaps. The results could be used in a more qualitative study perhaps, when a survey would be complemented with interviews and or other data.

I do think the difference in responses between on and off campus participants is striking and could warrant a more serious study!

Thanks again for your interest and support!

Cheers,
Jeroen


References
Ward, Peters, and Shelley. 2010. Student and faculty perceptions of the quality of online learning experiences. The international review of research in open and distance learning, 11 (3).

Open Teaching

David Wiley in a keynote on Open Education at Penn State University states that the idea that one can do the same things online as one can do in the classroom is ludacris.


Things are changing rapidly. From analogue to digital, from tethered to mobile, isolated to connected, generic to personal, from consuming media to designing media, and from closed to open. David argues that education falls behind in these changes.


Can education be more personal, just-in-time? Not just online, can it be for face-to-face education?
Research shows that students who listen to a pod-cast of a lecture perform better than those who attend the lecture. They can rewind, and speed up the lecture.


Why should education respond? The monopoly of (higher) ed is being challenged. The access to content is not limited to schools anymore. Asking support can be done online. Social life has expanded to the online realm. What about degrees? Perhaps that is the reason why students have to come to school. No, certain vocational credentials compete equal or better to college degrees. Everything universes provide, is being provided by someone else.


What about e-learning? Used to be advanced in 1995. Yes it is digital and mobile, but still old-fashioned on the other change factors mentioned above. Openness bridges the gap. Why? It facilitates access to content. Open licenses allows for personalisation. Sites like YouTube provide an outlet for creativity.


How can higher ed go about opening up then? Open 1.0: Currently some universities share open courseware. They are not very sustainable. It costs a lot.

The focus was on 'them'--sharing something with the world--with some benefits for the universities. Open 2.0 will focus on the 'us' first. And because they are open, there will be benefits for people out there.

Note: so that is how I benefit from this course!? 

Examples how to insert openness in the classroom (with mixed results):
- Public blogs instead of handing in assignments; success
- Syllabus in a wiki, open to change; none ever did it.

Note: Jonassen often would walk into the classroom, sitting down on the table and say 'so, what do you want to learn today?' While we joked about it over coffee, none ever stood up to suggest a topic or a lesson outline. 

- 'Sitcom script' in a wiki, to substitute for real time discussion from different perspectives; Students inserted characters.

Unintended things are often the most interesting.


What can we do to be even more open?
Course in Openness in education, in a classroom, and all materials and blogs openly available; students from other institutions registered as independent studies in their own schools and participated. And it was opened up to the world. 60 participants in total. While it seemed a daunting task, students start to self manage and respond to each others. The international perspectives were considered a benefit in this experience.

Note: Somehow I feel the interaction is lacking so far in the current version of a similar experiment. 

The foreigners asked what completing meant for them ... no credit from the university, but they got a certificate. People appreciate that, and some even list it in their resume.

How to create more student support without putting too much strain on the the professor? Wiley redesigned the course in the form of a Massive Open Online Game/Course. Different characters have different quests to develop expertise. The quests get more and more complicated, so that students must work together. For example students developed their own additional character.

What is in the future? Disaggregation (separating [something] in it's component parts).
New model university, for example Western Governors University uses a competency based model. They do not offer courses, only assessments. They do not care where you learned it; they don't care how you learned it. When you feel ready to take a test, you can come and do the test. Credentialing piece is pulled away from the content.

So, what is the value of keeping these things integrated?

In a blog in The Chronicle of Higher Education David Wiley defines open teaching as "freely allowing people outside the university to view course materials and informally participate in the course" and appeals to professors moral obligation to make their teaching efforts as broadly impactful as possible. 

MOOC
The second part of the Open Teaching topic discusses Massive Open Online Courses aka MOOCs. 
Wikipedia defines them as: 

"[..] a course where the participants are distributed and course materials also are dispersed across the web. This is possible only if the course is open, and works significantly better if the course is large. The course is not a gathering, but rather a way of connecting distributed instructors and learners across a common topic or field of discourse."

A variety of examples of MOOCs and their predecessors are listed in the Dowes' MOOC Guide. It tries to show and a historic overview of MOOCs so far and elicit major design elements from them. Which is not exactly coming out yet. 


Note: perhaps for someone to analyse as part of their OpenEd Researcher badge...?

In their 2010 article, The MOOC Model for Digital Practice, McAuley, Stewart, Siemens, and Cormier study the opportunities of MOOCs to develop the digital savvy citizens Canada will need to flourish in the digital age. They describe what a MOOC is, how it could model digital practice, how knowledge is created and how it might contribute to a digital economy. 

Question: How are these MOOCs different from previous Communities Of Practice?