Explaining Educational Data Analytics to Colleagues

For the purpose of the IEC Analytics Reconnoiter project I have been trying out ways of explaining the domain in ways that are readily understood by non-technical colleagues new to the ideas and staying clear of the different technical solutions that might need to be deployed. A key distinction here is that Educational Data Analytics takes a wider view than Learning Analytics using data about the the University as an enterprise.

The diagram below is based on two key dimensions that stand out for me based on readings, namely the time of use from collection dimension and a control and authority dimension (reminiscent to me of the Edinburgh Scenarios, Jonathan Star 2004). I think that a key point is to get across the potential in the top right hand quadrant and what it might mean for understanding the University as an enterprise (Educational Data Analytics ) beyond learning analytics and data mining that focus on pedagogical, learning and teaching aspects.

In addition, presenting the ideas as analytics for whom and for what purpose also appears useful:
learners: self reflection, cohort comparisons, automated learning, etc.
• teachers: retention, progression, student satisfaction/experience, etc.
• course developers: design for success – assessment, content, teaching strategies, etc.
• administrators: retention, progression, efficiencies, balancing HEFCE control numbers internally, etc.
• researchers: pedagogy, models, theory, etc.

On a more theoretical level the Viable System Model has potential application to better understand the central concept of feedback loops and how the different components of the systems in view interplay with each other and their external environments. For example, seeing that measures intended to address issues such as retention are inextricably wound up in the students wider environment where factors such as the need to work to fund studies may come into conflict with approaches like giving more contact time in the belief it will help students be successful. More work need on modeling this, but:

• real time feedback loop for learners & teachers – learning and retention are here and now issues;
• medium term for learners, administrators & teachers – universities plan, allocate resources & undertake quality processes in annual cycles. Progression is a medium term concern of academics and learners;
• long term for researchers & courses developers – the concern of theses groups tend to be more about aggregations of experience over a number of iterations.

An untidy thread that is niggling me and perhaps just needs more reading around is where Management Information Systems fit in all of this – they have been around for a very long time now and seem to occupy most of the Educational Data Analytics domain.

Enhancing Teaching and Learning through Educational Data Mining and Learning Analytics (observations)

I found the “Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics” (Draft report US Department of Education, 2012) publication an interesting explanation and introduction to ‘educational data mining’ and ‘learning analytics’.

My simple interpretation, is that Data Mining advocates seek to automate learning through extensive analysis of previous learners behaviours captured in learning systems. Whereas advocates of Learning analytics emphasise not prediction but on providing information (often visual) to inform stakeholders for decision-making purposes.

This report seeks to combine these two traditions in the concept of an ‘adaptive learning system’, the model below being self explanatory and at first glance convincing, but I think too tidy in practical terms.

Three deficiencies that stand out for me are:
1. the limited view of learning as shown by the exposure of a ‘Student’ to ‘Content’ as generating ‘student learning data’ – learning as process?
2. the narrow distinction of the learning system – courses (instructional) designers (not the same as administrators or teachers) and the curriculum as expressed by learning outcomes/aims, professional body requirements, etc. which are a significant part of the system;
3. and the fact that online courses seem to be the focus – there are lots of data to be analysed on f2f courses.

I think that the letter point is a more general observation of work on learning analytics.

As someone new to this domain of knowledge, two organisations appear to represent the different traditions, see the Society for Educational Data Mining and the Society for Learning Analytics Research – SoLAR. In a previous post, I conflated both of these under the Analytics badge, but now see more clearly a difference.

Generic ‘canvas generator’ tool for Archi!

As part of our work on the JISC funded Coeducate Project, Phil Beauvoir has developed a ‘blank canvas’ feature as an addition to the Archi modelling tool that offers open source software for Archimate modelling.

A significant generic application of the canvas tool ( fully editable and lockable / un-lockable making them very powerful and flexible) is in support of face-to-face workshops. On creation, a new ‘canvas’ can also be exported as a .png or .jpeg, enabling it to be printed out on large A0 sheets. These can then be provided for workshops participants to use with post-its in the normal way. However, a rapporteur can additionally use the tool to capture the post-its as they are created, thus providing a record that can be shared, built on in future, &/or handled by other applications.

Our motivation for developing the tool was our use of the Business Model Canvas (BMC) which as its name suggests is designed to support thinking around business models, something that we are not particularly good at when we develop new degree programmes in the UK Higher Education sector.

Rather than simply building a ‘hard wired’ representation of the BMC into Archi, we hope that we have added new dimension for people who wish to try and tie together different approaches and techniques with the practice of Enterprise Architecture using Archimate modelling language. An important point, is that this isn’t just a visual representation, but the tool (built in Eclipse) captures relationships between objects with associated name-value properties so that more can be done with the data in an automated way.

We think this is a cool bit of work and expect to see the commercial vendors like BizzDesign following suit:^) Phil’s work will ship with the next release of Archi in early December 2011.

An early example of the tool in use…

Cybernetics for clean toilets

Cybernetic explanations are based on a systems analytical approach and are concerned with feedback loops between a system and its environment. The Viable System Model developed by Stafford Beer had at its heart the notion of real time feedback loops that would enable an operation to make corrections to match the requirement of its environment.

Traveling to New Zealand, I came across these cybernetic devices at Singapore Chang Airport which I haven’t seen or noticed before. You can rate the service of the particular toilet attendant by selecting the appropriate button (hard to see but excellent on the left, poor on the right) so that he (and presumably his managers) have pretty much real time information on his performance.

These things are everywhere in the airport and it would be good to speak to the designer/s behind the system to find out what the thought process and inspiration was!

YouTube Captions & Subtitles: David Wiley Presentation

In sourcing material for our open learning project with Shaanxi Normal University, we found a useful resource on YouTube of David Wiley making a TED presentation. Using YouTube Cations & Subtitles for an initial transcription (most usefully it automates the time stamps for synchronisation with the video) followed up by a manual check, we have a resource that now has added value for helping students learn English as well as the subject material of the video.

Despite the subject of the presentation being about openess and the generosity required, I couldn’t see any creative commons licence so if asked to do so we would have to take it down.

Learning for the 21st Century Open Learning Project

A project we are currently working on with Shaanxi Normal University is taking a participative action research approach to developing a course with Masters level students working in the field of education technology. Although primarily using asynchronous learning approaches, we had our first synchronous session this week and this is just the start of us moving to a position where this is made available as an Open Learning course released under a creative commons licence. This is of particular interest to us as we seek to respond to the current economic climate by exploring new models & approaches for higher education as a part of the Universities Coeducate project.

Some of the research questions we have are:
– how to develop a sustainable business model that allows us to continue engage with learners from different Chinese universities at a cost that is not prohibitive to them
– how to allow anyone to participate in the course on a sliding scale from £0 – £cost of accreditation & support
– how to coordinate activities to retain the key element of synchronous conversational practice with proficient English speakers
– and what are the cultural, technological, pedagogical differences between Chinese teaching & learning and the UK.

Many people are well ahead with some of these questions and we are making the effort to learn from them, but at the end of the day what is possible is very context specific so we shall have to work out our own answers as well.

For this first iteration, we have a payment from our Chinese colleagues to cover the work we have undertaken in developing the course and for our facilitation of the online community & expert input into ‘hotseats’. In this particular case, we will also assess the students but credit awarded will be from the Chinese university not Bolton – lots of variations around teaching & assessment & accreditation are possible!

And of course no YouTube, but we do have the local equivalent youku…
Introductions from IEC staff