Justifying my methodology

What's the relationship of realism to positivism and interpretivism?

thus asked my mock examiner. So I went back to reread Tsoukas (1994) on realist perspectives and have again sunk in the mire of methodological approaches, perspectives, philosophies and epistemologies.

This web site has a nice diagram of the overlap between the three approaches or perspectives or ontologies.

• Realism overlaps positivism in that it accepts social structures have independent existences. "Like positivism, realism accepts social structures have some form of independent existence that is experienced as ‘external' to individuals. These structures act upon us - pressurising and constraining our behaviour."
• Realism overlaps interpretivism in that what we regard as real is significant. "Like interpretivism, realism accepts that what we regard as real is highly significant. E.g. if I believe myself to be middle class, while every indicator of social class holds I am working class, this will have important consequences for my behaviour."

For my research on engagement,
social phenomena exist whether or not people are aware of them so engagement can be taken as a real phenomenon even if people are not aware of it. It still exists and affects their actions.

Critical realism is an epistemological (not a research or sociological) perspective that responds to critisism of the positivist perspective. "It's a response", says Mingers, "to the difficulty of maintaining a realist position in the face of criticisms" (page 380). My erudite supervisor tells me that

'the "critical" label comes from the associated idea that such "real" phenomena can however only be known through using conceptual frameworks, so our knowledge of these real phenomena is always provisional, or subject to challenge when new concepts come along. So knowledge of this real social world has to be held "critically".'

Critical realism is "a way of resolving or dissolving" issues around positivism and extreme constructivist positions {Mingers, 2004: 374}.

---

Mingers, J. 2004. Real-Izing Information Systems: Critical Realism as an Underpinning Philosophy for Information Systems. Information and Organization, 14(2): 87-103.
Tsoukas, H. 1994. What Is Management? An Outline of a Metatheory. British Journal of Management, 5(4): 289.

Not sociologically critical

Supervisor #2 reminds me about the Masters in Research Methods, course B852, where we studied a discussion of critical realism by Ackroyd and Fleetwood. I don't remember, then I realise that they published that difficult article on a metatheory of management by Tsoukas. My notes on it indicate how difficult I found it then.

Apparently I'm taking a critical view of the social world, not being sociologically critical and the "critical" refers to an epistemological stance rather than a theoretical or political stance. So that's all right then. I've changed my methodology chapter paragraph to read:

However, this research takes a realist perspective because, although IS projects take place in real objective spaces, each individual participant constructs themselves in a particular way in relation to the project setting. An advantage of such an approach is that “it maintains reality whilst still recognizing the inherent meaningfulness of social interaction" {Mingers, 2004:99}. Hence, engagement can be explored as a real phenomenon, whilst still recognising that IT project participants may be unaware of the phenomenon.

So I think I've kept what I said before, and now managed to justify it too.

---

Tsoukas, H. (1994) 'What Is Management? An Outline of a Metatheory', British Journal of Management, 5 (4), pp. 289.

Critical realism pondered

I think the term 'critical' is about responding to criticisms. In IS philosophy, according to John Mingers (2004), critical realism seems to be a response to criticisms of an empirical and naturalist view of science. Mingers says that the original aims of critical realism were

"(i) to re-establish a realist view of being in the ontological domain whilst accepting the relativism of knowledge as socially and historically conditioned in the epistemological domain; and
(ii) to argue for a critical naturalism in social science."

In a context where research has been of machines, of software, of hard systems with less of a focus on people, I can see how there might be a realist approach that was closer to positivism, and I'm finding it a fine line to distinguish between positivism and realism. I had the impression that critical realism was a step further away from positivism, and a compromise between positivist and constructionist approaches. I find support in Mingers' statement

"The major advantage of a critical realist approach is that it maintains reality whilst still recognizing the inherent meaningfulness of social interaction."

I like the idea that critical realists "want to get beneath the surface to understand and explain why things are as they are, to hypothesise the structures and mechanisms that shape observable events. "(p100) because that's the way I thought I was working, so I thought I was using a critical realist approach.

---

Mingers, J. 2004. Real-Izing Information Systems: Critical Realism as an Underpinning Philosophy for Information Systems. Information and Organization, 14(2): 87-103. at http://www.sciencedirect.com/science/article/B6W7M-4C47BCF-4/2/873a061fe87e6979e71a48c3fe922ddd

Critical realist or realist?

Having pondered my methodological approach, supervisors are happier with this second attempt at the chapter. I've written:

A reason this research should use a constructivist approach is that project participants construct each other’s behaviours in the context of a project and shape each other’s understanding. Examples of this approach discussed in the literature review include Schein (Schein, 1997), Bovens’ (Bovens, 2007) constructions of client models, Wenger’s communities of practice (Wenger, 1998), the importance of space and time for understanding projects (Maaninen-Olsson & Müllern, 2009). However, this research takes a critical realist research perspective because, although organisations’ IS projects take place in real objective spaces, each individual participant constructs themselves in a particular way in relation to the project setting.

Supervisor #1 says that I should say it's a realist perspective, not a critical realist perspective. I'm not sure about this because I thought there was a difference that mattered, and isn't realism close to positivism? Or does the term 'critical' apply when research seeks to identify practices or structures in order to highlight inequalities or injustices, in which case, I should remove it, because I'm not attempting to highlight such inequalities. If I remove the word, then everything else I've written still holds, but if I leave it in, then I'm open to queries about my perspective.

I'll ponder.

Qualitative resources

Here is a a whole list of resources for a research student to use to get to grips with methodological approaches. The list includes a link to a video of Alan Bryman talking about conducting mixed methods research; it has helped me get a better idea of how people use the word 'paradigm' in the context of methodology.

I've now made the arguments that I want to make in my methodology chapter, though I daresay my supervisors will have plenty of critical comments to make. They can only be helpful to sorting out my confusion of terms. I shall submit a full draft of the thesis to them in a few hours, and then must wait three weeks to get their feedback.

But hey! a full draft!

Ridiculous confusion

All I have to write is a couple of paragraphs about different ends of the belief spectrum of enquiry (ye wha'?!). Belief system - how you believe reality. It's about what you believe is real, what your assumptions are. It seems reasonable to surface those assumptions, because they influence what you believe you know, and therefore how you go about researching, and I need to explain why I researched it the way I researched it, to justify it.

Except, the belief spectrum of enquiry is full of paradigms, perspectives and isms.

• Positivism
• Realism
• Constructivism

And on top of those there's post-positivism (don't ask me if there's a negativism), critical realism and constructionism. I don't know the difference between constructivism and constructionism. I had thought my approach was constructionist, but now I'm confused and it might be that my ontology is constructionist and my methodology hermeneutically dialectic, though I thought this research was taking a critical realist research perspective because, although the organisation’s IS projects took place in real objective spaces, each individual participant constructed themselves in a particular way in relation to the project or programme setting. Which is they constructed themselves, so aren’t I working to a constructionist philosophy?
Now I have all these different terms to work with:

• approach
• ontology
• methodology
  
• perspective
• philosophy

This bit of my methodology chapter is never going to get writ.

---

Guba, E. G., & Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions and emerging confluences. In N. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3 ed., Vol. 3). Thousand Oaks ;London: Sage Publications.

Philosophical stance

Like a horse at a big jump, I'm stalled. For over a day, I have not written the ten paragraphs I want to write about my research philosophical stance, in my methodology chapter but have considered

• realism
• constructivism (what's the difference between constructionism and constructivism?)
• positivism

I had argued myself into believing I was taking a constructionist approach because I was seeing my informants as constructing themselves in the context of an IT project, that they constructed their relationships and how they engaged with each other. But now I think I'm taking a critical realist research perspective because although the organisation's IS project development took place in real objective space, each individual participant constructed them selves in a particular way in relations to the project or programme setting. That is critical realist, isn't it?

Editing and writing and editing

The second draft of the methodology chapter is under way as I restructure it because it was not meeting supervisor's approval. Supervisor #1 wants something written on other approaches as well as the one I've chosen - but so many academics have already writtenon these, and much better than me so why write an essay on comparing research methods?

I'm off to compare other PhD theses methods and methodology chapters.

As I research qualitative approaches, I discover that 'quality' is next to 'qualm' in the dictionary. Quality ' the degree of excellence' is next to qualm, a 'momentary faint or sick feeling'. I have qualms about my writing, about the excellence in my writing, about the logic of my arguments. I don't feel good.

Positive bias

The type of case studies I have show a positive bias. People and organisations that are interested enough to agree to be case studies tend to engage with each other on their IT projects, and to perceive their projects and their relationships with their consultants and suppliers as successful. Those that don't engage, or don't perceive their projects or consultant-client relationships as successful, have not participated.

Case study approach

I have use case studies because:

• a case study approach "allows the investigator to take advantage of emergent themes and unique case features"{Eisenhardt, 1989 }
• a case helps understand the different views of people working on the same project, their "claims, concerns and issues" {Guba, 1989 : p184}. The context is the constant and the participants are the variables.
• a case study allows organisation of data around the themes of IT and the public sector

---

EISENHARDT, K. M. (1989) Building Theories from Case Study Research. Academy of Management Review, 14, 532-550.
GUBA, E. G. & LINCOLN, Y. S. (1989) Fourth Generation Evaluation, Newbury Park, CA; London, Sage.

Representative qualitative research

I was thinking about how to judge the goodness of my qualitative research - see chapter 10 of Miles and Huberman.

Are my informants representative? They are to some extent self-selected in that their organisation's management accepted my approach asking to research them. The management then suggested which IT project or programme to study. In that sense, the informants didn't volunteer, though the organisation did. So informants and organisations may not be representative.

But representative of what? They are representative case studies of successful IT projects, not failures. That's clear because:

1. the organisations made the cases available - I'd assume they'd not make failures so available
2. several of the cases have been submitted for awards
3. the pride is in the words and terms that the informants use

Informants tended to be articulate and elite in management, but outliers were available for interviews at very short notice, such as a clerk, a techie chap and a data input person. So does that mean I've got something representative - both management elite and operational grade informants?

---

Miles & Huberman in Miles, M. B. and A. M. Huberman (1994). Qualitative Data Analysis : An Expanded Sourcebook. Thousand Oaks, Sage Publications.

Generalisation

On using case studies in my methodology chapter I'll have to comment on generalisation. Mason {2002} says it's not easy, that you have to know what your argument is and its relationship to the production of theory - and that's what I'm struggling with - relating real practical engagement to the engagement literature and social capital theory. I have to support each claim I make with the relevant linking material, Mason says.

The points to make must include:

• Qualitative research must produce explanations that have wide resonance and are generalisable even though case studies provide a compact unit of research {Payne, 2005 }
  
• Case studies should be interesting
• Case studies should maximise what can be learnt from that particular case
• Acknowledgement that limited generalisation is possible or appropriate to qualitative research

I need to find how Stake distinguishes between types of case study. From the web, I gather it's:

1. The intrinsic case study where ‘ this case is of interest… in all its particularity and ordinariness’ , ‘let the case reveal its story’
2. The instrumental case study in which a case is examined mainly to provide insight into an issue or for refinement of a theory.
3. The collective case study – number of cases studied in order to investigate some general phenomenon.

I want instrumental case studies to gain insight to the phenomenon of engagement. Collective case studies would be nice if I could get them. I'm not using intrinsic case studies, so don't need to go into details of what the project or programmes are - and that helps to keep confidentiality and anonymity.

---

Mason, J. (2002). Qualitative researching. London, Sage.
Payne, G. and M. Williams (2005). "Generalization in Qualitative Research." Sociology 39(2): 295-314.
Stake, R. E. (1995) The Art of case study research, Sage. or in
Stake, R. (1994) Case studies, in Denzin, N. and Lincoln, Y.S. Handbook of Qualitative Research, London, Sage

Methodology

Different dissertations have different ways of tackling the methodology chapter. Cranfield dissertations have definite chapters that cover the why, the philosophy, the forms of knowing. Such chapters may or may not include the method, the how the question was researched, as well as the why the question was researched this way. I suspect other countries have other ways of approaching this chapter. I have a sample of only one - a dissertation from Denmark, where the methodology chapter is written in quite a different way.

The methodology for a qualitative approach has to be as rigorously made as for a quantitative approach.

What I'd like to find are other dissertations from the UK, preferably qualitative dissertations so I can how the argument is made for the research approach.

Writing overload

I may have a case study on paper, but it's only a draft, and it's still so so mixed up .

But I've more to write. For example,

• Supervisor #1 wants to be reminded of my framework.
• Supervisors want me to rewrite those parts that were "completely unclear"
  
• I've got some ideas to write down about control, risk and behaviour being involved with engagement.
• I still haven't written anything about methodology for the supervisors - I wonder if they haven't asked yet because they've been more concerned about collecting data.

Third year stress looms!

Interviewing

There is so much on interviewing that Kvale has written a whole book on it. Although it might have been useful to read earlier, reading it now with a little more experience of research interviewing means that it makes more sense to me. For example, chapter 7 has a table that matches research questions to interview questions. I have problems matching mine because I've let the social capital framework interfere. The social capital framework guides the interview questions. What I want is to relate the research questions and the social capital framework. So my interview structure is based round:

1. background (to project and to participant)
2. relationships
3. knowledge (or learning)
4. value

The background is important as a start to make sure I've got the context. The questions may sometimes elicit structural dimensions of social capital.
Relationships matter. That is the crux of my interest- how do the relationships create value? So the interview questions here concern social capital in the relationship and structural dimensions. But they don't elicit information about knowledge. I ask what's helped and hindered relationships. And specifically what challenges to relationships have there been and how have they been overcome. Another interview question address specificity by asking for an anecdote or story. the question sometimes falls on blank faces. perhaps I should reword it.
Knowledge I ask about learning - what have people learned from each other? And how do they use that knowledge. I can't see how these questions address engagement though.
Value Sometimes in asking about learning I get an answer that suggests qualitative value in the learning process through building shared meanings, which is part of the cognitive dimension of social capital, and probably the most valuable non-financial and immeasurable gain from the relationship. But the valuable relationships need not be client and consultant, but developer and user, perhaps, mediated by the outsider, i.e. the consultant. So the value is a value chain! A valuable learning comes from a valuable relationships mediated and catalyzed by a consultant intervention. I've drawn a diagram of it, starting from the left with the people creating the relationship and moving to the right where the outputs are learning and a new IT system.

So I've got somewhere in my thoughts but not arrived at what I set out to do - depict in a table the relationship between my research questions and my interview questions.

The diagram must also include the value added impact on the IT system being developed. It's the human element, the soft side that matters. Engagement is soft. Perhaps I should draw rich pictures for each case study, following Checkland's soft systems methodology.

And I don't have many why interview questions - it's my job (Kvale, p131) but one I did use once was quite revealing, when someone said "It's changed", and I asked "Why do you think it's changed?" The participant wondered if they were doing something different, and it was something do with using consultants.

---

Kvale, 1996, InterViews: introduction to qualitative research interviewing

Unit of analysis

What's your unit of analysis?

That was the coffee time question posed by our Resident Questioning Student. He often starts interesting discussions. I'd naïvely thought it a simple answer, but the more he posed questions, the more complicated the answer became.

The unit of analysis fits in with and stems from your research question. It governs the way you then design your research because it governs what kind of information you seek. That information builds up to give a central picture. I have a systems understanding of the unit of analysis; to me a unit of analysis must have a central focus, a purpose that all its components contribute to, and a unit of analysis consists of a number of components, any one of which, if it were removed, would create a different system. For example, RQS's unit of analysis concerns a process (a campaign) that takes input and outputs x or y, and along the way, there are regulatory processes, decision making processes and ethical processes that influence the outcome. If you took away the regulatory process, you'd have a different system (like the UK MPs could have a different expenses system if their regulatory process was different).

The B852 course that we did for the MRes gave us some useful readings on unit of analysis, especially the Block I papers by Michael Crotty on the research process, and the Bryman & Bell paper on research designs.

RQS is writing his methodology chapter, and warns us that it is pivotal to all your research, which is probably why my third party monitor suggested I start writing it now, this year.

PhD theses I've read

Pugh says in order to find how to write a thesis, a student should aim to read two or three theses a year. So far I've read:

• Diana White on a systems view of project management for how she used systems theory, and because her supervisor told me it was a good thesis.
• Kirstie Ball on surveillance for her structure because she wrote on several case studies as I do (or hope to have)
  
• Lupson on accountability because his PhD was filling a gap in the literature on accountability in the public sector
• Someone on consultants, which was a pain to read because I could only get a microfiche copy
• Marsh on the Feminine in Consulting because she wrote on consultants. But in fact it's not her PhD I've read, only the contents, because I'd have to go to Lancaster to see the original in the library; it's not available electronically. pooh! But she's written a book from her thesis and it's very readable. so I got the OU library to get a copy and that's what I'm reading.

Methodology chapter

Walsham (p166) refers to Guba and Lincoln's fourth generation evaluation. It's worth using this evaluation idea in the methodology chapter. Their proposed approach is "characterized as measurement oriented, description oriented and judgment oriented." Then Walsham says the key to this approach "the key dynamic" - what's a dynamic? - " is negotiation". He states that the approach is interpretive. What's the difference between an interpretive approach and the constructionist perspective that Guba and Lincoln think it to be?

"they take a position that evaluation outcomes are not description what things are or really work, but instead represent meaningful construction that aid an individual to make sense of situations."

Walsham also states that actors value shaping sense making and that power relationships are also important.

These ideas all feed into a practical methodology.

See Guba and Lincoln's twelve steps (p185) for how to use them to rationalise or justify an approach.

---

GUBA, E. G. & LINCOLN, Y. S. (1989) Fourth Generation Evaluation, Sage.
WALSHAM, G. (1995) Interpretive case studies in IS research: nature and method. European Journal of Information Systems, 4, 74-81.

Work plan for research project

It's a bit hard to plan two years ahead, but since I don't need to worry about coordinating financial and people resources, it's not a great problem. So here's first draft starting July 2008, with thanks to my student colleagues who shared their approaches.

Literature review
Month Activity
1- 27 Continue to read and review literature on public sector, consultants, clients and relevant method such as social network theory

Skills audit
Month Activity
1 - 22 Attend courses, seminars & conferences as appropriate
Rewrite my OUBS web page when it’s out of date
2-4 Learn any specific analysis skills such as the maths needed for social network analysis
6-10 Learn necessary software such as NVivo and software for social network analysis
1-28 Adjust plans as life gets in the way
9-10 Write a paper on the pilot case study results

Methodology
Month Activity
1-4 Review and decide methodology & methods
Choose options for analysis
4 Write something about the design, method and methods
4 Get ethics approval (skill B2, B3

Data collection
Month Activity
1-4 Identify 5 or 6 case studies, one of which can be a pilot study. Five might suffice, but six will cover contingencies such as lack of access
5 Interview pilot study
3-15 Identify & interview about 6 interviewees per case study, these being the equivalent of chief exec or primary client, senior responsible owner, project manager, a consultant, a user, relevant politician. .
5-15 From interview data and documentation identify potential relationships in networks and send questionnaires to confirm relationships. There can be hundreds on a project, so think about how many to choose, how to choose them and why
6-15 Devise and sent questionnaires on relationships to identified participants
6-20 Transcribe interview data and input questionnaire data

Analysis
Month Activity
7-20 Analyse earlier cases while collecting data for later cases
Meet supervisors regularly

Writing
Month Activity
15-23 Write drafts for supervisors
Continue to meet supervisors regularly
26 Proof read dissertation
28 Submit

Theoretical framework

Choosing a methodology for a research project involves choosing appropriate methods, depending on what access is possible and how the researcher is thinking. The research process is informed by the researcher’s theoretical perspective, which is a philosophical stance. The theoretical perspective is itself informed by the epistemology and ontology. So the research process has four elements that inform one another: epistemology, theoretical perspective, methodology and methods (Crotty, 1998).

What the constructionist approach is
Constructionism is the idea that there is no objective truth but that we all construct our own reality, meaning is constructed and different people construct meaning in different ways. People need to have mechanisms for understanding what each of them is doing and need rich forms of conversations that are adequate to dealing with the complexity of social relationships so it’s sense making.

(Guba & Lincoln refer to the constructivist paradigm (Guba and Lincoln, 1989))

Why use the constructionist approach here
Clients and consultants can construct each others roles and clients are actors or agents who shape each others standing of what they do. Schein categorises those roles and so does Bovens.

Disadvantages of the constructionist approach
Czarniawska warns of the difficulties of employing the constructionist stance because the logic of representation is conventionally used between researchers but the medium of everyday organisational life is the logic of practice so there can be difficulties of communication between researchers and managers (Czarniawska, 2001).

---

Crotty, M. (1998) 'Introduction: the research process'. The foundations of social research: meaning and perspective in the research process, Sage, London.
Czarniawska, B. (2001) 'Is it Possible to Be a Constructionist Consultant?' Management Learning, 32 (2), pp. 253.
Guba, E. G. and Lincoln, Y. S. (1989) Fourth Generation Evaluation, Sage.