Source
urlhttps://doi.org/10.1016/j.tele.2025.102283
rawraw/Zhang_1-s2.0-S2949882125000283-main.pdf

TL;DR: An HCI study on how qualitative researchers perceive and use ChatGPT, combining 17 interviews with a co-design session with 13 researchers. The key finding: perception shifts from skeptical to positive when researchers gain transparency into how ChatGPT works and explicit frameworks for prompting. Also surfaces an underexplored risk — novice researchers constructing inappropriate AI ethical expectations.

Problem

Most research on AI-assisted qualitative analysis focuses on outputs: does AI produce themes that match human themes? Does it achieve acceptable kappa? Zhang et al. ask a different question: how do qualitative researchers actually perceive, understand, and use ChatGPT, and what shifts those perceptions?

This is an HCI problem as much as a methods problem. ChatGPT is accessible to researchers without technical expertise, but accessibility does not imply usability for research purposes. Researchers may have systematically incorrect mental models of what ChatGPT does — leading to misapplication, over-trust, or under-utilization. The study also raises a concern largely absent from the methods literature: novice qualitative researchers who form their ethical expectations about AI early in their careers may propagate methodologically shallow practices as they advance.

Approach

Phase 1 — Interviews. Semi-structured interviews with 17 qualitative researchers about their experience and perceptions of ChatGPT. The sample skewed toward junior researchers and non-technical stakeholders — the population most likely to adopt AI quickly and with the least methodological scaffolding.

Phase 2 — Co-design. A participatory session with 13 qualitative researchers to collaboratively develop prompt design principles. The co-design format is unusual in this literature: rather than prescribing a prompting framework, the study elicits one from practitioners, testing whether participants who actively construct knowledge about prompt design develop different attitudes than passive recipients of guidance.

The paper also examined researcher perception before and after the co-design session, creating a natural comparison point for the attitude-shift finding.

AI’s Role

AI is treated primarily as an object of study rather than an analytic tool — the paper is about how researchers relate to AI, not about using AI to do the analysis. Within the co-design output, however, AI is positioned as a research assistant under researcher direction: transparent in its reasoning, constrained in its autonomy, and used to scaffold rather than replace interpretive work.

The hallucination warning is embedded in the paper’s introduction: existing studies had shown that ChatGPT generates descriptive themes but may fabricate quotations. The paper does not dwell on this but treats it as a baseline caveat shaping the trust relationship between researcher and AI.

Epistemological Stance

Pragmatist / HCI-oriented. The paper does not engage with the qualitative epistemology literature (small q vs. Big Q, constructionism, interpretivism). Its evaluation criteria are perceptual and behavioral: do researchers trust ChatGPT more after the co-design intervention? Do the prompt frameworks they develop reflect methodologically sound principles?

This is a pragmatist stance: the test is whether the intervention works, not whether it aligns with a specific epistemological tradition. The paper would not satisfy a Big Q reviewer asking about reflexivity or researcher positionality, but it would satisfy a design-science or implementation-science reviewer asking about adoption and usability.

Rigor and Trustworthiness

The dual-phase design — interviews followed by co-design — provides complementary perspectives. Interviews reveal current perceptions and pain points; co-design tests whether structured participation shifts them. The before/after comparison (pre- and post-co-design attitudes) is a pragmatic within-study evaluation.

The sample size (17 interviews, 13 co-designers) is modest but appropriate for qualitative research. The focus on junior researchers and non-technical stakeholders is a principled scope decision — these are the users for whom the prompt-design framework is most consequential.

The paper’s transparency is relatively high for its domain: findings are illustrated with participant quotes, and the prompt framework principles are reported rather than left implicit.

Limitations

The study does not follow up on whether participants actually changed their practice after the co-design session. Attitude shift is documented; behavior change is not. The long-term effects of the intervention — whether researchers who attended the session subsequently prompted ChatGPT differently — are unknown.

The sample is drawn from researchers in the HCI/information science space at US institutions, which raises questions about generalizability to humanities, health, or social science researchers with different disciplinary norms and tool access.

The novice-risk finding — that junior researchers constructing poor AI ethical expectations creates downstream methodological problems — is the paper’s most important sociological claim, but it is the least empirically developed. It is raised as a concern and a future research direction, not a finding with evidence.

The prompt-design framework produced by the co-design session is not published in systematic detail. Readers looking for operational prompt guidance will need to look elsewhere — bijker-chatgpt-qca-2024, salazar-gpt4-qualitative-2025, or naeem-chatgpt-ta-steps-2025 provide more concrete prompting documentation.

Connections

What links here