JackDaw is an AI-powered conversational assistant that enables stakeholders to explore territorial and environmental information through map-based analysis, using the platform’s available geospatial and environmental datasets for a selected area. Through J-VERSE (JackDaw Validation in Real Spatial Environments), funded under the PoliRuralPlus initiative, we set out to evaluate how the platform performs in real-world settings and how people interact with it in practice.
To do that, we put JackDaw in the hands of real users and invited them to explore, question, and challenge it. Their experiences, observations and feedback provided valuable insights into the platform's usability, strengths and areas for improvement.
Here's what happened, and what we learned.
01 WHO TOOK PART
Bringing people into the loop
We wanted a real mix of voices, so we opened the door wide. Thirty-five participants took part, joining through an onsite session at Catalink’s premises, an online session, and an open invitation shared across our networks. The group that showed up was genuinely diverse, a near-even gender balance, seven stakeholder categories spanning expert and non-expert voices, and participants across four countries (Figure 1).
Figure 1. Participant profile (n=35): gender balance, stakeholder category, prior experience with map-based tools, and geographic distribution.
02 THE METHOD
How the sessions worked
Rather than hand people a manual, we built a single guided web session (available at jverse.catalink.eu; Figure 2) that walked everyone through the same nine steps: an introduction; a short video presenting the project, with an embedded demo of how to use JackDaw; a consent form; a step to set up a JackDaw account or use our demo account; three guided scenarios; a free-exploration phase; and a final questionnaire. The three scenarios each explored a different side of the platform: land use, environmental conditions, and rural–urban patterns.
Figure 2. The welcome page of the guided validation session that led each participant through the nine-step flow.
After the structured scenarios, participants were set free to ask JackDaw whatever they liked. This phase produced a wide range of questions — for example, on the suitability of an area for building, the best time to sow winter wheat in a locality, public-transport access to a selected area, foot-and-mouth disease measures, and wildfire risk. These are a few examples among many.
03 THE RESULTS
Easy and clear — but held back by data
The headline finding is encouraging: people found JackDaw genuinely easy to use, fast, and clear. But the final questionnaire revealed a sharp, honest split. Everything about interacting with the tool scored well; everything about the perceived value of its answers sat below the neutral mid-point. One chart (Figure 3) captures the whole story.
Figure 3. Final user-experience scores (n = 35, scale 1–5). Green sits above the neutral line; clay falls below it. Response speed was not a limiting factor — 84% of participants received answers in under a minute.
Read together with the open comments, the message was clear. When relevant data existed for an area, answers were rated highly; when they didn’t, the system returned generic or “no records” responses, and ratings dropped noticeably. The gap wasn’t about how JackDaw communicates, but about the data sitting behind its answers.
Participants also flagged concrete, fixable issues, the map occasionally answering about a previously selected area, and a wish for simple controls like a button to clear the selection box. And many noticed something reassuring: JackDaw tended to know which geospatial data were relevant to a question, even when the data weren’t there to back it up. JackDaw often appeared to identify the right type of geospatial evidence for a question, even when the available datasets were not sufficient to support a strong answer.
To everyone who took part, explored, and told us honestly what worked and what didn’t, thank you. You’ve given us a sharper, more grounded sense of where JackDaw stands and where it needs to go.
Existing Comments